Exhort JavaScript API

• Looking for our Java API? Try Exhort Java API.
• Looking for our Backend implementation? Try Exhort.

The Exhort JavaScript API module is deployed to GitHub Package Registry.

Click here for configuring GHPR registry access.

Configure Registry Access

Create a token with the read:packages scope

Based on GitHub documentation, In Actions you can use GITHUB_TOKEN

Add the following line to the .npmrc file in your user home ( See GH Docs):

//npm.pkg.github.com/:_authToken=<your-ghp-token-goes-here>

Usage

Configuring NPM to look in GHPR for the RHEcosystemAppEng namespace is done by adding @RHEcosystemAppEng:registry=https://npm.pkg.github.com to .npmrc in the project root or user home.

echo "@RHEcosystemAppEng:registry=https://npm.pkg.github.com" >> .npmrc

• Use as ESM Module from an ESM module

  npm install @RHEcosystemAppEng/exhort-javascript-api

  import exhort from '@RHEcosystemAppEng/exhort-javascript-api'
  import fs from 'node:fs'
  
  // Get stack analysis in JSON format
  let stackAnalysis = await exhort.stackAnalysis('/path/to/pom.xml')
  // Get stack analysis in HTML format (string)
  let stackAnalysisHtml = await exhort.stackAnalysis('/path/to/pom.xml', true)
  
  // Get component analysis in JSON format
  let buffer = fs.readFileSync('/path/to/pom.xml')
  let componentAnalysis = await exhort.componentAnalysis('pom.xml', buffer.toString())
  
  // Get Component Analysis in JSON Format for gradle
  let caGradle = await exhort.componentAnalysis("build.gradle","",{},"path/to/build.gradle")

• Use as ESM Module from Common-JS module

  npm install @RHEcosystemAppEng/exhort-javascript-api

  async function loadExhort()
  {
  // dynamic import is the only way to import ESM module into commonJS module
    const { default: exhort } = await import('@RHEcosystemAppEng/exhort-javascript-api');
    return exhort
  }
  const runExhort = (manifestPath) => {
    return new Promise(async ( resolve, reject) => {
      try {
        let stackAnalysisReport = await (await loadExhort()).stackAnalysis(manifestPath,false)
        resolve(stackAnalysisReport)
  
      } catch (error)
      {
        reject(error)
      }
    });
  };
  
  runExhort("./path/to/manifest").then(resp => console.log(JSON.stringify(resp,null,4)))

• Use as CLI Script
  Click for help menu

  $ npx @RHEcosystemAppEng/exhort-javascript-api help
  
  Usage: exhort-javascript-api {component|stack}
  
  Commands:
    exhort-javascript-api stack </path/to/manifest> [--html|--summary]               produce stack report for manifest path
    exhort-javascript-api component <manifest-name> <manifest-content> [--summary]   produce component report for a manifest type and content
  
  Options:
    --help  Show help                                                    [boolean]

  # get stack analysis in json format
  $ npx @RHEcosystemAppEng/exhort-javascript-api stack /path/to/pom.xml
  
  # get stack analysis in json format (summary only)
  $ npx @RHEcosystemAppEng/exhort-javascript-api stack /path/to/pom.xml --summary
  
  # get stack analysis in html format format
  $ npx @RHEcosystemAppEng/exhort-javascript-api stack /path/to/pom.xml --html
  
  # get component analysis
  $ npx @RHEcosystemAppEng/exhort-javascript-api component pom.xml "$(</path/to/pom.xml)"

• Use as Global Binary

  npm install --global @RHEcosystemAppEng/exhort-javascript-api

  # get stack analysis in json format
  $ exhort-javascript-api stack /path/to/pom.xml
  
  # get stack analysis in json format (summary only)
  $ exhort-javascript-api stack /path/to/pom.xml --summary
  
  # get stack analysis in html format format
  $ exhort-javascript-api stack /path/to/pom.xml --html
  
  # get component analysis
  $ exhort-javascript-api component pom.xml "$(</path/to/pom.xml)"

Supported Ecosystems

• Java - Maven
• JavaScript - Npm
• Golang - Go Modules
• Python - pip Installer
• Gradle - Gradle Installation

Excluding Packages

Excluding a package from any analysis can be achieved by marking the package for exclusion.

• Java Maven users can add a comment in pom.xml

  <dependency> <!--exhortignore-->
    <groupId>...</groupId>
    <artifactId>...</artifactId>
    <version>...</version>
  </dependency>

• Javascript NPM users can add a root (key, value) pair with value of list of names (strings) to be ignored (without versions), and key called exhortignore in package.json, example:

  {
    "name": "sample",
    "version": "1.0.0",
    "description": "",
    "main": "js",
    "keywords": [],
    "author": "",
    "license": "ISC",
    "dependencies": {
      "dotenv": "^8.2.0",
      "express": "^4.17.1",
      "jsonwebtoken": "^8.5.1",
      "mongoose": "^5.9.18"
    },
    "exhortignore": [
      "jsonwebtoken"
    ]
  }

  Golang users can add in go.mod a comment with //exhortignore next to the package to be ignored, or to "piggyback" on existing comment ( e.g - //indirect) , for example:

  module github.com/RHEcosystemAppEng/SaaSi/deployer
  
  go 1.19
  
  require (
          github.com/gin-gonic/gin v1.9.1
          github.com/google/uuid v1.1.2
          github.com/jessevdk/go-flags v1.5.0 //exhortignore
          github.com/kr/pretty v0.3.1
          gopkg.in/yaml.v2 v2.4.0
          k8s.io/apimachinery v0.26.1
          k8s.io/client-go v0.26.1
  )
  
  require (
          github.com/davecgh/go-spew v1.1.1 // indirect exhortignore
          github.com/emicklei/go-restful/v3 v3.9.0 // indirect
          github.com/go-logr/logr v1.2.3 // indirect //exhortignore
  
  )

  Python pip users can add in requirements.txt a comment with #exhortignore(or # exhortignore) to the right of the same artifact to be ignored, for example:

  anyio==3.6.2
  asgiref==3.4.1
  beautifulsoup4==4.12.2
  certifi==2023.7.22
  chardet==4.0.0
  click==8.0.4 #exhortignore
  contextlib2==21.6.0
  fastapi==0.75.1
  Flask==2.0.3
  h11==0.13.0
  idna==2.10
  immutables==0.19
  importlib-metadata==4.8.3
  itsdangerous==2.0.1
  Jinja2==3.0.3
  MarkupSafe==2.0.1
  pydantic==1.9.2 # exhortignore
  requests==2.25.1
  six==1.16.0
  sniffio==1.2.0
  soupsieve==2.3.2.post1
  starlette==0.17.1
  typing_extensions==4.1.1
  urllib3==1.26.16
  uvicorn==0.17.0
  Werkzeug==2.0.3
  zipp==3.6.0
  

  Gradle users can add in build.gradle a comment with //exhortignore next to the package to be ignored:

  plugins {
  id 'java'
  }
  
  group = 'groupName'
  version = 'version'
  
  repositories {
      mavenCentral()
  }
  
  dependencies {
      implementation "groupId:artifactId:version" // exhortignore
  }
  test {
      useJUnitPlatform()
  }

  All of the 5 above examples are valid for marking a package to be ignored

Customization

There are 2 approaches for customizing Exhort JavaScript API. Whether you're using this API as a Global Module, a Remote Script, or an ESM Module, you can use Environment Variables for various customization.

However, ESM Module users, can opt for customizing programmatically:

import exhort from '@RHEcosystemAppEng/exhort-javascript-api'
import fs from 'node:fs'

let options = {
  'EXHORT_MVN_PATH': '/path/to/my/mvn',
  'EXHORT_NPM_PATH': '/path/to/npm',
  'EXHORT_GO_PATH': '/path/to/go',
  //python - python3, pip3 take precedence if python version > 3 installed
  'EXHORT_PYTHON3_PATH' : '/path/to/python3',
  'EXHORT_PIP3_PATH' : '/path/to/pip3',
  'EXHORT_PYTHON_PATH' : '/path/to/python',
  'EXHORT_PIP_PATH' : '/path/to/pip',
  'EXHORT_GRADLE_PATH' : '/path/to/gradle'

}

// Get stack analysis in JSON format ( all package managers, pom.xml is as an example here)
let stackAnalysis = await exhort.stackAnalysis('/path/to/pom.xml', false, options)
// Get stack analysis in HTML format in string ( all package managers, pom.xml is as an example here)
let stackAnalysisHtml = await exhort.stackAnalysis('/path/to/pom.xml', true, options)

// Get component analysis in JSON format
let buffer = fs.readFileSync('/path/to/pom.xml')
let componentAnalysis = await exhort.componentAnalysis('pom.xml', buffer.toString(), options)

// Get component analysis in JSON format For gradle
let caGradle = await exhort.componentAnalysis("build.gradle","",{},"path/to/build.gradle")

Environment variables takes precedence.

Customizing Executables

This project uses each ecosystem's executable for creating dependency trees. These executables are expected to be present on the system's PATH environment. If they are not, or perhaps you want to use custom ones. Use can use the following keys for setting custom paths for the said executables.

Ecosystem	Default	Executable Key
Maven	mvn	EXHORT_MVN_PATH
NPM	npm	EXHORT_NPM_PATH
Go Modules	go	EXHORT_GO_PATH
Python programming language	python3	EXHORT_PYTHON3_PATH
Python pip Package Installer	pip3	EXHORT_PIP3_PATH
Python programming language	python	EXHORT_PYTHON_PATH
Python pip Package Installer	pip	EXHORT_PIP_PATH
Gradle	gradle	EXHORT_GRADLE_PATH

Match Manifest Versions Feature

Background

In Python pip and in golang go modules package managers ( especially in Python pip) , There is a big chance that for a certain manifest and a given package inside it, the client machine environment has different version installed/resolved for that package, which can lead to perform the analysis on the installed packages' versions , instead on the declared versions ( in manifests - that is requirements.txt/go.mod ), and this can cause a confusion for the user in the client consuming the API and leads to inconsistent output ( in THE manifest there is version X For a given Package A , and in the analysis report there is another version for the same package A - Y).

Usage

To eliminate confusion and improve clarity as discussed above, the following setting was introduced - MATCH_MANIFEST_VERSIONS, in the form of environment variable/key in opts ( as usual , environment variable takes precedence ) for two ecosystems:

• Golang - Go Modules
• Python - pip

Two possible values for this setting:

1. MATCH_MANIFEST_VERSIONS="false" - means that if installed/resolved versions of packages are different than the ones declared in the manifest, the process will ignore this difference and will continue to analysis with installed/resolved versions ( this is the original logic flow )

2. MATCH_MANIFEST_VERSIONS="true" - means that before starting the analysis, the api will compare all the versions of packages in manifest against installed/resolved versions on client' environment, in case there is a difference, it will throw an error to the client/user with message containing the first encountered versions mismatch, including package name, and the versions difference, and will suggest to set setting MATCH_MANIFEST_VERSIONS="false" to ignore all differences

Golang Support

By default, all go.mod' packages' transitive modules will be taken to analysis with their original package version, that is, if go.mod has 2 modules, a and b, and each one of them has the same package c with same major version v1, but different minor versions:

• namespace/c/[email protected]
• namespace/c/[email protected]

Then both of these packages will be entered to the generated sbom and will be included in analysis returned to client. In golang, in an actual build of an application into an actual application executable binary, only one of the minor versions will be included in the executable, as only packages with same name but different major versions considered different packages , hence can co-exist together in the application executable.

Go ecosystem knows how to select one minor version among all the minor versions of the same major version of a given package, using the MVS Algorithm.

In order to enable this behavior, that only shows in analysis modules versions that are actually built into the application executable, please set system property/environment variable - EXHORT_GO_MVS_LOGIC_ENABLED=true(Default is false)

Python Support

By default, For python support, the api assumes that the package is installed using the pip/pip3 binary on the system PATH, or using the customized Binaries passed to environment variables. In any case, If the package is not installed , then an error will be thrown.

There is an experimental feature of installing the requirements.txt on a virtual env(only python3 or later is supported for this feature) - in this case, it's important to pass in a path to python3 binary as EXHORT_PYTHON3_PATH or instead make sure that python3 is on the system path. in such case, You can use that feature by setting environment variable EXHORT_PYTHON_VIRTUAL_ENV to true.

"Best Efforts Installation"

Since Python pip packages are very sensitive/picky regarding python version changes( every small range of versions is only tailored for a certain python version), I'm introducing this feature, that tries to install all packages in requirements.txt onto created virtual environment while disregarding versions declared for packages in requirements.txt. This increasing the chances and the probability that the automatic installation will succeed.

Usage

A New setting is introduced - EXHORT_PYTHON_INSTALL_BEST_EFFORTS (as both env variable/key in options object)

1. EXHORT_PYTHON_INSTALL_BEST_EFFORTS="false" - install requirements.txt while respecting declared versions for all packages.
2. EXHORT_PYTHON_INSTALL_BEST_EFFORTS="true" - install all packages from requirements.txt, not respecting the declared version, but trying to install a version tailored for the used python version, when using this setting,you must set setting MATCH_MANIFEST_VERSIONS="false"

Using pipdeptree

By Default, The API algorithm will use native commands of PIP installer as data source to build the dependency tree. It's also possible, to use lightweight Python PIP utility pipdeptree as data source instead, in order to activate this, Need to set environment variable/option - EXHORT_PIP_USE_DEP_TREE to true.

Known Issues

• For pip requirements.txt - It's been observed that for python versions 3.11.x, there might be slowness for invoking the analysis. If you encounter a performance issue with version >= 3.11.x, kindly try to set environment variable/option EXHORT_PIP_USE_DEP_TREE=true, before calling the analysis.

• For maven pom.xml, it has been noticed that using java 17 might cause stack analysis to hang forever. This is caused by maven dependency Plugin bug when running with JDK/JRE' JVM version 17.

  To overcome this, you can use any other java version (14,20,21, etc..). ( best way is to install JDK/JRE version different from 17 , and set the location of the installation in environment variable JAVA_HOME so maven will use it.)