Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: apigee/apigeelint
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v2.56.0
Choose a base ref
...
head repository: apigee/apigeelint
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref

Commits on Sep 19, 2024

  1. Copy the full SHA
    4e00e80 View commit details
  2. v2.56.1

    ssvaidyanathan committed Sep 19, 2024
    Copy the full SHA
    5715504 View commit details

Commits on Sep 23, 2024

  1. v2.56.2

    ssvaidyanathan committed Sep 23, 2024
    Copy the full SHA
    eb78e44 View commit details

Commits on Sep 27, 2024

  1. fix Xmldom resolution (#474)

    * fix: make xmldom location resolution more reliable
    * v2.56.2
    * remove trailing slash ...to allow it to work on windows
    
    ---------
    
    Co-authored-by: Sai Saran Vaidyanathan <ssvaidyanathan@google.com>
    DinoChiesa and ssvaidyanathan authored Sep 27, 2024
    Copy the full SHA
    51e2eab View commit details
  2. feat: BN014 detect duplicate policies (#468)

    * feat: BN014 duplicate policy checker
    
    * add README update
    
    * remove the code in EP001 for detecting duplicates
    DinoChiesa authored Sep 27, 2024
    Copy the full SHA
    e21e568 View commit details
  3. Copy the full SHA
    45c7c88 View commit details
  4. Copy the full SHA
    db94be1 View commit details
  5. Copy the full SHA
    0c88e64 View commit details
  6. Merge pull request #472 from DinoChiesa/target-lb-checks

    feat: TD008 and TD009 to perform checks on LoadBalancer
    ssvaidyanathan authored Sep 27, 2024
    Copy the full SHA
    0e85491 View commit details
  7. v2.57.0

    ssvaidyanathan committed Sep 27, 2024
    Copy the full SHA
    88d3b3f View commit details
  8. Copy the full SHA
    f77742d View commit details

Commits on Oct 7, 2024

  1. chore: refactor TD004 into multiple different plugins (#478)

    * chore: refactor TD004 into multiple different plugins. Now, we have
       * TD004 - Enabled + Enforce on SSLInfo
       * TD011 - IgnoreValidationErrors is not present or false
       * TD012 - exactly one SSLInfo element
       * TD013 - hygiene w.r.t. ClientAuthEnabled
       * TD014 - exactly one URL or LoadBalancer element
       * TD015 - check for MaxFailures when LoadBalancer is present
    
    * add tests for all of the above plugins
    
    * fixup EP002 test to account for new TD012 error
    DinoChiesa authored Oct 7, 2024
    Copy the full SHA
    4e64b92 View commit details

Commits on Nov 1, 2024

  1. Copy the full SHA
    9812af5 View commit details
  2. Merge pull request #484 from DinoChiesa/issue483-CC005

    fix: issue483 CC005 no longer modifies truthtable
    ssvaidyanathan authored Nov 1, 2024
    Copy the full SHA
    6cdd5f8 View commit details
  3. v2.58.0

    ssvaidyanathan committed Nov 1, 2024
    Copy the full SHA
    a3695db View commit details
  4. feat: PO038 check KeyValueMapOperations MapName and identifier (#485)

    * feat: PO038 check KeyValueMapOperations MapName and identifier
    
    * add README update
    DinoChiesa authored Nov 1, 2024
    Copy the full SHA
    c98c203 View commit details
  5. fix: BN013 correct handling of resource shortname (#486)

    * fix: BN013 correct handling of resource shortname
    
    * clean comments from test module
    DinoChiesa authored Nov 1, 2024
    Copy the full SHA
    c52ef7d View commit details

Commits on Nov 2, 2024

  1. fix: look more thoroughly for xmldom/xmldom (#487)

    * fix: look more thoroughly for xmldom/xmldom
    
    * update package.json to require node v20 or later
    
    * feat: PO038 check KeyValueMapOperations MapName and identifier (#485)
    
    * feat: PO038 check KeyValueMapOperations MapName and identifier
    
    * add README update
    
    * fix: BN013 correct handling of resource shortname (#486)
    
    * fix: BN013 correct handling of resource shortname
    
    * clean comments from test module
    
    * fix: look more thoroughly for xmldom/xmldom
    
    * update package.json to require node v20 or later
    
    * fix: replace myUtil with lintUtil.js,and cleanup xmldom test
    DinoChiesa authored Nov 2, 2024
    Copy the full SHA
    099d2b5 View commit details

Commits on Nov 4, 2024

  1. Copy the full SHA
    ce945e8 View commit details
  2. v2.59.0

    ssvaidyanathan committed Nov 4, 2024
    Copy the full SHA
    896eb5e View commit details

Commits on Nov 14, 2024

  1. Copy the full SHA
    b84e88d View commit details
  2. Merge pull request #489 from DinoChiesa/issue479

    feat: TD016 check that HealthMonitor present only when applicable
    ssvaidyanathan authored Nov 14, 2024
    Copy the full SHA
    12e2bdb View commit details

Commits on Dec 6, 2024

  1. Copy the full SHA
    1693a77 View commit details
  2. Copy the full SHA
    dbf2524 View commit details
  3. Copy the full SHA
    6047b6d View commit details
  4. Copy the full SHA
    cc6c320 View commit details
  5. Copy the full SHA
    8bd4ce8 View commit details

Commits on Dec 7, 2024

  1. Merge pull request #490 from DinoChiesa/ml-resourcetype

    feat: P039 to scan MessageLogging/CloudLogging
    ssvaidyanathan authored Dec 7, 2024
    Copy the full SHA
    914ccf9 View commit details
  2. Merge pull request #491 from DinoChiesa/handle-zip

    feat: handle ingest of zipped bundles
    ssvaidyanathan authored Dec 7, 2024
    Copy the full SHA
    9a06125 View commit details
  3. Merge pull request #492 from DinoChiesa/policies-after-raisefault

    feat: ST008 check for policies attached after RaiseFault
    ssvaidyanathan authored Dec 7, 2024
    Copy the full SHA
    26b6db6 View commit details
  4. Copy the full SHA
    45ef95a View commit details
  5. Merge pull request #493 from DinoChiesa/po039-test-fixup

    fix: correct errant tests for PO039
    ssvaidyanathan authored Dec 7, 2024
    Copy the full SHA
    1c21957 View commit details
  6. v2.60.0

    ssvaidyanathan committed Dec 7, 2024
    Copy the full SHA
    9f04192 View commit details

Commits on Dec 19, 2024

  1. Copy the full SHA
    1d602c4 View commit details
  2. Copy the full SHA
    26bb956 View commit details
  3. Copy the full SHA
    b736fb8 View commit details
  4. Copy the full SHA
    98b05de View commit details
  5. Merge pull request #494 from DinoChiesa/code-cleanup

    chore: code cleanup. add new test for PO039.
    ssvaidyanathan authored Dec 19, 2024
    Copy the full SHA
    5ce7ae9 View commit details
  6. Merge pull request #495 from DinoChiesa/download-feature

    implement the download feature for the cli
    ssvaidyanathan authored Dec 19, 2024
    Copy the full SHA
    ebde757 View commit details

Commits on Dec 29, 2024

  1. v2.61.0

    ssvaidyanathan committed Dec 29, 2024
    Copy the full SHA
    0f8434d View commit details

Commits on Jan 4, 2025

  1. fix: minor typo in README

    DinoChiesa committed Jan 4, 2025
    Copy the full SHA
    276dd05 View commit details

Commits on Jan 10, 2025

  1. Update downloader.js fixes Issue #497

    Fixes Issue #497, sorts numerically
    kurtkanaskie authored Jan 10, 2025
    Copy the full SHA
    fee76c4 View commit details
  2. Merge pull request #498 from kurtkanaskie/Issue497

    Update downloader.js fixes Issue #497
    ssvaidyanathan authored Jan 10, 2025
    Copy the full SHA
    7258a0c View commit details
  3. Merge pull request #496 from DinoChiesa/fix-readme-1

    fix: minor typo in README
    ssvaidyanathan authored Jan 10, 2025
    Copy the full SHA
    5c73ac2 View commit details
Showing 331 changed files with 11,582 additions and 3,314 deletions.
105 changes: 105 additions & 0 deletions PO039-test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
/*
Copyright 2019-2024 Google LLC
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

const testID = "PO039",
assert = require("assert"),
fs = require("fs"),
path = require("path"),
bl = require("../../lib/package/bundleLinter.js"),
plugin = require(bl.resolvePlugin(testID)),
debug = require("debug")("apigeelint:" + testID),
Policy = require("../../lib/package/Policy.js"),
Dom = require("@xmldom/xmldom").DOMParser,
rootDir = path.resolve(__dirname, "../fixtures/resources/PO039");

const test = (suffix, cb) => {
const filename = `ML-test-${suffix}.xml`;
it(`should correctly process ${filename}`, () => {
const fqfname = path.join(rootDir, filename),
policyXml = fs.readFileSync(fqfname, "utf-8"),
doc = new Dom().parseFromString(policyXml),
p = new Policy(rootDir, filename, this, doc);

p.getElement = () => doc.documentElement;

//plugin.onBundle({ profile: "apigee" });

plugin.onPolicy(p, (e, foundIssues) => {
assert.equal(e, undefined, "should be undefined");
cb(p, foundIssues);
});
});
};

describe(`${testID} - MessageLogging RessourceType element`, () => {
// test all the valid cases
fs.readdirSync(rootDir)
.map((shortFileName) => {
let m = shortFileName.match("^.+-(valid.+)\\.xml$");
if (m) {
return m[1];
}
})
.filter((suffix) => suffix)
.forEach((suffix) => {
test(suffix, (p, foundIssues) => {
const messages = p.getReport().messages;
assert.ok(messages, "messages undefined");
debug(messages);
assert.equal(foundIssues, false);
});
});

test("invalid1", (p, foundIssues) => {
assert.equal(foundIssues, true);
const messages = p.getReport().messages;
assert.ok(messages, "messages undefined");
debug(messages);
assert.equal(messages.length, 1, "unexpected number of messages");
assert.ok(messages[0].message, "did not find message 0");
assert.equal(
messages[0].message,
"The value 'gce_instance' should not be used here. ResourceType should be 'api'",
);
});

test("invalid2", (p, foundIssues) => {
assert.equal(foundIssues, true);
const messages = p.getReport().messages;
assert.ok(messages, "messages undefined");
debug(messages);
assert.equal(messages.length, 1, "unexpected number of messages");
assert.ok(messages[0].message, "did not find message 0");
assert.equal(
messages[0].message,
"The value 'apigee.googleapis.com/Environment' should not be used here. ResourceType should be 'api'",
);
});

test("invalid3", (p, foundIssues) => {
assert.equal(foundIssues, true);
const messages = p.getReport().messages;
assert.ok(messages, "messages undefined");
debug(messages);
assert.equal(messages.length, 2, "unexpected number of messages");
assert.ok(messages[0].message, "did not find message 0");
assert.equal(messages[0].message, "Unsupported element 'NotKey'");
assert.equal(
messages[1].message,
"Label is missing a required Element: Key.",
);
});
});
120 changes: 102 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -14,31 +14,25 @@ This utility is intended to capture the best practices knowledge from across Api

## Status

At this point, we are focused on plugin execution and modelling the various lintable assets including Bundles, Proxies, SharedFlows, Targets, Flows, Steps, and Policies.
This tool is mature and stable, works with proxy and sharedflow bundles, and
continues to get enhancements. There are a variety of plugins that test
Bundles, Policies, ProxyEndpoints, and more.

Plugins that test these abstractions are being developed concurrently.

Reporters (the means to report out results), Ingesters (bundle loaders) are to be developed with Filesystem being the only supported means of loading a bundle and all reporting now going to console.
The tool can report results out to the console, or to a file. The tool can
ingest from an exploded directory, or from a zipped bundle.

## Installation

You can install apigeellint using npm. But, there is a minimum version of `npm` required.

1. First verify the version of npm:
1. First verify the version of node and npm:
```
npm --version
node --version
```

If the version is 8.3.0 or later, then proceed to step 2.
If the version is less than `8.3.0`, then update:
```
npm install npm@8.3.0 -g
```

Alternatively, you may choose to get the latest npm:
```
npm install npm@latest -g
```
If the npm version is 10.5.0 or later, and node version is 20 or later, then proceed to step 2.
Otherwise, you need to update npm and/or node.

2. Then install apigeelint:
```
@@ -54,7 +48,8 @@ Usage: apigeelint [options]
Options:
-V, --version output the version number
-s, --path <path> Path of the proxies
-s, --path <path> Path of the proxy to analyze
-d, --download [value] Download the API proxy or sharedflow to analyze. Exclusive of -s / --path. Example: org:ORG,api:PROXYNAME or org:ORG,sf:SHAREDFLOWNAME
-f, --formatter [value] Specify formatters (default: json.js)
-w, --write [value] file path to write results
-e, --excluded [value] The comma separated list of tests to exclude (default: none)
@@ -72,12 +67,88 @@ Example:
apigeelint -s sampleProxy/apiproxy -f table.js
```

Where `-s` points to the apiProxy source directory and `-f` is the output formatter desired.
Where `-s` points to the apiProxy source directory or bundled zip file, and `-f` is the output
formatter desired.

Possible formatters are: "json.js" (the default), "stylish.js", "compact.js", "codeframe.js", "codeclimate.js", "html.js", "table.js", "unix.js", "visualstudio.js", "checkstyle.js", "jslint-xml.js", "junit.js" and "tap.js".

## Examples

### Basic usage: ingest from a directory
```
apigeelint -f table.js -s path/to/your/apiproxy
```

The path here should be a directory name, probably ending in "apiproxy". The
contents of that directory should be like this:

```
apiproxy/
apiproxy/proxies/
apiproxy/proxies/endpoint1.xml
apiproxy/servicecallout-async-test.xml
apiproxy/resources/
apiproxy/resources/jsc/
apiproxy/resources/jsc/...
apiproxy/policies/
apiproxy/policies/RF-Unknown-Request.xml
apiproxy/policies/AM-Response.xml
apiproxy/policies/...
...
```


### Basic usage: ingest from a zipped proxy bundle

You can export API Proxy or Sharedflow bundles from Apigee, producing a zip
archive. This tool also can read and analyze these zipped bundles:

```
apigeelint -f table.js -s path/to/your/apiproxy.zip
```

The tool will unzip the bundle into a temporary directory, perform the analysis,
and then remove the temporary directory.


### Basic usage: downloading a proxy bundle to analyze

You can ask apigeelint to export an API Proxy or Sharedflow bundle from Apigee,
and analyze the resulting zip archive. This connects to apigee.googleapis.com to
perform the export, which means it will work only with Apigee X or hybrid.

```
# to download and then analyze a proxy bundle
apigeelint -f table.js -d org:your-org-name,api:name-of-your-api-proxy
# to download and then analyze a sharedflow bundle
apigeelint -f table.js -d org:your-org-name,sf:name-of-your-shared-flow
```

With this invocation, the tool will:
- obtain a token using the `gcloud auth print-access-token` command
- use the token to inquire the latest revision of the proxy or sharedflow
- use the token to download the bundle for the latest revision
- unzip the bundle into a temporary directory
- perform the lint analysis
- render the result
- and then remove the temporary directory

If you do not have the [`gcloud` command line
tool](https://cloud.google.com/sdk/gcloud) installed, and available on your
path, this will fail.


You can also specify a token you have obtained previously:

```
apigeelint -f table.js -d org:your-org-name,api:name-of-your-api-proxy,token:ACCESS_TOKEN_HERE
```

In this case, apigeelint does not try to use `gcloud` to obtain an access token.



### Using External Plugins:
```
apigeelint -x ./externalPlugins -s path/to/your/apiproxy -f table.js
@@ -320,6 +391,7 @@ This is the current list:
| &nbsp; |:white_check_mark:| BN011 | Check each XML file for well-formedness.|
| &nbsp; |:white_check_mark:| BN012 | unreferrenced Target Endpoints | Check that each TargetEndpoint can be reached. |
| &nbsp; |:white_check_mark:| BN013 | Unreferenced resources. | Warn for resources that not referenced in any policy. Unreferenced resources are dead code. |
| &nbsp; |:white_check_mark:| BN014 | Duplicate policies. | Warn if there are identically configured, if differently named, policies. |
| Proxy Definition | &nbsp; | &nbsp; | &nbsp; | &nbsp; |
| &nbsp; |:white_check_mark:| PD001 | RouteRules to Targets | RouteRules should map to defined Targets. |
| &nbsp; |:white_check_mark:| PD002 | Unreachable Route Rules - defaults | Only one RouteRule should be present without a condition. |
@@ -330,10 +402,19 @@ This is the current list:
| &nbsp; |:white_check_mark:| TD001 | Mgmt Server as Target | Discourage calls to the Management Server from a Proxy via target. |
| &nbsp; |:white_check_mark:| TD002 | Use Target Servers | Encourage the use of target servers. |
| &nbsp; |:white_check_mark:| TD003 | TargetEndpoint name | TargetEndpoint name should match basename of filename. |
| &nbsp; |:white_check_mark:| TD004 | TargetEndpoint SSLInfo | TargetEndpoint HTTPTargetConnection should enable TLS/SSL. |
| &nbsp; |:white_check_mark:| TD004 | TargetEndpoint SSLInfo | TargetEndpoint HTTPTargetConnection should enable and Enforce TLS/SSL. |
| &nbsp; |:white_check_mark:| TD005 | TargetEndpoint SSLInfo references | TargetEndpoint SSLInfo should use references for KeyStore and TrustStore. |
| &nbsp; |:white_check_mark:| TD006 | TargetEndpoint SSLInfo | When using a LoadBalancer, the SSLInfo should not be configured under HTTPTargetConnection. |
| &nbsp; |:white_check_mark:| TD007 | TargetEndpoint SSLInfo | TargetEndpoint HTTPTargetConnection SSLInfo should use TrustStore. |
| &nbsp; |:white_check_mark:| TD008 | TargetEndpoint LoadBalancer Servers | LoadBalancer should not have multiple IsFallback Server entries. |
| &nbsp; |:white_check_mark:| TD009 | TargetEndpoint LoadBalancer | TargetEndpoint HTTPTargetConnection should have at most one LoadBalancer. |
| &nbsp; |:white_check_mark:| TD010 | TargetEndpoint LoadBalancer Servers | LoadBalancer should have at least one Server entry, and no duplicate Server entries. |
| &nbsp; |:white_check_mark:| TD011 | TargetEndpoint SSLInfo | TargetEndpoint HTTPTargetConnection SSLInfo should not Ignore validation errors. |
| &nbsp; |:white_check_mark:| TD012 | TargetEndpoint SSLInfo | TargetEndpoint HTTPTargetConnection should have exactly one SSLInfo. |
| &nbsp; |:white_check_mark:| TD013 | TargetEndpoint SSLInfo | TargetEndpoint HTTPTargetConnection should correctly configure ClientAuthEnbled. |
| &nbsp; |:white_check_mark:| TD014 | TargetEndpoint SSLInfo | TargetEndpoint HTTPTargetConnection should use exctly one of URL, LoadBalancer. |
| &nbsp; |:white_check_mark:| TD015 | TargetEndpoint LoadBalancer | if TargetEndpoint HTTPTargetConnection uses a LoadBalancer, it should specify MaxFailures. |
| &nbsp; |:white_check_mark:| TD016 | TargetEndpoint HealthMonitor | TargetEndpoint HTTPTargetConnection must use a HealthMonitor only with a LoadBalancer. |
| Flow | &nbsp; | &nbsp; | &nbsp; | &nbsp; |
| &nbsp; |:white_check_mark:| FL001 | Unconditional Flows | Only one unconditional flow will get executed. Error if more than one was detected. |
| Step | &nbsp; | &nbsp; | &nbsp; | &nbsp; |
@@ -344,6 +425,7 @@ This is the current list:
| &nbsp; |:white_check_mark:| ST005 | Extract Variables Step with FormParam | A check for message content should be performed before policy execution. |
| &nbsp; |:white_check_mark:| ST006 | JSON Threat Protection Step | A check for message content should be performed before policy execution. |
| &nbsp; |:white_check_mark:| ST007 | XML Threat Protection Step | A check for message content should be performed before policy execution. |
| &nbsp; |:white_check_mark:| ST008 | Unreachable policies | Policies should not be attached after RaiseFault policies. |
| Policy | &nbsp; | &nbsp; | &nbsp; | &nbsp; |
| &nbsp; |:white_check_mark:| PO006 | Policy Name &amp; filename agreement | Policy name attribute should coincide with the policy filename. |
| &nbsp; |:white_check_mark:| PO007 | Policy Naming Conventions - type indication | It is recommended that the policy name use a prefix or follow a pattern that indicates the policy type. |
@@ -377,6 +459,8 @@ This is the current list:
| &nbsp; |:white_check_mark:| PO035 | Quota policy hygiene | In a Quota policy, check element placement and other hygiene. |
| &nbsp; |:white_check_mark:| PO036 | ServiceCallout Response element usage | The Response element, when present, should specify a text value and no attributes. |
| &nbsp; |:white_check_mark:| PO037 | DataCapture policy hygiene | Checks that a Capture should uses a Source of type request when the policy is attached to the Response flow, and other checks. |
| &nbsp; |:white_check_mark:| PO038 | KeyValueMapOperations policy hygiene | Checks that MapName or mapIdentifier is specified, and not both.|
| &nbsp; |:white_check_mark:| PO039 | MessageLogging policy hygiene | Checks that ResourceType is not used, or is "api".|
| FaultRules | &nbsp; | &nbsp; | &nbsp; | &nbsp; |
| &nbsp; |:white_check_mark:| FR001 | No Condition on FaultRule | Use Condition elements on FaultRules, unless it is the fallback rule. |
| &nbsp; |:white_check_mark:| FR002 | DefaultFaultRule Structure | DefaultFaultRule should have only supported child elements, at most one AlwaysEnforce element, and at most one Condition element. |
Loading