From d54cf495c6159a7fd0a193b41bc659affd61bc48 Mon Sep 17 00:00:00 2001 From: Nikita Wootten Date: Thu, 2 Mar 2023 21:09:34 -0500 Subject: [PATCH] Profile resolution how-to readmes (#1666) * Profile resolution how-to readmes Fixes #1178 * Delete old `resolver-2018` folder * Update resolver-pipeline readme.md with more generic instructions on running the pipeline Co-authored-by: A.J. Stein * Remove notice to only use Saxon 10 in line with previous feedback * Added instructions for the wrapper script --------- Co-authored-by: A.J. Stein --- src/utils/util/resolver-pipeline/README.md | 66 ++ src/utils/util/resolver-pipeline/TESTING.md | 7 + .../oscal-profile-resolve.sh | 37 + src/utils/util/resolver-pipeline/readme.md | 49 -- .../resolver-2018/profile-resolver.xsl | 656 ------------------ .../resolver-2018/resolution-notes.md | 146 ---- 6 files changed, 110 insertions(+), 851 deletions(-) create mode 100644 src/utils/util/resolver-pipeline/README.md create mode 100644 src/utils/util/resolver-pipeline/TESTING.md create mode 100755 src/utils/util/resolver-pipeline/oscal-profile-resolve.sh delete mode 100644 src/utils/util/resolver-pipeline/readme.md delete mode 100644 src/utils/util/resolver-pipeline/resolver-2018/profile-resolver.xsl delete mode 100644 src/utils/util/resolver-pipeline/resolver-2018/resolution-notes.md diff --git a/src/utils/util/resolver-pipeline/README.md b/src/utils/util/resolver-pipeline/README.md new file mode 100644 index 0000000000..48e7dff106 --- /dev/null +++ b/src/utils/util/resolver-pipeline/README.md @@ -0,0 +1,66 @@ +# Profile Resolver Pipeline + +Profile resolution is implemented here as a sequence of XSLT transformations. The sequence applies to defined inputs (a **source profile** with imported **catalog** sources) and produces defined outputs (a **profile resolution result** in the form of a catalog). The word **baseline** also refers to a particular profile in application, whether in its unprocessed form or its resolved, serialized form. + +The sequence of XSLT transformations reflects and roughly corresponds to the steps in profile resolution described for OSCAL in the [Profile Resolution Specification](https://pages.nist.gov/OSCAL/concepts/processing/profile-resolution/): + +- **selection**: importing catalogs or profiles, and selecting controls from them + +- **organization (merging)**: organizing the selected controls for the output representation + +- **modification**: setting parameters and potentially supplementing, amending or editing control text + +## Usage + +The entry point for the transformation pipeline is `oscal-profile-RESOLVE.xsl`, which invokes the transformation steps in sequence, taking the source profile document as primary input. + +The following additional parameters can be used to modify the transformation pipeline's behavior: + +| Name | Description | Default | +| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- | +| `hide-source-profile-uri` | If `true`, the output catalog's metadata does not record the source profile's URI. | `false` | +| `path-to-source` | Path from output catalog to location of source profile. | None | +| `top-uuid` | UUID value for top-level element in output catalog, if `uuid-method` is `user-provided`. | None | +| `uuid-method` | Method for computing UUID of top-level element in output catalog. Valid values are: `random-xslt`, in which case the `random-number-generator` XPath function must be available; `random-java`, in which case the `java.util.UUID.randomUUID()` Java method must be available; `user-provided`, in which case you must specify the `top-uuid` parameter; `web-service`, referring to the `uuid-service` parameter value; and `fixed`. | `fixed` | +| `uuid-service` | URI for a web service that produces a UUID, if `uuid-method` is `web-service`. | `https://www.uuidgenerator.net/api/version4` | +| `trace` | If set to `on`, additional debug information will be printed to the console and `opr:ERROR`, `opr:WARNING` messages will be retained in the output document. | None | + +Note that URIs (addresses) given in a profile document must link correctly as absolute or relative paths to their imported catalogs, as demonstrated in [examples](../../../specifications/profile-resolution/profile-resolution-examples). + +A serialized output of profile resolution takes the form of an OSCAL catalog. Assuming inputs are correctly formed, the output is valid to the catalog schema. + +## Invoking the Pipeline + +The profile resolver and other utilities in this repo rely on open-source libraries and utilities developed by third parties. To review the software version used by this project, please review [the `build` directory for manifests of software and the specific versions](../../../../build) we recommend for running utilities from this repo, specifically the recommended [version of Saxon XSLT processor defined in the pom.xml manifest](../../../../build/pom.xml). + +### Using Saxon Manually From the Command Line + +Before you begin: + +- Install the Java runtime using your preferred method. +- Download a compatible version of [Saxon Java Home Edition](https://www.saxonica.com/download/java.xml) and extract the `saxon-he` jar file. + +Invoke Saxon with your document and the profile resolution stylesheet as follows: + +``` +$ java -jar /path/to/saxon-he-10.x.jar -t -s:YOUR_PROFILE_DOCUMENT.xml -xsl:path/to/oscal-profile-RESOLVE.xsl -o:YOUR_RESULT_BASELINE.xml +``` + +Saxon allows users to specify additional parameters (such as the ones specified [above](#usage)) by adding arguments with the format `name=value` to the end of the above command. As an example, see the command: + +```bash +$ java -jar /path/to/saxon-he-10.x.jar -t -s:YOUR_PROFILE_DOCUMENT.xml -xsl:path/to/oscal-profile-RESOLVE.xsl -o:YOUR_RESULT_BASELINE.xml uuid-method=random-xslt hide-source-profile-uri=true +``` + +### Using the `oscal-profile-resolve.sh` Wrapper Script + +The [`oscal-profile-resolve.sh`](./oscal-profile-resolve.sh) wrapper script only requires [Maven](https://maven.apache.org/) to be installed. +It manages external dependencies, and can be invoked from other directories. + +The syntax for invoking the wrapper script is as follows: + +```bash +$ ./oscal-profile-resolve.sh YOUR_PROFILE_DOCUMENT.xml YOUR_RESULT_BASELINE.xml [additional arguments] +``` + +Additional arguments are passed into Saxon verbatim and should be in the same `name=value` format described [above](#using-saxon-manually-from-the-command-line). diff --git a/src/utils/util/resolver-pipeline/TESTING.md b/src/utils/util/resolver-pipeline/TESTING.md new file mode 100644 index 0000000000..91b62496d9 --- /dev/null +++ b/src/utils/util/resolver-pipeline/TESTING.md @@ -0,0 +1,7 @@ +# Profile Resolver Pipeline Tests + +The `testing/*` folders contain XSpec tests that indicate expected interim results of each XSLT transformation in the sequence. + +Note that these interim results are _not necessarily valid to any OSCAL schema_, although they are quite close to OSCAL profile and catalog syntax. + +The files in `testing/*` are only for this implementation. Implementation-independent tests and sample files for profile resolution are [with the specification](../../../specifications/profile-resolution). diff --git a/src/utils/util/resolver-pipeline/oscal-profile-resolve.sh b/src/utils/util/resolver-pipeline/oscal-profile-resolve.sh new file mode 100755 index 0000000000..7ac3e3d3b5 --- /dev/null +++ b/src/utils/util/resolver-pipeline/oscal-profile-resolve.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# OSCAL Profile Resolution Pipeline Wrapper + +set -Eeuo pipefail + +usage() { + cat <&2 + exit 1 +fi + +[[ -z "${1-}" ]] && { echo "Error: TARGET_PROFILE not specified"; usage; exit 1; } +TARGET_PROFILE=$1 +[[ -z "${2-}" ]] && { echo "Error: OUTPUT_BASELINE not specified"; usage; exit 1; } +OUTPUT_BASELINE=$2 + +ADDITIONAL_ARGS=$(shift 2; echo $@) + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)" +OSCAL_DIR="${SCRIPT_DIR}/../../../.." +POM_FILE="${OSCAL_DIR}/build/pom.xml" +ENTRY_STYLESHEET="${SCRIPT_DIR}/oscal-profile-RESOLVE.xsl" + +mvn \ + -f $POM_FILE \ + exec:java \ + -Dexec.mainClass="net.sf.saxon.Transform" \ + -Dexec.args="-t -s:$TARGET_PROFILE -xsl:$ENTRY_STYLESHEET -o:$OUTPUT_BASELINE $ADDITIONAL_ARGS" diff --git a/src/utils/util/resolver-pipeline/readme.md b/src/utils/util/resolver-pipeline/readme.md deleted file mode 100644 index 0e8913a01b..0000000000 --- a/src/utils/util/resolver-pipeline/readme.md +++ /dev/null @@ -1,49 +0,0 @@ -## Profile Resolver Pipeline - -Profile resolution is implemented here as a sequence of XSLT transformations. The sequence applies to defined inputs (a **source profile** with imported **catalog** sources) and produces defined outputs (a **profile resolution result** in the form of a catalog). The word **baseline** also refers to a particular profile in application, whether in its unprocessed form or its resolved, serialized form. - -The sequence of XSLT transformations reflects and roughly corresponds to the steps in profile resolution described for OSCAL in the [Profile Resolution Specification](https://pages.nist.gov/OSCAL/concepts/processing/profile-resolution/): - -- **selection**: importing catalogs or profiles, and selecting controls from them - -- **organization (merging)**: organizing the selected controls for the output representation - -- **modification**: setting parameters and potentially supplementing, amending or editing control text - -### Tests for this Implementation - -The `testing/*` folders contain XSpec tests that indicate expected interim results of each XSLT transformation in the sequence. - -Note that these interim results are *not necessarily valid to any OSCAL schema*, although they are quite close to OSCAL profile and catalog syntax. - -The files in `testing/*` are only for this implementation. Implementation-independent tests and sample files for profile resolution are [with the specification](../../../specifications/profile-resolution). - -### Invoking the XSLT - -Use a recent version of Saxon for best results — although we would also be *very interested* to hear from users of other XSLT engines conformant to the 3.1 family of XML standards (XSLT/XPath/XDM/XQuery). - -The entry point for the transformation pipeline is `oscal-profile-RESOLVE.xsl`, which invokes the transformation steps in sequence, taking the source profile document as primary input. Load Saxon with your document and this stylesheet as follows (for example): - -```bash -> java -cp saxon-he-10.0.jar net.sf.saxon.Transform -t -s:YOUR_PROFILE_DOCUMENT.xml -xsl:path/to/oscal-profile-RESOLVE.xsl -o:YOUR_RESULT_BASELINE.xml -``` - -You can optionally set one or more of the parameters listed in the following table, using syntax `name=value` at the end of the command above. The sequence of parameters is not significant. - -For example, -```bash -> java -cp saxon-he-10.0.jar net.sf.saxon.Transform -t -s:YOUR_PROFILE_DOCUMENT.xml -xsl:path/to/oscal-profile-RESOLVE.xsl -o:YOUR_RESULT_BASELINE.xml uuid-method=random-xslt hide-source-profile-uri=true -``` -| Name | Description | Default | -|---|---|---| -| `hide-source-profile-uri` | If `true`, the output catalog's metadata does not record the source profile's URI. | `false` | -| `path-to-source` | Path from output catalog to location of source profile. | None | -| `top-uuid` | UUID value for top-level element in output catalog, if `uuid-method` is `user-provided`. | None | -| `uuid-method` | Method for computing UUID of top-level element in output catalog. Valid values are: `random-xslt`, in which case the `random-number-generator` XPath function must be available; `random-java`, in which case the `java.util.UUID.randomUUID()` Java method must be available; `user-provided`, in which case you must specify the `top-uuid` parameter; `web-service`, referring to the `uuid-service` parameter value; and `fixed`. | `fixed`| -| `uuid-service` | URI for a web service that produces a UUID, if `uuid-method` is `web-service`.| `https://www.uuidgenerator.net/api/version4`| - -Alternatively, set up the bindings in an IDE or programming environment that has XSLT 3.1 support. - -Note that URIs (addresses) given in a profile document must link correctly as absolute or relative paths to their imported catalogs, as demonstrated in [examples](../../../specifications/profile-resolution/profile-resolution-examples). - -A serialized output of profile resolution takes the form of an OSCAL catalog. Assuming inputs are correctly formed, the output is valid to the catalog schema. diff --git a/src/utils/util/resolver-pipeline/resolver-2018/profile-resolver.xsl b/src/utils/util/resolver-pipeline/resolver-2018/profile-resolver.xsl deleted file mode 100644 index cdcd2f8ab3..0000000000 --- a/src/utils/util/resolver-pipeline/resolver-2018/profile-resolver.xsl +++ /dev/null @@ -1,656 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Can't resolve profile against {$authorityURI}, already imported in this chain: - {string-join($authorities-so-far,' / ')} - - - - - - - - - - - - - - - - - - - - - - - - - importing { $authority/*/local-name() } - - - - - - - - - - - - - - Matched profile unexpectedly - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/src/utils/util/resolver-pipeline/resolver-2018/resolution-notes.md b/src/utils/util/resolver-pipeline/resolver-2018/resolution-notes.md deleted file mode 100644 index f382b5d385..0000000000 --- a/src/utils/util/resolver-pipeline/resolver-2018/resolution-notes.md +++ /dev/null @@ -1,146 +0,0 @@ -# Profile semantics notes - -OSCAL Profile semantics: old profile resolver (2018) notes ... - -## Importing catalogs and including controls -`include/match[@pattern]` and `include/call[@control-id|@subcontrol-id]` *both* provide selection of controls. - -### Including controls (or subcontrols) by calling them by ID - -The ID of a control is the `@id` in the XML of that control. - -``` - - - - -``` - -`call` with `control-id` works to select the control with with given ID (identifier) in its catalog. - -Same for `call` with `subcontrol-id`, except subcontrols are included. - -Note that subcontrols may be included without their controls. Depending on other considerations (see below) this may not be an error. - -It is also possible to use a `with-subcontrols` flag when calling controls by ID, to include all subcontrols along with it. - - -``` - - - - -``` - -### Including controls by matching their ID - -``` - - - - -``` - -The value of `@pattern` is an (XSD) regular expression. It is matched against the ID. Both/any controls and subcontrols whose IDs match the patterns, are selected - so in this case, any whose ID starts with "pm". - -In future we may add the feature to match any associated string e.g. a title or property value, not only the ID. - -### Orphan or "loose" subcontrols - -It is a requirement that a subcontrol may be included in one import clause, with its control included (only) in another one. (Perhaps the first clause adds a subcontrol to a profile included in the second.) This should be supported as long as the control is indeed included in another import branch (with which it can eventually be merged subject to its logic). "The same control" for these purposes means the control with the same @id from the same catalog - so, to be more precise, a subcontrol can be reunited to any control with the same ID as its parent in (its own) home directory. An orphan subcontrol included without its control included anywhere (in the same profile albeit not the same import), may be an error (TBD). - -NB currently this feature (merging loose subcontrols) is still missing, subject to merge rules altogether. 2018/02/27 - -proposal 20180228 - resolution w/o merge returns import branches w/ orphan subcontrols still orphaned; then merge combines all controls from same catalogs together (w/o regrouping them); merge/build also does the regrouping - -test this on simple cases then come back to worse errors (e.g. duplicative/conflicting control imports) - -## Merge proposal - -`merge` - simply merges import branches together on their source catalogs (reuniting controls and subcontrols) - -However this is considered normal indeed to skip it would only be done ordinarily for diagnostic purposes. - -`merge/as-is` - rebuilds into hierarchy of home catalog (we have this working) - -`merge/custom` - instead, provides a way to "stack" the controls into a new organization (optionally reporting errors for dropped controls): - -``` - - - <match pattern="cm"/> -</group> -``` - -for purposes of error handling, remember that designers should be using exclusions, not letting profiles conflict even apparently. So `merge` might complain *any* time controls appear duplicative (there is always a remedy). - -Should running a resolution w/o merge, also make it impossible to patch? (B/c we don't know what we're patching?) - -## Modify logic ("patching") - -As implemented, modifications permit both removals and additions. - - -``` -<remove id-ref="e95b8652"/> -<remove class-ref="baseline"/> -<remove item-name="title"/> -``` - - * Remove the element with `@id` (id attribute) `e95b8652` - * Remove any element with `@class` token `baseline` - * Remove any `title` element - -These can be combined. - -`<add>` elements permit two attributes to indicate the position of the addition. `@target` (optional) indicates an element where the patch will be made. `@position` indicates where (in relation to the target) the patch will be placed. Four values are recognized for `@position`: `before`, `after`, `starting` and `ending`. - -You can add new components (stuff) to the front (after the title, before everything else) or at the end of any control or subcontrol, by including your addition in an `add` element *without a target*, like so: - -``` -<add position="starting"> - <prop class="cat">Category A</prop> -</add> -``` - -The same for `position="ending"`, which puts the new contents at the end of the control or subcontrol. - -If instead of patching your control or subcontrol at its start or end, you wish to locate precisely where you wish an addition. For this, use a `@target`: - -``` -<add position="after" target="advice"> - <part class="more_advice"> - <title>More advice -

More advice ...

- -``` - -The target works to identify an element by its `@id` or by a class token. - -When elements are matched using class tokens, there is the possibility that multiple elements of the given class are present in the control. The element where the insertion is to be made is then determined by the given `@position`: `position='before'` and `position='starting'` work to select the *first* of the candidates in the control as given, while `position='after'` and `position='ending'` work to select only the *last* element of the given class, within the control or subcontrol. - -The difference between `position='before'` and `position='starting'` is that "before" places the inserted content *before* the targeted element, while "starting" places it *inside* it, at the front. "ending" will place the inserted content at the end, inside the identified element. - -When the targeted element is a control or subcontrol, `before` is a synonym for `starting` - -If no position is given, `position='ending'` is inferred: the insertion happens inside the targeted element (or the matched control or subcontrol), at the end. - -Note that this default works along with the rule that if no target (class or ID) is indicated, the control or subcontrol itself is the point of insertion. So: - -``` - - - Local guidelines - - - - Supplementary Guidance -

More advice ...

-
-
-
-``` - -This adds the new `part` at the end of the `ac.1` control. - -Note also that position "before" and "after" work only when @target is also used (to identify some contents inside a control); they are inoperable when the target is a control or subcontrol. They result in no addition being made. (A Schematron check could be made for this.)