Profile Resolution Only Supports Explicit Paths for imports, not @href anchor links

[aj-stein-nist]

Describe the bug

While following the the website tutorials and mimicking the SSP authoring tools demo, I was unable to run commands like so in a newly created workspace to create a SSP because profile import directives only work with local paths and not URI fragments.

To Reproduce

Steps to reproduce the behavior:

$ pip3 install compliance-trestle
$ trestle version
Trestle version v0.34.0 based on OSCAL version 1.0.0
$ trestle init
$ trestle import -f https://raw.githubusercontent.com/usnistgov/oscal-content/master/nist.gov/SP800-53/rev5/json/NIST_SP-800-53_rev5_catalog.json -o nist_rev5_all
$ trestle import -f https://raw.githubusercontent.com/GSA/fedramp-automation/master/dist/content/baselines/rev4/json/FedRAMP_rev4_MODERATE-baseline_profile.json -o fedramp_rev4_moderate
$ trestle author ssp-generate -p fedramp_rev4_moderate -s 'ImplGuidance:Implementation Guidance,ExpectedEvidence:Expected Evidence' -o blossom_ssp
trestle.core.commands.author.ssp:106 ERROR: Error creating the resolved profile catalog: Invalid uri not recognized as a readable file path with extension: #ad005eae-cc63-4e64-9109-3905a9a825e4

For official FedRAMP profiles, #ad005eae-cc63-4e64-9109-3905a9a825e4 is the import directive href, as indicated by anchor, that points to the following profile in the back-matter resources of the relevant FedRAMP profile:

        {
          "uuid": "ad005eae-cc63-4e64-9109-3905a9a825e4",
          "title": "NIST Special Publication (SP) 800-53",
          "props": [
            {
              "name": "version",
              "value": "Revision 4"
            }
          ],
          "rlinks": [
            {
              "href": "https://raw.githubusercontent.com/usnistgov/oscal-content/v1.0.0/nist.gov/SP800-53/rev4/xml/NIST_SP-800-53_rev4_catalog.xml",
              "media-type": "application/xml"
            },
            {
              "href": "https://raw.githubusercontent.com/usnistgov/oscal-content/v1.0.0/nist.gov/SP800-53/rev4/json/NIST_SP-800-53_rev4_catalog.json",
              "media-type": "application/oscal.catalog+json"
            },
            {
              "href": "https://raw.githubusercontent.com/usnistgov/oscal-content/v1.0.0/nist.gov/SP800-53/rev4/yaml/NIST_SP-800-53_rev4_catalog.yaml",
              "media-type": "application/oscal.catalog+yaml"
            }

Judging the type signature of the relevant code, it would appear anchor links retrieved by href are not considered, and maybe processing multiple rlinks would require additional feature enhancement for the project.

URI fragment (aka "anchor link") resolution has been a requirement of the OSCAL profile resolution spec in 1.0.0, as has been documented in the spec. GSA is just a very prominent user.

Expected behavior

Profile resolution would work.

Screenshots / Logs.

Terminal output provided above.

Environment

• OS: Windows 10 and WSL2 providing Ubuntu 20.04 LTS (recently updated)
• Python version: 3.8.10
• Installed packages:
  • ansible==5.1.0
  • ansible-core==2.12.1
  • anyio==3.5.0
  • argcomplete==2.0.0
  • attrs==19.3.0
  • aws-amicleaner==0.2.2
  • awscli==1.22.24
  • bcrypt==3.2.0
  • beautifulsoup4==4.10.0
  • black==22.1.0
  • blessings==1.7
  • boto==2.49.0
  • boto3==1.20.24
  • botocore==1.23.24
  • certifi==2021.10.8
  • cffi==1.15.0
  • chardet==4.0.0
  • charset-normalizer==2.0.9
  • click==8.0.3
  • cmarkgfm==0.6.0
  • colorama==0.4.3
  • compliance-trestle==0.34.0
  • cryptography==36.0.1
  • datamodel-code-generator==0.11.19
  • defusedxml==0.7.1
  • dnspython==2.2.0
  • docutils==0.15.2
  • email-validator==1.1.3
  • et-xmlfile==1.1.0
  • furl==2.1.3
  • genson==1.2.2
  • h11==0.12.0
  • httpcore==0.14.7
  • httpx==0.22.0
  • idna==3.3
  • ilcli==0.3.2
  • inflect==5.4.0
  • isodate==0.6.1
  • isort==5.10.1
  • Jinja2==3.0.3
  • jmespath==0.10.0
  • jsonschema==3.2.0
  • MarkupSafe==2.0.1
  • mypy-extensions==0.4.3
  • ntlm-auth==1.5.0
  • openapi-schema-validator==0.1.6
  • openapi-spec-validator==0.3.3
  • openpyxl==3.0.9
  • orderedmultidict==1.0.1
  • orjson==3.6.6
  • packaging==21.3
  • paramiko==2.9.2
  • pathspec==0.9.0
  • platformdirs==2.4.1
  • prance==0.21.8.0
  • prettytable==2.5.0
  • pyasn1==0.4.8
  • pycparser==2.21
  • pydantic==1.9.0
  • PyNaCl==1.5.0
  • pyparsing==3.0.6
  • pyrsistent==0.16.1
  • PySnooper==1.1.0
  • python-dateutil==2.8.2
  • python-dotenv==0.19.2
  • python-frontmatter==1.0.0
  • PyYAML==5.4.1
  • requests==2.26.0
  • requests-ntlm==1.1.0
  • resolvelib==0.5.4
  • rfc3986==1.5.0
  • rsa==4.7.2
  • ruamel.yaml==0.17.20
  • ruamel.yaml.clib==0.2.6
  • s3transfer==0.5.0
  • semver==2.13.0
  • six==1.16.0
  • sniffio==1.2.0
  • soupsieve==2.3.1
  • toml==0.10.2
  • tomli==2.0.1
  • typed-ast==1.5.2
  • typing-extensions==4.0.1
  • urllib3==1.26.7
  • wcwidth==0.2.5

[aj-stein-nist]

Sorry this might be a duplicate of #832.

[fsuits]

@aj-stein-nist Thanks! Apparently it also duplicates #759
Will look into it.

[aj-stein-nist]

I wanted this sooner rather than later, so I might try to send you a PR tomorrow or the day after for your review, unless you would prefer someone in the team work on it. Let me know. Thanks!

[fsuits]

I think that would be fine and we would welcome your contribution. As long as it is only during resolution and not during the assembly of a new profile it should be well defined and simply involve following a link in the json. Thanks!