Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error if Tern calls Scancode-Toolkit #1202

Closed
Jeeppler opened this issue Dec 21, 2022 · 25 comments · Fixed by #1237
Closed

Error if Tern calls Scancode-Toolkit #1202

Jeeppler opened this issue Dec 21, 2022 · 25 comments · Fixed by #1237
Labels
bug Something went wrong

Comments

@Jeeppler
Copy link

Describe the bug

Tern is unable to scan using Scancode-Toolkit.

With Tern only the image can be scanned and results in an report.

To Reproduce

Scan Alpine based image using Tern + Scancode-Toolkit

tern report -f spdxjson -w "container.tar" -o "result.spdx.json" -x scancode

Error in terminal

2022-12-21 20:48:28,485 - DEBUG - __main__ - Starting...
2022-12-21 20:48:28,485 - DEBUG - prep - Setting up...
2022-12-21 20:48:28,486 - DEBUG - rootfs - Running command: chmod +x /usr/local/lib/python3.9/dist-packages/tern/tools/fs_hash.sh
2022-12-21 20:48:28,488 - DEBUG - run - Starting analysis...
2022-12-21 20:48:28,496 - DEBUG - rootfs - Running command: tar -tf /workspace/workspace/9a44489b-96c1-49f7-a627-cbffc58e366b/upload/extracted/binaries/test.tar
2022-12-21 20:48:28,498 - DEBUG - rootfs - Running command: tar -x -f /workspace/workspace/9a44489b-96c1-49f7-a627-cbffc58e366b/upload/extracted/binaries/test.tar -C /root/.tern/temp
2022-12-21 20:48:28,504 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/3e571912155d9bac1a5285bf1c21105bea53585f77a159316eed491882710ab2/layer.tar
2022-12-21 20:48:28,507 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/3e571912155d9bac1a5285bf1c21105bea53585f77a159316eed491882710ab2/layer.tar -C /root/.tern/temp/3e571912155d9bac1a5285bf1c21105bea53585f77a159316eed491882710ab2/contents
2022-12-21 20:48:28,528 - DEBUG - rootfs - Running command: /usr/local/lib/python3.9/dist-packages/tern/tools/fs_hash.sh /root/.tern/temp/3e571912155d9bac1a5285bf1c21105bea53585f77a159316eed491882710ab2/contents
2022-12-21 20:48:28,632 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/eb2f2a3b73763d543b7256e39e4a42ea190bd18a1e1c8197aaefa6ae5005ae94/layer.tar
2022-12-21 20:48:28,634 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/eb2f2a3b73763d543b7256e39e4a42ea190bd18a1e1c8197aaefa6ae5005ae94/layer.tar -C /root/.tern/temp/eb2f2a3b73763d543b7256e39e4a42ea190bd18a1e1c8197aaefa6ae5005ae94/contents
2022-12-21 20:48:28,635 - DEBUG - rootfs - Running command: /usr/local/lib/python3.9/dist-packages/tern/tools/fs_hash.sh /root/.tern/temp/eb2f2a3b73763d543b7256e39e4a42ea190bd18a1e1c8197aaefa6ae5005ae94/contents
2022-12-21 20:48:28,639 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/cf9a1a27ba287bed93e131f5cb5831b706905811bf91d5e6b5b323187d7cac92/layer.tar
2022-12-21 20:48:28,641 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/cf9a1a27ba287bed93e131f5cb5831b706905811bf91d5e6b5b323187d7cac92/layer.tar -C /root/.tern/temp/cf9a1a27ba287bed93e131f5cb5831b706905811bf91d5e6b5b323187d7cac92/contents
2022-12-21 20:48:28,643 - DEBUG - rootfs - Running command: /usr/local/lib/python3.9/dist-packages/tern/tools/fs_hash.sh /root/.tern/temp/cf9a1a27ba287bed93e131f5cb5831b706905811bf91d5e6b5b323187d7cac92/contents
2022-12-21 20:48:28,646 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/ac1cc3694703233d4731e335b6bd4cedc7bd6fe00dead7b76068f1653b147d3a/layer.tar
2022-12-21 20:48:28,647 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/ac1cc3694703233d4731e335b6bd4cedc7bd6fe00dead7b76068f1653b147d3a/layer.tar -C /root/.tern/temp/ac1cc3694703233d4731e335b6bd4cedc7bd6fe00dead7b76068f1653b147d3a/contents
2022-12-21 20:48:28,649 - DEBUG - rootfs - Running command: /usr/local/lib/python3.9/dist-packages/tern/tools/fs_hash.sh /root/.tern/temp/ac1cc3694703233d4731e335b6bd4cedc7bd6fe00dead7b76068f1653b147d3a/contents
2022-12-21 20:48:28,652 - DEBUG - common - Loading packages from cache: layer "d1a00b82ba"
2022-12-21 20:48:28,654 - DEBUG - common - Loading files from cache: layer "d1a00b82ba"
2022-12-21 20:48:28,657 - DEBUG - rootfs - Running command: /usr/local/bin/scancode -ilpcu --quiet --timeout 300 -n 5 --json - /root/.tern/temp/3e571912155d9bac1a5285bf1c21105bea53585f77a159316eed491882710ab2/contents
Traceback (most recent call last):
  File "/usr/local/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/tern/__main__.py", line 311, in main
    do_main(args)
  File "/usr/local/lib/python3.9/dist-packages/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/usr/local/lib/python3.9/dist-packages/tern/analyze/default/container/run.py", line 80, in execute_image
    cimage.default_analyze(full_image, args)
  File "/usr/local/lib/python3.9/dist-packages/tern/analyze/default/container/image.py", line 70, in default_analyze
    passthrough.run_extension_layer(image_obj.layers[0], options.extend,
  File "/usr/local/lib/python3.9/dist-packages/tern/analyze/passthrough.py", line 92, in run_extension_layer
    return mgr.driver.execute_layer(image_layer, redo)
  File "/usr/local/lib/python3.9/dist-packages/tern/extensions/scancode/executor.py", line 219, in execute_layer
    file_list, package_list = collect_layer_data(image_layer)
  File "/usr/local/lib/python3.9/dist-packages/tern/extensions/scancode/executor.py", line 162, in collect_layer_data
    files.append(get_scancode_file(f))
  File "/usr/local/lib/python3.9/dist-packages/tern/extensions/scancode/executor.py", line 65, in get_scancode_file
    fd.copyrights = [c['value'] for c in file_dict['copyrights']]
  File "/usr/local/lib/python3.9/dist-packages/tern/extensions/scancode/executor.py", line 65, in <listcomp>
    fd.copyrights = [c['value'] for c in file_dict['copyrights']]
KeyError: 'value'

Expected behavior

Tern can call Scancode-Toolkit without errors.

Environment you are running Tern on

  • Tern version 2.11.0
  • ScanCode version: 31.2.1
  • Debian 11
  • Python 3.9.2
@rnjudge
Copy link
Contributor

rnjudge commented Jan 6, 2023

Interesting! Thanks @Jeeppler I will take a look. Do you have an example container so I can reproduce + debug?

@rnjudge rnjudge added the bug Something went wrong label Jan 6, 2023
@Jeeppler
Copy link
Author

@rnjudge I used an Alpine Image. You can find the build/run script here: https://github.com/mercedes-benz/sechub/tree/develop/sechub-pds-solutions/tern/tests. The Dockerfile can be found here: https://github.com/mercedes-benz/sechub/tree/develop/sechub-pds-solutions/tern/tests/alpine.

I think the main issue is, that the report structure changed from Scancode 30 to 31. I used the new Scancode 31 + Tern.
Previously, I did a similar scan with Scancode 30 + Tern and it worked well.

@rnjudge
Copy link
Contributor

rnjudge commented Jan 17, 2023

@Jeeppler I have not forgot about this. The issue I am trying to workaround right now is simply installing scancode on my VM on my Mac M1 so I can debug and fix. I opened an issue for it with Scancode here. What I've been told is that they are re-releasing Scancode with fixes for some of the dependent libraries so I'll have to wait for that...
Screen Shot 2023-01-17 at 1 29 01 PM

@amallayev
Copy link

@rnjudge It seems only some images produce this error, I wonder what do they have in common:

# tern report -x scancode -i docker-daemon:node:18.12.1 -o bitnami-node.18.12.1-prod.tern.$(date +"%d.%m.%Y").txt
2023-01-19 15:17:33,198 - DEBUG - __main__ - Starting...                                                                     [21/535]2023-01-19 15:17:33,198 - DEBUG - prep - Setting up...                                                                               2023-01-19 15:17:37,099 - DEBUG - rootfs - Running command: chmod +x /root/ternenv/lib/python3.10/site-packages/tern/tools/fs_hash.sh2023-01-19 15:17:37,101 - DEBUG - run - Starting analysis...                                                                         2023-01-19 15:17:37,102 - DEBUG - skopeo - Attempting to pull image "docker-daemon:node:18.12.1"                                     2023-01-19 15:17:37,102 - DEBUG - rootfs - Running command: skopeo copy docker-daemon:node:18.12.1 dir:/root/.tern/temp
2023-01-19 15:18:05,324 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/58861e9c7a9b85fcb52553413d8725a7ae8d215349b6325cfdb9d0c38b7f5583                                                                                                                     2023-01-19 15:18:05,375 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/58861e9c7a9b85fcb52553413d8725a7ae8d215349b6325cfdb9d0c38b7f5583 -C /root/.tern/temp/1/contents                                                                                    2023-01-19 15:18:06,022 - DEBUG - rootfs - Running command: /root/ternenv/lib/python3.10/site-packages/tern/tools/fs_hash.sh /root/.tern/temp/1/contents                                                                                                                  2023-01-19 15:18:18,438 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/8a8dfd8ce7a77963ad1d0d8009a79f22f71c674bcf6198699bfd20ce868ff27b                                                                                                                     2023-01-19 15:18:18,601 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/8a8dfd8ce7a77963ad1d0d8009a79f22f71c674bcf6198699bfd20ce868ff27b -C /root/.tern/temp/2/contents
2023-01-19 15:18:21,603 - DEBUG - rootfs - Running command: /root/ternenv/lib/python3.10/site-packages/tern/tools/fs_hash.sh /root/.tern/temp/2/contents
2023-01-19 15:19:33,438 - DEBUG - common - Loading packages from cache: layer "9d22d5e891"
2023-01-19 15:19:33,932 - DEBUG - common - Loading files from cache: layer "9d22d5e891"
2023-01-19 15:19:35,235 - DEBUG - rootfs - Running command: /root/ternenv/bin/scancode -ilpcu --quiet --timeout 300 -n 1 --json - /root/.tern/temp/1/contents
Traceback (most recent call last):
  File "/root/ternenv/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/root/ternenv/lib/python3.10/site-packages/tern/__main__.py", line 311, in main
    do_main(args)
  File "/root/ternenv/lib/python3.10/site-packages/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/root/ternenv/lib/python3.10/site-packages/tern/analyze/default/container/run.py", line 80, in execute_image
    cimage.default_analyze(full_image, args)
  File "/root/ternenv/lib/python3.10/site-packages/tern/analyze/default/container/image.py", line 70, in default_analyze
    passthrough.run_extension_layer(image_obj.layers[0], options.extend,
  File "/root/ternenv/lib/python3.10/site-packages/tern/analyze/passthrough.py", line 92, in run_extension_layer
    return mgr.driver.execute_layer(image_layer, redo)
  File "/root/ternenv/lib/python3.10/site-packages/tern/extensions/scancode/executor.py", line 219, in execute_layer
    file_list, package_list = collect_layer_data(image_layer)
  File "/root/ternenv/lib/python3.10/site-packages/tern/extensions/scancode/executor.py", line 162, in collect_layer_data
    files.append(get_scancode_file(f))
  File "/root/ternenv/lib/python3.10/site-packages/tern/extensions/scancode/executor.py", line 65, in get_scancode_file
    fd.copyrights = [c['value'] for c in file_dict['copyrights']]
  File "/root/ternenv/lib/python3.10/site-packages/tern/extensions/scancode/executor.py", line 65, in <listcomp>
    fd.copyrights = [c['value'] for c in file_dict['copyrights']]
KeyError: 'value'

# scancode --version; tern --version; python --version
ScanCode version: 31.2.4
ScanCode Output Format version: 2.0.0
SPDX License list version: 3.18
Tern version 2.11.0
   python version = 3.10.6 (main, Nov 14 2022, 16:10:14)
Python 3.10.6

@Jeeppler
Copy link
Author

@rnjudge and @amallayev is there anything I can do to help to get this fixed?

@rnjudge
Copy link
Contributor

rnjudge commented May 10, 2023

@Jeeppler let me try to install the latest release of scancode. It's hard for me to debug and fix this when I can't install scancode :/

@rnjudge
Copy link
Contributor

rnjudge commented May 19, 2023

@Jeeppler I still cannot install scanacode which is quite concerning for me wrt Tern's integration with the library. I will try to setup a linux environment to debug on my old mac. Stay tuned.

@rnjudge
Copy link
Contributor

rnjudge commented May 23, 2023

@Jeeppler I am able to install scancode-toolkit-mini-31.2.6 and reproduce this error. I'll take a look now.

rnjudge added a commit to rnjudge/tern that referenced this issue May 23, 2023
Scancode v31.0.0 includes changes[1] to JSON output attribute
names which was causing processing KeyErrors when Tern would run with
Scancode. This commit adds code that can accomodate the new attribute
property names in Scancode, as well as the older value names (in case we
have users still using older Scancode versions).

[1]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#v3100---2022-08-17

Resolves tern-tools#1202

Signed-off-by: Rose Judge <[email protected]>
@rnjudge
Copy link
Contributor

rnjudge commented May 23, 2023

Hi @Jeeppler -- since I am unable to install scancode==30.* to compare the results of my fix, are you able to help? Could you run tern with scancode==30.* (the version before you saw this error) on alpine:latest and share the results with me so I can compare? It should be a 1:1 comparison but I want to make sure.

If you also want to test my changes, which should fix the error you are seeing, they are available here: https://github.com/rnjudge/tern/tree/issue-1202

Also cc amallayev. The issue was with changes to Scancode's JSON attribute names but should be fixed now.

@Jeeppler
Copy link
Author

@rnjudge I will have a look, but probably towards the end of the week.

@Jeeppler
Copy link
Author

Jeeppler commented May 27, 2023

@rnjudge I tried to scan with scancode=30.* and tern=2.9.1, but it did not work. I will try again.

@Jeeppler
Copy link
Author

Jeeppler commented Jun 1, 2023

Setup:

Scancode v30.1.0 (full)
Tern 2.9.1
alpine:3.18 (latest)

{"SPDXID": "SPDXRef-DOCUMENT", "spdxVersion": "SPDX-2.2", "creationInfo": {"created": "2023-06-01T18:21:06Z", "creators": ["Tool: tern-2.9.1"], "licenseListVersion": "3.8"}, "name": "Tern report for /workspace/workspace/ce3214bf-a0e3-47ed-bcc9-03a3ce48e939/upload/extracted/binaries/sechub-test-alpine.tar", "dataLicense": "CC0-1.0", "comment": "This document was generated by the Tern Project: https://github.com/tern-tools/tern", "documentNamespace": "https://spdx.org/spdxdocs/tern-report-2.9.1-/workspace/workspace/ce3214bf-a0e3-47ed-bcc9-03a3ce48e939/upload/extracted/binaries/sechub-test-alpine.tar-765777f8-576c-4b74-a86d-cd5311ec0923", "documentDescribes": ["SPDXRef--workspace-workspace-ce3214bf-a0e3-47ed-bcc9-03a3ce48e939-upload-extracted-binaries-sechub-test-alpine.tar"], "packages": [{"name": "/workspace/workspace/ce3214bf-a0e3-47ed-bcc9-03a3ce48e939/upload/extracted/binaries/sechub-test-alpine.tar", "SPDXID": "SPDXRef--workspace-workspace-ce3214bf-a0e3-47ed-bcc9-03a3ce48e939-upload-extracted-binaries-sechub-test-alpine.tar", "versionInfo": "", "downloadLocation": "NOASSERTION", "filesAnalyzed": false, "licenseConcluded": "NOASSERTION", "licenseDeclared": "NOASSERTION", "copyrightText": "NOASSERTION"}, {"name": "layer.tar", "SPDXID": "SPDXRef-bb01bd7e32", "packageFileName": "2f1efefa42dbd6a8375b45c97e21453d2de1343a814ecaf1e098eff5a18a1e62/layer.tar", "downloadLocation": "NONE", "filesAnalyzed": true, "checksums": [{"algorithm": "SHA256", "checksumValue": "bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c"}], "licenseConcluded": "NOASSERTION", "licenseDeclared": "NOASSERTION", "copyrightText": "NOASSERTION", "hasFiles": ["SPDXRef-d92f88f", "SPDXRef-86fe0aa", "SPDXRef-dae5406", "SPDXRef-1985b0f", "SPDXRef-b71f00f", "SPDXRef-aa3c220", "SPDXRef-fda53c5", "SPDXRef-87792e2", "SPDXRef-f6d8f70", "SPDXRef-1c4d897", "SPDXRef-2d0f861", "SPDXRef-279034a", "SPDXRef-633d19a", "SPDXRef-dc4d9fc", "SPDXRef-c18387e", "SPDXRef-1b4e9bb", "SPDXRef-a21ebf0", "SPDXRef-7ba46d2", "SPDXRef-33969a0", "SPDXRef-a3012c9", "SPDXRef-a3e9b71", "SPDXRef-d424e27", "SPDXRef-66be5cb", "SPDXRef-5a833f1", "SPDXRef-5a8796f", "SPDXRef-91fe54c", "SPDXRef-cdb85a6", "SPDXRef-2fc2f4e", "SPDXRef-153c40c", "SPDXRef-146c4cf", "SPDXRef-c484e7b", "SPDXRef-0ec35be", "SPDXRef-b76c387", "SPDXRef-ea641f2", "SPDXRef-33b3640", "SPDXRef-999a530", "SPDXRef-496a21c", "SPDXRef-8ac6419", "SPDXRef-7bda380", "SPDXRef-686b678", "SPDXRef-5f3baa7", "SPDXRef-aa963ea", "SPDXRef-eb6a093", "SPDXRef-bf59c5b", "SPDXRef-e0cb6b4", "SPDXRef-78e444d", "SPDXRef-9637d53", "SPDXRef-df56d8a", "SPDXRef-98ee4a4", "SPDXRef-ebe9b09", "SPDXRef-badb010", "SPDXRef-e7ea5c6", "SPDXRef-3bc3290", "SPDXRef-37a98ca", "SPDXRef-e49126a", "SPDXRef-614e0ba", "SPDXRef-7881d4d", "SPDXRef-a77a4a7", "SPDXRef-045922d", "SPDXRef-7248e09", "SPDXRef-7672107", "SPDXRef-ff732b8", "SPDXRef-5d3d492", "SPDXRef-554cb0f", "SPDXRef-ca09358", "SPDXRef-d7e7f1c", "SPDXRef-fd00e48", "SPDXRef-f2411fa", "SPDXRef-4840548", "SPDXRef-141f02a", "SPDXRef-a4e24dd", "SPDXRef-ad99cd4", "SPDXRef-61eb6d2", "SPDXRef-7b6ca29", "SPDXRef-b65f81c", "SPDXRef-472a5eb", "SPDXRef-55ef821", "SPDXRef-01882bf", "SPDXRef-c1b8cad", "SPDXRef-84951ea", "SPDXRef-43bef7f", "SPDXRef-af04a49", "SPDXRef-dde0b0b", "SPDXRef-b653f98", "SPDXRef-4a5cf31", "SPDXRef-24208ae", "SPDXRef-d865831", "SPDXRef-1eb62bd", "SPDXRef-be7d7c2", "SPDXRef-d3d5fae"], "packageVerificationCode": {"packageVerificationCodeValue": null}, "comment": "Generated with ScanCode and provided on an \"AS IS\" BASIS, WITHOUT WARRANTIES\nOR CONDITIONS OF ANY KIND, either express or implied. No content created from\nScanCode should be considered or used as legal advice. Consult an Attorney\nfor any legal advice.\nScanCode is a free software code scanning tool from nexB Inc. and others.\nVisit https://github.com/nexB/scancode-toolkit/ for support and download.\n", "licenseInfoFromFiles": ["LicenseRef-71aedc7", "LicenseRef-6ddea61", "LicenseRef-96e2b0f", "LicenseRef-2730428", "LicenseRef-8ba26df", "LicenseRef-da38037", "LicenseRef-a7811ea"]}, {"name": "layer.tar", "SPDXID": "SPDXRef-9cf4351fab", "packageFileName": "e840b5e4e95b5fec597af6bc1a1c1d1613df77cd6d21c686db8039e9db8257c0/layer.tar", "downloadLocation": "NONE", "filesAnalyzed": true, "checksums": [{"algorithm": "SHA256", "checksumValue": "9cf4351fab2f32d3f294f82da2e447e8b9129a3ea9c5ba69becf93cd869950f6"}], "licenseConcluded": "NOASSERTION", "licenseDeclared": "NOASSERTION", "copyrightText": "NOASSERTION", "hasFiles": ["SPDXRef-d08dd5c", "SPDXRef-e4639c1", "SPDXRef-ba70e82"], "packageVerificationCode": {"packageVerificationCodeValue": "6e753c4987577fadf40b7447b817bcf805b62b6a"}, "comment": "Generated with ScanCode and provided on an \"AS IS\" BASIS, WITHOUT WARRANTIES\nOR CONDITIONS OF ANY KIND, either express or implied. No content created from\nScanCode should be considered or used as legal advice. Consult an Attorney\nfor any legal advice.\nScanCode is a free software code scanning tool from nexB Inc. and others.\nVisit https://github.com/nexB/scancode-toolkit/ for support and download.\n", "licenseInfoFromFiles": ["LicenseRef-bdceeb6", "LicenseRef-a7811ea", "LicenseRef-5fc4e2a", "LicenseRef-8ba26df"]}, {"name": "layer.tar", "SPDXID": "SPDXRef-22d4e5deb8", "packageFileName": "e721fed7524cd0fabefc8f2cac99a08aad5106ddcc5d08dbc4299fa5d55a8e16/layer.tar", "downloadLocation": "NONE", "filesAnalyzed": true, "checksums": [{"algorithm": "SHA256", "checksumValue": "22d4e5deb8aa860c0e416f6624ab56bfe22fbc903fe2037d23acf2739ec71110"}], "licenseConcluded": "NOASSERTION", "licenseDeclared": "NOASSERTION", "copyrightText": "NOASSERTION", "hasFiles": ["SPDXRef-f937350"], "packageVerificationCode": {"packageVerificationCodeValue": "3959c845170fb82e27e429b1867c44a9570b65d7"}, "comment": "Generated with ScanCode and provided on an \"AS IS\" BASIS, WITHOUT WARRANTIES\nOR CONDITIONS OF ANY KIND, either express or implied. No content created from\nScanCode should be considered or used as legal advice. Consult an Attorney\nfor any legal advice.\nScanCode is a free software code scanning tool from nexB Inc. and others.\nVisit https://github.com/nexB/scancode-toolkit/ for support and download.\n", "licenseInfoFromFiles": ["LicenseRef-b9d3bc5"]}, {"name": "layer.tar", "SPDXID": "SPDXRef-ebe91295c1", "packageFileName": "00e1169787a9be8d6643cae38a68aa2ec13519ad8e43b7f8a15f211b45f41868/layer.tar", "downloadLocation": "NONE", "filesAnalyzed": true, "checksums": [{"algorithm": "SHA256", "checksumValue": "ebe91295c13fb9c64b2577f205129ea12bc25543c4856f512eabc77937412b22"}], "licenseConcluded": "NOASSERTION", "licenseDeclared": "NOASSERTION", "copyrightText": "NOASSERTION", "hasFiles": ["SPDXRef-cde5ad2"], "packageVerificationCode": {"packageVerificationCodeValue": "3959c845170fb82e27e429b1867c44a9570b65d7"}, "comment": "Generated with ScanCode and provided on an \"AS IS\" BASIS, WITHOUT WARRANTIES\nOR CONDITIONS OF ANY KIND, either express or implied. No content created from\nScanCode should be considered or used as legal advice. Consult an Attorney\nfor any legal advice.\nScanCode is a free software code scanning tool from nexB Inc. and others.\nVisit https://github.com/nexB/scancode-toolkit/ for support and download.\n", "licenseInfoFromFiles": ["LicenseRef-b9d3bc5"]}], "relationships": [{"spdxElementId": "SPDXRef-DOCUMENT", "relatedSpdxElement": "SPDXRef--workspace-workspace-ce3214bf-a0e3-47ed-bcc9-03a3ce48e939-upload-extracted-binaries-sechub-test-alpine.tar", "relationshipType": "DESCRIBES"}, {"spdxElementId": "SPDXRef--workspace-workspace-ce3214bf-a0e3-47ed-bcc9-03a3ce48e939-upload-extracted-binaries-sechub-test-alpine.tar", "relatedSpdxElement": "SPDXRef-bb01bd7e32", "relationshipType": "CONTAINS"}, {"spdxElementId": "SPDXRef--workspace-workspace-ce3214bf-a0e3-47ed-bcc9-03a3ce48e939-upload-extracted-binaries-sechub-test-alpine.tar", "relatedSpdxElement": "SPDXRef-9cf4351fab", "relationshipType": "CONTAINS"}, {"spdxElementId": "SPDXRef-bb01bd7e32", "relatedSpdxElement": "SPDXRef-9cf4351fab", "relationshipType": "HAS_PREREQUISITE"}, {"spdxElementId": "SPDXRef--workspace-workspace-ce3214bf-a0e3-47ed-bcc9-03a3ce48e939-upload-extracted-binaries-sechub-test-alpine.tar", "relatedSpdxElement": "SPDXRef-22d4e5deb8", "relationshipType": "CONTAINS"}, {"spdxElementId": "SPDXRef-9cf4351fab", "relatedSpdxElement": "SPDXRef-22d4e5deb8", "relationshipType": "HAS_PREREQUISITE"}, {"spdxElementId": "SPDXRef--workspace-workspace-ce3214bf-a0e3-47ed-bcc9-03a3ce48e939-upload-extracted-binaries-sechub-test-alpine.tar", "relatedSpdxElement": "SPDXRef-ebe91295c1", "relationshipType": "CONTAINS"}, {"spdxElementId": "SPDXRef-22d4e5deb8", "relatedSpdxElement": "SPDXRef-ebe91295c1", "relationshipType": "HAS_PREREQUISITE"}], "files": [{"fileName": "lib/libcrypto.so.3", "SPDXID": "SPDXRef-d92f88f", "checksums": [{"algorithm": "SHA1", "checksumValue": "0235120a70f32d7b6d69e6b83078cfad93047936"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"], "fileContributors": ["<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>", "<[email protected]>"]}, {"fileName": "lib/libssl.so.3", "SPDXID": "SPDXRef-86fe0aa", "checksums": [{"algorithm": "SHA1", "checksumValue": "dc76da41505b923236e09815a2fdfda1106fffe6"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "lib/apk/db/triggers", "SPDXID": "SPDXRef-dae5406", "checksums": [{"algorithm": "SHA1", "checksumValue": "d237beb9bcae20936dab87ce944d8383c852e273"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "lib/apk/db/installed", "SPDXID": "SPDXRef-1985b0f", "checksums": [{"algorithm": "SHA1", "checksumValue": "8a95001bb3ef8aa5019e699f4afa0ef28edba870"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-6ddea61", "LicenseRef-96e2b0f", "LicenseRef-2730428", "LicenseRef-8ba26df", "LicenseRef-da38037", "LicenseRef-a7811ea"]}, {"fileName": "lib/apk/db/lock", "SPDXID": "SPDXRef-b71f00f", "checksums": [{"algorithm": "SHA1", "checksumValue": "None"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "licenseInfoInFiles": ["NONE"]}, {"fileName": "lib/apk/db/scripts.tar", "SPDXID": "SPDXRef-aa3c220", "checksums": [{"algorithm": "SHA1", "checksumValue": "f5abe5b3b6ef53d24d45d0f0666d7e09b2e4b109"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "lib/ld-musl-x86_64.so.1", "SPDXID": "SPDXRef-fda53c5", "checksums": [{"algorithm": "SHA1", "checksumValue": "5adcbfa23f38f04c7cb7ba4e74177dfd2b77bf06"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "lib/sysctl.d/00-alpine.conf", "SPDXID": "SPDXRef-87792e2", "checksums": [{"algorithm": "SHA1", "checksumValue": "1e9125cd6d7112098a7c446d4f2ee8a269a7aba7"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "lib/libapk.so.2.14.0", "SPDXID": "SPDXRef-f6d8f70", "checksums": [{"algorithm": "SHA1", "checksumValue": "82db0af9c582fc1474ba2787c5e6785c89d5fa31"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "lib/libz.so.1.2.13", "SPDXID": "SPDXRef-1c4d897", "checksums": [{"algorithm": "SHA1", "checksumValue": "db729620754c73e0fafb66ec33f7bade6e1e519b"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"], "noticeText": "Copyright 1995-2022 Jean-loup Gailly and Mark Adler\nCopyright 1995-2022 Mark Adler\n"}, {"fileName": "etc/os-release", "SPDXID": "SPDXRef-2d0f861", "checksums": [{"algorithm": "SHA1", "checksumValue": "747e9fb3c8b8fd20715323359e9de42fcfdd4dfc"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/secfixes.d/alpine", "SPDXID": "SPDXRef-279034a", "checksums": [{"algorithm": "SHA1", "checksumValue": "77d75e5c584d86677fff1e11221837d088922ca9"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/shells", "SPDXID": "SPDXRef-633d19a", "checksums": [{"algorithm": "SHA1", "checksumValue": "a239b661da4227a07f6a9183699fd275bdb12640"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/sysctl.conf", "SPDXID": "SPDXRef-dc4d9fc", "checksums": [{"algorithm": "SHA1", "checksumValue": "e2ea73ded7e7371664204b148569fb5e88b0f7a8"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/motd", "SPDXID": "SPDXRef-c18387e", "checksums": [{"algorithm": "SHA1", "checksumValue": "48b912f610627546cfc30af0f974745a1bf7c30f"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/fstab", "SPDXID": "SPDXRef-1b4e9bb", "checksums": [{"algorithm": "SHA1", "checksumValue": "d50ee135ef10a434b9df582ea8276b5c1ce803fa"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/services", "SPDXID": "SPDXRef-a21ebf0", "checksums": [{"algorithm": "SHA1", "checksumValue": "a0d7a229bf049f7fe17e8445226236e4024535d0"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/modprobe.d/kms.conf", "SPDXID": "SPDXRef-7ba46d2", "checksums": [{"algorithm": "SHA1", "checksumValue": "ca76cb9f71980e9bda8db6bf95da759e26b27a88"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/modprobe.d/aliases.conf", "SPDXID": "SPDXRef-33969a0", "checksums": [{"algorithm": "SHA1", "checksumValue": "5946e1e930583552bb7b863eb94bcbb3feef8aa9"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/modprobe.d/i386.conf", "SPDXID": "SPDXRef-a3012c9", "checksums": [{"algorithm": "SHA1", "checksumValue": "a676b2fe78e7ea897d702b2c2fb2a2659f1eb657"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/modprobe.d/blacklist.conf", "SPDXID": "SPDXRef-a3e9b71", "checksums": [{"algorithm": "SHA1", "checksumValue": "e1376014791376ddee402f8d06dae7b4e9e6f67e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/alpine-release", "SPDXID": "SPDXRef-d424e27", "checksums": [{"algorithm": "SHA1", "checksumValue": "3605fa447e4623f5ff4a6adc97b1fde9a257b8f2"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/udhcpd.conf", "SPDXID": "SPDXRef-66be5cb", "checksums": [{"algorithm": "SHA1", "checksumValue": "1202c58e3ebba2edde32aa789b7af66639d0ed05"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/inittab", "SPDXID": "SPDXRef-5a833f1", "checksums": [{"algorithm": "SHA1", "checksumValue": "4ecb616e15bb4335917b513f34ac133ae0f8a477"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/ssl/openssl.cnf", "SPDXID": "SPDXRef-5a8796f", "checksums": [{"algorithm": "SHA1", "checksumValue": "2765f580ad4b51233ca5d8fc0ea7b67819204a51"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/ssl/openssl.cnf.dist", "SPDXID": "SPDXRef-91fe54c", "checksums": [{"algorithm": "SHA1", "checksumValue": "2765f580ad4b51233ca5d8fc0ea7b67819204a51"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/ssl/misc/CA.pl", "SPDXID": "SPDXRef-cdb85a6", "checksums": [{"algorithm": "SHA1", "checksumValue": "99ca8b6ceea241ef139900a8443a3315649c8ac4"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-71aedc7"], "noticeText": "Copyright 2000-2021 The OpenSSL Project Authors\n"}, {"fileName": "etc/ssl/misc/tsget.pl", "SPDXID": "SPDXRef-2fc2f4e", "checksums": [{"algorithm": "SHA1", "checksumValue": "ba3b722ff39b9200698043272f2a67d1e94dbcff"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-71aedc7"], "noticeText": "Copyright 2002-2021 The OpenSSL Project Authors\nCopyright (c) 2002 The OpenTSA Project\n"}, {"fileName": "etc/ssl/ct_log_list.cnf", "SPDXID": "SPDXRef-153c40c", "checksums": [{"algorithm": "SHA1", "checksumValue": "a2587c4e97408b64274e5e052b74e3754892c13a"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/ssl/ct_log_list.cnf.dist", "SPDXID": "SPDXRef-146c4cf", "checksums": [{"algorithm": "SHA1", "checksumValue": "a2587c4e97408b64274e5e052b74e3754892c13a"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/ssl/certs/ca-certificates.crt", "SPDXID": "SPDXRef-c484e7b", "checksums": [{"algorithm": "SHA1", "checksumValue": "b132b312a42c8be5d632069aecc6797b629f1264"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/group", "SPDXID": "SPDXRef-0ec35be", "checksums": [{"algorithm": "SHA1", "checksumValue": "dcafa89498396b2cc7495354920819257b81fbd6"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/modules", "SPDXID": "SPDXRef-b76c387", "checksums": [{"algorithm": "SHA1", "checksumValue": "b68a208d48a91c670c8040a03c95fae12c144f53"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/arch", "SPDXID": "SPDXRef-ea641f2", "checksums": [{"algorithm": "SHA1", "checksumValue": "a8fccbd2ab32f4aa628ac5c7704e8e4767eea0fb"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/keys/[email protected]", "SPDXID": "SPDXRef-33b3640", "checksums": [{"algorithm": "SHA1", "checksumValue": "3671ae0ec7503b1e193587c1dcdf7b78bc863e42"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/keys/[email protected]", "SPDXID": "SPDXRef-999a530", "checksums": [{"algorithm": "SHA1", "checksumValue": "95995311236b7a55933642ffa10ce6014f1af7d0"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/keys/[email protected]", "SPDXID": "SPDXRef-496a21c", "checksums": [{"algorithm": "SHA1", "checksumValue": "3af08548ef78cfdedcf349880c2c6a1a48763a0e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/keys/[email protected]", "SPDXID": "SPDXRef-8ac6419", "checksums": [{"algorithm": "SHA1", "checksumValue": "bfb616658cc05a872568b0c8e398c482e23b60dd"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/keys/[email protected]", "SPDXID": "SPDXRef-7bda380", "checksums": [{"algorithm": "SHA1", "checksumValue": "58d5ba4b2f3b1e927721d7a6432f298eedf72a6b"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/world", "SPDXID": "SPDXRef-686b678", "checksums": [{"algorithm": "SHA1", "checksumValue": "41af545d541d0739bd05f113f693c0e1d68b8e9f"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/protected_paths.d/alpine-release.list", "SPDXID": "SPDXRef-5f3baa7", "checksums": [{"algorithm": "SHA1", "checksumValue": "26db008ad787eb4132591e91e43396141af17167"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/apk/repositories", "SPDXID": "SPDXRef-aa963ea", "checksums": [{"algorithm": "SHA1", "checksumValue": "7c2de9e135334080567f1d185e9c66364f030ca2"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/issue", "SPDXID": "SPDXRef-eb6a093", "checksums": [{"algorithm": "SHA1", "checksumValue": "755ea6503fa431b4b3320e2e781cb29966599924"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/securetty", "SPDXID": "SPDXRef-bf59c5b", "checksums": [{"algorithm": "SHA1", "checksumValue": "981f791ead8d513679f7d443892b23f70e45ace5"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/nsswitch.conf", "SPDXID": "SPDXRef-e0cb6b4", "checksums": [{"algorithm": "SHA1", "checksumValue": "f4306c327bf44767da8da4e3a13bf40bdd4d3aaa"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/shadow", "SPDXID": "SPDXRef-78e444d", "checksums": [{"algorithm": "SHA1", "checksumValue": "258ab61690a3d3c96c2447483f55c6761ed21b01"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/hostname", "SPDXID": "SPDXRef-9637d53", "checksums": [{"algorithm": "SHA1", "checksumValue": "ea75706155cffed0a1bd43ddba4543da27d73a67"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/network/if-up.d/dad", "SPDXID": "SPDXRef-df56d8a", "checksums": [{"algorithm": "SHA1", "checksumValue": "3917fe94f44ab9881d90105c2a87af475b7ad10e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"], "noticeText": "Copyright (c) 2016-2018 Kaarle Ritvanen\n"}, {"fileName": "etc/profile.d/color_prompt.sh.disabled", "SPDXID": "SPDXRef-98ee4a4", "checksums": [{"algorithm": "SHA1", "checksumValue": "d5733d99d7b5676f6d58c19a3a47a8bc3fe6e2e5"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/profile.d/20locale.sh", "SPDXID": "SPDXRef-ebe9b09", "checksums": [{"algorithm": "SHA1", "checksumValue": "96adbd950ccf992085295990f9bbe667f0cf4c4e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/profile.d/README", "SPDXID": "SPDXRef-badb010", "checksums": [{"algorithm": "SHA1", "checksumValue": "df9396b02cf3be70767e6171eb691baa6d40c759"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/crontabs/root", "SPDXID": "SPDXRef-e7ea5c6", "checksums": [{"algorithm": "SHA1", "checksumValue": "bdf9356a9516238c8b2468613517749098b17ef6"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/profile", "SPDXID": "SPDXRef-3bc3290", "checksums": [{"algorithm": "SHA1", "checksumValue": "844cab296bb258be8d3190a4a5c276ce146455fe"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/protocols", "SPDXID": "SPDXRef-37a98ca", "checksums": [{"algorithm": "SHA1", "checksumValue": "d5f9654539089b96f1b1956848d783527da6fb47"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/logrotate.d/acpid", "SPDXID": "SPDXRef-e49126a", "checksums": [{"algorithm": "SHA1", "checksumValue": "4f29720883559a74be03f4de69de2f66113b064b"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/passwd", "SPDXID": "SPDXRef-614e0ba", "checksums": [{"algorithm": "SHA1", "checksumValue": "4dc86eb8b51fbabd22cef7d9419c6037f2c9841f"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/busybox-paths.d/busybox", "SPDXID": "SPDXRef-7881d4d", "checksums": [{"algorithm": "SHA1", "checksumValue": "d512bba3b8a5c9dfe6a033e919f5af0ef1ced993"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "etc/hosts", "SPDXID": "SPDXRef-a77a4a7", "checksums": [{"algorithm": "SHA1", "checksumValue": "043eb324a653456caa1a73e2e2d49f77792bb0c5"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "sbin/ldconfig", "SPDXID": "SPDXRef-045922d", "checksums": [{"algorithm": "SHA1", "checksumValue": "2a36b6f8f3992b112450e66ac128c2ea499a103e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "sbin/apk", "SPDXID": "SPDXRef-7248e09", "checksums": [{"algorithm": "SHA1", "checksumValue": "d3f68d36dcb01397bf3b209155f948fbbf28a4d8"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "bin/busybox", "SPDXID": "SPDXRef-7672107", "checksums": [{"algorithm": "SHA1", "checksumValue": "3a62e123dc3b8382eef9614ade91b44b5b915ee3"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["LicenseRef-a7811ea"], "noticeText": "copyrighted by many authors\n(c) 2018 Gavin D. Howard and contributors\n"}, {"fileName": "usr/lib/ossl-modules/legacy.so", "SPDXID": "SPDXRef-ff732b8", "checksums": [{"algorithm": "SHA1", "checksumValue": "debcff716af214f91aaaa887387f03667f5af9c3"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/lib/engines-3/capi.so", "SPDXID": "SPDXRef-5d3d492", "checksums": [{"algorithm": "SHA1", "checksumValue": "a765698f6fb1eeed57efe6d699f19a9175c1024c"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/lib/engines-3/loader_attic.so", "SPDXID": "SPDXRef-554cb0f", "checksums": [{"algorithm": "SHA1", "checksumValue": "b06288921183f2e09cfb3e7e10c5e6db180b4c62"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/lib/engines-3/padlock.so", "SPDXID": "SPDXRef-ca09358", "checksums": [{"algorithm": "SHA1", "checksumValue": "835cc154623e1cd640c28a1fd5f1e46447c2800e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"], "fileContributors": ["<[email protected]>"]}, {"fileName": "usr/lib/engines-3/afalg.so", "SPDXID": "SPDXRef-d7e7f1c", "checksums": [{"algorithm": "SHA1", "checksumValue": "4e4e66f72660f148561ea4ed30bdce982e3c3121"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/bin/iconv", "SPDXID": "SPDXRef-fd00e48", "checksums": [{"algorithm": "SHA1", "checksumValue": "16af49b4328cf786a0c824f2d221cfa54a215601"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/bin/ldd", "SPDXID": "SPDXRef-f2411fa", "checksums": [{"algorithm": "SHA1", "checksumValue": "c850211a08262fb11181b200eca431c93cdfde4b"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/bin/scanelf", "SPDXID": "SPDXRef-4840548", "checksums": [{"algorithm": "SHA1", "checksumValue": "6ca2e3fff9ad0d6816681e98f2e95206ee4c5d8e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/bin/ssl_client", "SPDXID": "SPDXRef-141f02a", "checksums": [{"algorithm": "SHA1", "checksumValue": "946753ca4928b91ed98ceb9f70bd971276a99d91"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/bin/getconf", "SPDXID": "SPDXRef-a4e24dd", "checksums": [{"algorithm": "SHA1", "checksumValue": "22d6923c1e2d23ddb98e09363e8e9006d660a358"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/bin/getent", "SPDXID": "SPDXRef-ad99cd4", "checksums": [{"algorithm": "SHA1", "checksumValue": "1e17a3622f873e8a7905cee63bfbddf8abc894bc"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["BINARY"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/udhcpc/default.script", "SPDXID": "SPDXRef-61eb6d2", "checksums": [{"algorithm": "SHA1", "checksumValue": "1d6a46dde403f14a22e2692cd84dd24af3805216"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["NONE"], "noticeText": "Copyright (c) 2008 Natanael Copa <[email protected]>\n"}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-7b6ca29", "checksums": [{"algorithm": "SHA1", "checksumValue": "57f6b93fda4a4496fab62844ddef0eeb168f80b5"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-b65f81c", "checksums": [{"algorithm": "SHA1", "checksumValue": "3529ec82670c6d4e20ee3e4968db34b551e91d50"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-472a5eb", "checksums": [{"algorithm": "SHA1", "checksumValue": "39ac5d72c6ba018a0f74b8b453894edc9db07b5f"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-55ef821", "checksums": [{"algorithm": "SHA1", "checksumValue": "3671ae0ec7503b1e193587c1dcdf7b78bc863e42"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-01882bf", "checksums": [{"algorithm": "SHA1", "checksumValue": "c8fabeb2eeb992c368c77b9707e0d1ecfd7cf905"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-c1b8cad", "checksums": [{"algorithm": "SHA1", "checksumValue": "053a92f87fd4532850bb31f0881978efe0532ae5"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-84951ea", "checksums": [{"algorithm": "SHA1", "checksumValue": "95995311236b7a55933642ffa10ce6014f1af7d0"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-43bef7f", "checksums": [{"algorithm": "SHA1", "checksumValue": "55a301064e11c6fe9ba0f2ca17e234f3943ccb61"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-af04a49", "checksums": [{"algorithm": "SHA1", "checksumValue": "3af08548ef78cfdedcf349880c2c6a1a48763a0e"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-dde0b0b", "checksums": [{"algorithm": "SHA1", "checksumValue": "de1241307014aae3dba798e900f163408d98d6f4"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-b653f98", "checksums": [{"algorithm": "SHA1", "checksumValue": "5d4743128353b6396fad2fa2ba793ace21602295"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-4a5cf31", "checksums": [{"algorithm": "SHA1", "checksumValue": "df02c9adc2906a3aa5e5ad69f50e3953e65710d0"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-24208ae", "checksums": [{"algorithm": "SHA1", "checksumValue": "825090fde25bbc0e71a9cb3076316b5afe459e4d"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-d865831", "checksums": [{"algorithm": "SHA1", "checksumValue": "bfb616658cc05a872568b0c8e398c482e23b60dd"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-1eb62bd", "checksums": [{"algorithm": "SHA1", "checksumValue": "58d5ba4b2f3b1e927721d7a6432f298eedf72a6b"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-be7d7c2", "checksums": [{"algorithm": "SHA1", "checksumValue": "329643357d0b78b1ef48ec155325e25f1d7534dd"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "usr/share/apk/keys/[email protected]", "SPDXID": "SPDXRef-d3d5fae", "checksums": [{"algorithm": "SHA1", "checksumValue": "23d0f2ea1af269c2f66165e0f8a944e96bf011de"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["TEXT"], "licenseInfoInFiles": ["NONE"]}, {"fileName": "data/gpl3.py", "SPDXID": "SPDXRef-d08dd5c", "checksums": [{"algorithm": "SHA1", "checksumValue": "fda9a7131498297d7414ec00ab86d3bbbf9cdb47"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-bdceeb6", "LicenseRef-5fc4e2a"]}, {"fileName": "data/gpl2.c", "SPDXID": "SPDXRef-e4639c1", "checksums": [{"algorithm": "SHA1", "checksumValue": "67c8fccceead37ecc22cf878aefcf83023962016"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-a7811ea", "LicenseRef-5fc4e2a"]}, {"fileName": "data/apache2.go", "SPDXID": "SPDXRef-ba70e82", "checksums": [{"algorithm": "SHA1", "checksumValue": "ca553b51ad4d5cc13eed66de3147f03f07af3b74"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-8ba26df"]}, {"fileName": "run.sh", "SPDXID": "SPDXRef-f937350", "checksums": [{"algorithm": "SHA1", "checksumValue": "be87e6a9c6283987ce437172ed37013ae77c5808"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-b9d3bc5"]}, {"fileName": "run.sh", "SPDXID": "SPDXRef-cde5ad2", "checksums": [{"algorithm": "SHA1", "checksumValue": "be87e6a9c6283987ce437172ed37013ae77c5808"}], "licenseConcluded": "NOASSERTION", "copyrightText": "NOASSERTION", "fileTypes": ["SOURCE"], "licenseInfoInFiles": ["LicenseRef-b9d3bc5"]}], "hasExtractedLicensingInfos": [{"extractedText": "OpenSSL/SSLeay License", "licenseId": "LicenseRef-71aedc7"}, {"extractedText": "GPL 2.0 or later", "licenseId": "LicenseRef-6ddea61"}, {"extractedText": "GPL 3.0 or later", "licenseId": "LicenseRef-bdceeb6"}, {"extractedText": "MPL 2.0", "licenseId": "LicenseRef-96e2b0f"}, {"extractedText": "BSD-2-Clause", "licenseId": "LicenseRef-2730428"}, {"extractedText": "GPL 1.0 or later", "licenseId": "LicenseRef-5fc4e2a"}, {"extractedText": "Apache 2.0", "licenseId": "LicenseRef-8ba26df"}, {"extractedText": "BSD-3-Clause", "licenseId": "LicenseRef-da38037"}, {"extractedText": "GPL 2.0", "licenseId": "LicenseRef-a7811ea"}, {"extractedText": "MIT License", "licenseId": "LicenseRef-b9d3bc5"}]}

Was able to get it to work. I hope that helps.

@rnjudge
Copy link
Contributor

rnjudge commented Jun 1, 2023

Hi @Jeeppler can you test this fix with my clone of Tern? This fix is not in a release so you will have to use my local changes... https://github.com/rnjudge/tern/tree/issue-1202.

In my tests the issue you were seeing is now fixed.

@rnjudge
Copy link
Contributor

rnjudge commented Jun 1, 2023

Once I get confirmation from you that it's fixed, I'll cut a release.

@Jeeppler
Copy link
Author

Jeeppler commented Jun 2, 2023

Failed using Tern and Scancode 32.0.2

Details:

To install Tern I used the following commands in a container:

RUN git clone https://github.com/rnjudge/tern.git && \
    cd tern && \
    git checkout issue-1202 && \
    pip install -e .

If I issued the tern --version command I received:

tern --version
Tern version 0.3.1
   python version = 3.9.2 (default, Feb 28 2021, 17:03:44) 

Scancode:

scancode --version
ScanCode version: 32.0.2
ScanCode Output Format version: 3.0.0
SPDX License list version: 3.20

error:

2023-06-02 20:26:59,801 - DEBUG - __main__ - Starting...
2023-06-02 20:26:59,801 - DEBUG - prep - Setting up...
2023-06-02 20:26:59,802 - DEBUG - rootfs - Running command: chmod +x /pds/tern/tern/tools/fs_hash.sh
2023-06-02 20:26:59,808 - DEBUG - run - Starting analysis...
2023-06-02 20:26:59,809 - DEBUG - rootfs - Running command: tar -tf /workspace/workspace/f1c71bca-66d0-4b32-a6db-c1cbab2f0057/upload/extracted/binaries/sechub-test-alpine.tar
2023-06-02 20:26:59,810 - DEBUG - rootfs - Running command: tar -x -f /workspace/workspace/f1c71bca-66d0-4b32-a6db-c1cbab2f0057/upload/extracted/binaries/sechub-test-alpine.tar -C /root/.tern/temp
2023-06-02 20:26:59,818 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/2f1efefa42dbd6a8375b45c97e21453d2de1343a814ecaf1e098eff5a18a1e62/layer.tar
2023-06-02 20:26:59,820 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/2f1efefa42dbd6a8375b45c97e21453d2de1343a814ecaf1e098eff5a18a1e62/layer.tar -C /root/.tern/temp/2f1efefa42dbd6a8375b45c97e21453d2...
2023-06-02 20:27:14.779  INFO 8 --- [pool-2-thread-1] c.m.sechub.pds.job.PDSWorkspaceService   : Removed workspace folder for job f1c71bca-66d0-4b32-a6db-c1cbab2f0057
2023-06-02 20:27:14.781  INFO 8 --- [pool-2-thread-1] c.m.s.s.s.spring.SharedVolumeJobStorage  : deleted all inside /shared_volumes/uploads/pds/TERN_CLUSTER/f1c71bca-66d0-4b32-a6db-c1cbab2f0057
2023-06-02 20:27:14.781  INFO 8 --- [pool-2-thread-1] c.m.sechub.pds.job.PDSWorkspaceService   : Removed storage for job f1c71bca-66d0-4b32-a6db-c1cbab2f0057
2023-06-02 20:27:14.781  INFO 8 --- [pool-2-thread-1] c.m.s.p.execution.PDSExecutionCallable   : Finished execution of job f1c71bca-66d0-4b32-a6db-c1cbab2f0057 with exitCode=1, failed=true, cancelOperationsHasBeenStarted=false
2023-06-02 20:27:14.782  INFO 8 --- [pool-2-thread-1] c.m.s.p.execution.PDSExecutionCallable   : Job error stream = in $pkgs; do apk -a info $p 2>/dev/null | tail -2 | head -1; done
2023-06-02 20:27:00,098 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`apk info 2>/dev/null` && for p in $pkgs; do apk -a info $p --print-arch; done
2023-06-02 20:27:00,120 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`apk info 2>/dev/null` && for p in $pkgs; do files=`apk -L info $p 2>/dev/null`; for file in $files; do if [ -f $file ]; then echo $file; fi; done; echo LICF; done
2023-06-02 20:27:00,176 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`apk info 2>/dev/null` && for p in $pkgs; do apk info $p 2>/dev/null | head -5 | tail -1; done
2023-06-02 20:27:00,226 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c apk info 2>/dev/null
2023-06-02 20:27:00,231 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`apk info 2>/dev/null` && for p in $pkgs; do lic=`apk info $p 2>/dev/null | head -1 | awk '{print $1}'`; echo $lic | sed -e "s/^$p-//"; done
2023-06-02 20:27:00,290 - WARNING - core - Some metadata may be missing
2023-06-02 20:27:00,309 - DEBUG - rootfs - Running command: /usr/local/bin/scancode -ilpcu --quiet --timeout 300 -n 5 --json - /root/.tern/temp/2f1efefa42dbd6a8375b45c97e21453d2de1343a814ecaf1e098eff5a18a1e62/contents
Traceback (most recent call last):
  File "/usr/local/bin/tern", line 10, in <module>
    sys.exit(main())
  File "/pds/tern/tern/__main__.py", line 311, in main
    do_main(args)
  File "/pds/tern/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/pds/tern/tern/analyze/default/container/run.py", line 80, in execute_image
    cimage.default_analyze(full_image, args)
  File "/pds/tern/tern/analyze/default/container/image.py", line 71, in default_analyze
    passthrough.run_extension_layer(image_obj.layers[0], options.extend,
  File "/pds/tern/tern/analyze/passthrough.py", line 92, in run_extension_layer
    return mgr.driver.execute_layer(image_layer, redo)
  File "/pds/tern/tern/extensions/scancode/executor.py", line 240, in execute_layer
    file_list, package_list = collect_layer_data(image_layer)
  File "/pds/tern/tern/extensions/scancode/executor.py", line 177, in collect_layer_data
    files.append(get_scancode_file(f))
  File "/pds/tern/tern/extensions/scancode/executor.py", line 61, in get_scancode_file
    if file_dict['licenses']:
KeyError: 'licenses'

@Jeeppler
Copy link
Author

Jeeppler commented Jun 2, 2023

Failed with Tern and Scancode 31.2.6.

Details:

Installation same as before.

Scancode:

scancode --version
ScanCode version: 31.2.6
ScanCode Output Format version: 2.0.0
SPDX License list version: 3.18

error:

2023-06-02 20:36:05,641 - DEBUG - __main__ - Starting...
2023-06-02 20:36:05,641 - DEBUG - prep - Setting up...
2023-06-02 20:36:05,641 - DEBUG - rootfs - Running command: chmod +x /pds/tern/tern/tools/fs_hash.sh
2023-06-02 20:36:05,649 - DEBUG - run - Starting analysis...
2023-06-02 20:36:05,649 - DEBUG - rootfs - Running command: tar -tf /workspace/workspace/8813677f-bd33-4156-9f2e-b081115d9881/upload/extracted/binaries/sechub-test-alpine.tar
2023-06-02 20:36:05,651 - DEBUG - rootfs - Running command: tar -x -f /workspace/workspace/8813677f-bd33-4156-9f2e-b081115d9881/upload/extracted/binaries/sechub-test-alpine.tar -C /root/.tern/temp
2023-06-02 20:36:05,659 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/2f1efefa42dbd6a8375b45c97e21453d2de1343a814ecaf1e098eff5a18a1e62/layer.tar
2023-06-02 20:36:05,661 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/2f1efefa42dbd6a8375b45c97e21453d2de1343a814ecaf1e098eff5a18a1e62/layer.tar -C /root/.tern/temp/2f1efefa42dbd6a8375b45c97e21453d2...
2023-06-02 20:36:29.400  INFO 8 --- [pool-2-thread-1] c.m.sechub.pds.job.PDSWorkspaceService   : Removed workspace folder for job 8813677f-bd33-4156-9f2e-b081115d9881
2023-06-02 20:36:29.402  INFO 8 --- [pool-2-thread-1] c.m.s.s.s.spring.SharedVolumeJobStorage  : deleted all inside /shared_volumes/uploads/pds/TERN_CLUSTER/8813677f-bd33-4156-9f2e-b081115d9881
2023-06-02 20:36:29.402  INFO 8 --- [pool-2-thread-1] c.m.sechub.pds.job.PDSWorkspaceService   : Removed storage for job 8813677f-bd33-4156-9f2e-b081115d9881
2023-06-02 20:36:29.402  INFO 8 --- [pool-2-thread-1] c.m.s.p.execution.PDSExecutionCallable   : Finished execution of job 8813677f-bd33-4156-9f2e-b081115d9881 with exitCode=1, failed=true, cancelOperationsHasBeenStarted=false
2023-06-02 20:36:29.403  INFO 8 --- [pool-2-thread-1] c.m.s.p.execution.PDSExecutionCallable   : Job error stream = sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && distro=`/bin/cat /etc/os-release | grep NAME | sed -n '1p' | cut -f 2 -d '=' | cut -d '"' -f2` && pkgs=`apk info 2>/dev/null` && for p in $pkgs; do echo $distro; done
2023-06-02 20:36:29,030 - WARNING - core - Some metadata may be missing
2023-06-02 20:36:29,034 - DEBUG - rootfs - Running command: cp -r /root/.tern/temp/00e1169787a9be8d6643cae38a68aa2ec13519ad8e43b7f8a15f211b45f41868/contents/etc /root/.tern/temp/00e1169787a9be8d6643cae38a68aa2ec13519ad8e43b7f8a15f211b45f41868/contents/run.sh /root/.tern/temp/mergedir
2023-06-02 20:36:29,066 - DEBUG - generator - Generating SPDX JSON document...
2023-06-02 20:36:29,076 - CRITICAL - spdx_common - File lib/apk/db/lock does not have a sha1 checksum. Failed to generate a SPDX tag-value report
Traceback (most recent call last):
  File "/usr/local/bin/tern", line 10, in <module>
    sys.exit(main())
  File "/pds/tern/tern/__main__.py", line 311, in main
    do_main(args)
  File "/pds/tern/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/pds/tern/tern/analyze/default/container/run.py", line 82, in execute_image
    report.report_out(args, full_image)
  File "/pds/tern/tern/report/report.py", line 86, in report_out
    report = generate_report(args, *images)
  File "/pds/tern/tern/report/report.py", line 41, in generate_report
    return generate_format(
  File "/pds/tern/tern/report/report.py", line 56, in generate_format
    return mgr.driver.generate(images, print_inclusive)
  File "/pds/tern/tern/formats/spdx/spdxjson/generator.py", line 162, in generate
    report = get_document_dict(image_obj, template)
  File "/pds/tern/tern/formats/spdx/spdxjson/generator.py", line 73, in get_document_dict
    pkgs_dict_list = phelpers.get_packages_list(image_obj, template)
  File "/pds/tern/tern/formats/spdx/spdxjson/package_helpers.py", line 107, in get_packages_list
    package_dicts.append(get_package_dict(package, template))
  File "/pds/tern/tern/formats/spdx/spdxjson/package_helpers.py", line 82, in get_package_dict
    if spdx_common.get_purl(package):
  File "/pds/tern/tern/formats/spdx/spdx_common.py", line 245, in get_purl
    purl = PackageURL(purl_type, purl_namespace, package_obj.name.lower(), package_obj.version,
  File "/usr/local/lib/python3.9/dist-packages/packageurl/__init__.py", line 375, in __new__
    raise ValueError(f"Invalid purl: {key} is a required argument.")
ValueError: Invalid purl: type is a required argument.

The error in Scancode 31.2.6 differs from Scancode 32.0.2.

@rnjudge
Copy link
Contributor

rnjudge commented Jun 5, 2023

@Jeeppler thanks. Can you provide the image you're working with so I can reproduce? The alpine base image doesn't fail.

@Jeeppler
Copy link
Author

Jeeppler commented Jun 6, 2023

Image:

image.zip

Tern command used:

tern report -f spdxjson -w "sechub-test-alpine.tar" -o "my_result.spdx.json" -x scancode

Alpine Dockerfile:

# SPDX-License-Identifier: MIT

FROM alpine:3.18

ENV DATA="/data"

COPY data "$DATA"

COPY shared/run.sh /run.sh
RUN chmod +x /run.sh

CMD ["/run.sh"]

Data is a folder with some source files with licenses annotations: https://github.com/mercedes-benz/sechub/tree/develop/sechub-pds-solutions/tern/tests/data

run.sh

#!/usr/bin/env sh
# SPDX-License-Identifier: MIT

while true
do
  echo "Press [CTRL+C] to stop.."
  sleep 120
done

In general, you can find all the test images/files for the containers here: https://github.com/mercedes-benz/sechub/tree/develop/sechub-pds-solutions/tern/tests.

@armintaenzertng
Copy link

I was able to reproduce the issue by using the example commands from the readme. That is,

python3 -m venv scanenv
cd scanenv/
source bin/activate
pip install tern scancode-toolkit
tern report -x scancode -i golang:1.12-alpine

results in

Traceback (most recent call last):
  File "/home/armin/python_venvs/scanenv/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/__main__.py", line 311, in main
    do_main(args)
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/analyze/default/container/run.py", line 80, in execute_image
    cimage.default_analyze(full_image, args)
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/analyze/default/container/image.py", line 71, in default_analyze
    passthrough.run_extension_layer(image_obj.layers[0], options.extend,
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/analyze/passthrough.py", line 92, in run_extension_layer
    return mgr.driver.execute_layer(image_layer, redo)
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/extensions/scancode/executor.py", line 219, in execute_layer
    file_list, package_list = collect_layer_data(image_layer)
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/extensions/scancode/executor.py", line 162, in collect_layer_data
    files.append(get_scancode_file(f))
  File "/home/armin/python_venvs/scanenv/lib/python3.10/site-packages/tern/extensions/scancode/executor.py", line 61, in get_scancode_file
    if file_dict['licenses']:
KeyError: 'licenses'

@rnjudge
Copy link
Contributor

rnjudge commented Jun 29, 2023

I was able to reproduce the issue by using the example commands from the readme. That is,

python3 -m venv scanenv
cd scanenv/
source bin/activate
pip install tern scancode-toolkit
tern report -x scancode -i golang:1.12-alpine

Ok, working on this. What version of scancode are you running here? I can't reproduce the error with 31.2.6 and my latest changes, but let me try upgrading to 32.0 to see if they changed their data model again.

UPDATE: I can reproduce with 32.0.4 so scancode dictionary key names have changed again. Sigh.

rnjudge added a commit to rnjudge/tern that referenced this issue Jun 29, 2023
Scancode v31.0.0 includes changes[1] to JSON output attribute
names which was causing processing KeyErrors when Tern would run with
Scancode. Scancode v32.0.0 also includes changes[2] to license_detection
output which was similarly causing parsing KeyErrors when Tern ran
with Scancode. This commit adds code that can accomodate the new
attribute property names in the newer versions of Scancode, as well as
the older value names (in case we have users still using older Scancode
versions). At some point in the future, it probably makes sense to
re-visit some of these changes and see if we want to continue to support
older versions of scancode.

[1]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#v3100---2022-08-17

[2]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#license-detection

Resolves tern-tools#1202

Signed-off-by: Rose Judge <[email protected]>
@rnjudge
Copy link
Contributor

rnjudge commented Jun 29, 2023

Confirmed that there was a big change to license_detection reporting which was causing the error difference in v31 vs v32.

@Jeeppler @armintaenzertng Hopefully this fixes all the errors you are seeing. Please give it a try: https://github.com/rnjudge/tern/tree/issue-1202

I still have some parsing to fix from the scancode updates but if it fixes your issues I'd like to get a new release cut with the fixes.

@armintaenzertng
Copy link

Thanks, @rnjudge, it works now with the branch you provided! :)

rnjudge added a commit to rnjudge/tern that referenced this issue Jul 11, 2023
Scancode v31.0.0 includes changes[1] to JSON output attribute
names which was causing processing KeyErrors when Tern would run with
Scancode. Scancode v32.0.0 also includes changes[2] to license_detection
output which was similarly causing parsing KeyErrors when Tern ran
with Scancode. This commit adds code that can accomodate the new
attribute property names in the newer versions of Scancode, as well as
the older value names (in case we have users still using older Scancode
versions). At some point in the future, it probably makes sense to
re-visit some of these changes and see if we want to continue to support
older versions of scancode.

[1]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#v3100---2022-08-17

[2]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#license-detection

Resolves tern-tools#1202

Signed-off-by: Rose Judge <[email protected]>
rnjudge added a commit to rnjudge/tern that referenced this issue Jul 11, 2023
Scancode v31.0.0 includes changes[1] to JSON output attribute
names which was causing processing KeyErrors when Tern would run with
Scancode. Scancode v32.0.0 also includes changes[2] to license_detection
output which was similarly causing parsing KeyErrors when Tern ran
with Scancode. This commit adds code that can accomodate the new
attribute property names in the newer versions of Scancode, as well as
the older value names (in case we have users still using older Scancode
versions). At some point in the future, it probably makes sense to
re-visit some of these changes and see if we want to continue to support
older versions of scancode.

This commit also has small changes that updated the README instructions
for how to install newer Scancode versions on M1/ARM hardware and also
fixes a small bug that was causing purl generation to fail when Scancode
doesn't detect a package format.

[1]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#v3100---2022-08-17

[2]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#license-detection

Resolves tern-tools#1202

Signed-off-by: Rose Judge <[email protected]>
rnjudge added a commit that referenced this issue Jul 13, 2023
Scancode v31.0.0 includes changes[1] to JSON output attribute
names which was causing processing KeyErrors when Tern would run with
Scancode. Scancode v32.0.0 also includes changes[2] to license_detection
output which was similarly causing parsing KeyErrors when Tern ran
with Scancode. This commit adds code that can accomodate the new
attribute property names in the newer versions of Scancode, as well as
the older value names (in case we have users still using older Scancode
versions). At some point in the future, it probably makes sense to
re-visit some of these changes and see if we want to continue to support
older versions of scancode.

This commit also has small changes that updated the README instructions
for how to install newer Scancode versions on M1/ARM hardware and also
fixes a small bug that was causing purl generation to fail when Scancode
doesn't detect a package format.

[1]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#v3100---2022-08-17

[2]https://github.com/nexB/scancode-toolkit/blob/e3099637b195daca54942df9f695f58990097896/CHANGELOG.rst#license-detection

Resolves #1202

Signed-off-by: Rose Judge <[email protected]>
@Jeeppler
Copy link
Author

@rnjudge thanks for working on this.

@Jeeppler
Copy link
Author

Will there be a new release soon?

@rnjudge
Copy link
Contributor

rnjudge commented Jul 14, 2023

@Jeeppler Working on this today. Should be out EOD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something went wrong
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants