Reproducible Comparing jobs didn't show as part of dev.functional jenkins job

[sophia-guo]

Reproducible Comparing jobs were enabled as part of dev.functional, which didn't run in https://ci.adoptium.net/job/Test_openjdk21_hs_dev.functional_x86-64_linux/2/

Issues known:

• by default VENDOR_TEST_REPOS parameter of dev.functional job parameter VENDOR_TEST_REPOS is not set.
• dev.functional can't guarantee running on agent with 'docker installed'. Plus we can't force dev.functional running with docker installed only as ssl-tests, installler wintps tests don't or couldn't require docker installed.
• jq installed, but job complains jq: error: Could not open file : No such file or directory 13:23:46 jq x86_64 1.6-2.el7 epel 167 k - this one I met before can't remember the reason
• with dev.functional as target job failed with [make[1]: *** [settings.mk:356: dev.functional-..] Terminated](https://ci.adoptium.net/job/Test_openjdk21_hs_dev.functional_x86-64_linux/6/console) -- This error feels like intermittent, rerun on same machine and didn't get the error https://ci.adoptium.net/view/Test_functional/job/Test_openjdk21_hs_dev.functional_x86-64_linux/6/ https://ci.adoptium.net/view/Test_functional/job/Test_openjdk21_hs_dev.functional_x86-64_linux/8/

[sophia-guo]

Manually set the VENDOR_TEST_REPOS and VENDOR_TEST_BRANCH and tests still didn't run https://ci.adoptium.net/job/Test_openjdk21_hs_dev.functional_x86-64_linux/4/. Must be some other issue here.

[sophia-guo]

With VENDOR_TEST_DIRS tests are running VENDOR_TEST_DIRS. However dev.functional can't guarantee running on agent with 'docker installed'

[smlambert]

We should create a separate job for this test with the right parameters set (including ADDITIONAL_LABEL.

[sophia-guo]

Can you clarify the separate job? Under current naming convention could the new job output be consumed by TRSS? Or we also need to update TRSS for the new job?

[smlambert]

I think we can add it to the smoke test job with the following changes:

https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/common/openjdk_build_pipeline.groovy#L145 - change to dev,extended

https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/common/openjdk_build_pipeline.groovy#L148 to functional/buildAndPackage,functional/reproducibleCompare

Alternatively, move all smoke tests into dev

https://github.com/adoptium/temurin-build/blob/master/test/functional/buildAndPackage/playlist.xml#L21 and https://github.com/adoptium/temurin-build/blob/master/test/functional/buildAndPackage/playlist.xml#L52 to dev
https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/common/openjdk_build_pipeline.groovy#L145 - change to dev

---

Or,

Create a separate job, whatever name to call it, and start monitoring that job in TRSS (much like we have added AQA_Test_Pipeline for monitoring, downside of this approach would be that it is not hooked into the regular build pipelines.

[smlambert]

The [make[1]: *** [settings.mk:356: dev.functional-..] Terminated](https://ci.adoptium.net/job/Test_openjdk21_hs_dev.functional_x86-64_linux/6/console) issue mentioned above was related to a dockerhost node being labelled with ci.role.test. dockerhost nodes should not get that label, as they host several static test nodes, and if tests are run directly on dockerhost, then tests on the static nodes are randomly Terminated. (see discussion in Slack thread)

[sophia-guo]

As discussed those tests will move to system.dev category, which currently will only include those tests and hence resolve the docker requirement. Note this is a temp solution as going forward reproducible would not require to run in a docker container.

[smlambert]

dev.system - yes, also to move into the system group for the reason that a reproducibility test is more of a system test than a discreet functional test. It implies that many parts of an overall system are being tested, the contents of the SBOM are correct, and all components of the product source code and the tools used to build it remain in a state of reproducibility.