Updating platform jobs for pluggable scm #30

dsingh07 · 2017-01-24T17:58:03Z

This pull request will update miscellaneous platform jobs to support Pluggable SCM:

A new platform job to deploy the pluggable scm library in your Job DSL additional classpath
Updates to Generate_Project to allow you to add a new parameter to Load_Cartridge, letting you set a custom namespace for your repos to be loaded under
Updates to Load_Cartridge with a new parameter for SCM_PROVIDER and making it completely agnostic of your chosen SCM provider i.e. no references to Gerrit
Supports running load cartridge and masters and unix/linux slaves.

Test Scenarios

Backwards compatibility: Load current Java Cartridge
** Expect spring petclinic repos to be created in Gerrit
Backwards compatibility: Load current DOA Cartridge Collection

Test Gerrit SCM Provider with SSH (Java Cartridge)
Test Gerrit SCM Provider with SSH and without a cartridge folder name
Test Gerrit SCM Provider with HTTPS (Java Cartridge)
Test Gerrit SCM Provider with HTTPS and without a cartridge folder name (Java Cartridge)

As the build label is also set to !master the above tests should also confirm we can run the cartridge loader from unix/linux based slaves.

For all tests it is expected the respective repositories captured in urls.txt will be created with the respective SCM provider (Gerrit, BitBucket)

This PR is dependent on:
~~Accenture/adop-jenkins#29~~
~~Accenture/adop-docker-compose#187~~
~~Accenture/adop-pluggable-scm#4~~

nickdgriffin · 2017-02-10T08:47:13Z

bootstrap/Platform_Management/Setup_Pluggable_Library.groovy

+// Setup setup_cartridge
+setupPluggable.with{
+    environmentVariables {
+        keepBuildVariables()


I think this needs a "true" in the brackets: https://jenkinsci.github.io/job-dsl-plugin/#path/freeStyleJob-environmentVariables-keepBuildVariables

nickdgriffin · 2017-02-10T08:47:28Z

bootstrap/Platform_Management/Setup_Pluggable_Library.groovy

+setupPluggable.with{
+    environmentVariables {
+        keepBuildVariables()
+        keepSystemVariables()


I think this needs a "true" in the brackets: https://jenkinsci.github.io/job-dsl-plugin/#path/freeStyleJob-environmentVariables-keepSystemVariables

dsingh07 · 2017-02-13T17:47:30Z

Thanks for the review @nickdgriffin

I have fixed that bug now.

Waiting for this PR to get merged in: dsingh07#1 to my branch, and this PR should be in a reviewable state again.

dsingh07 · 2017-02-22T17:58:07Z

There have been changes introduced by @BuleGeorge and @RobertNorthard which will be tested by me, after rebasing with all the other changes that have been introduced.

I shall update this PR with exciting developments :)

nickdgriffin · 2017-02-23T18:22:32Z

projects/jobs/jobs.groovy

+        systemGroovyCommand('''
+import jenkins.model.*
+import groovy.io.FileType
+>>>>>>> Updating platform jobs for pluggable scm


Can this be trimmed out please?

RobertNorthard · 2017-02-28T14:19:19Z

projects/jobs/jobs.groovy

+# Check if SCM namespace is specified
+if [ -z ${SCM_NAMESPACE} ] ; then
+    echo "SCM_NAMESPACE not specified, setting to PROJECT_NAME..."
+    SCM_NAMESPACE="${PROJECT_NAME}"


SCM_NAMESPACE="${PROJECT_NAME}/${CARTRIDGE_FOLDER}" if cartridge folder has been specified. If not I suspect it will just be empty and default to using the project name.

RobertNorthard · 2017-02-28T14:20:02Z

projects/jobs/jobs.groovy


 # Find the cartridge
 export CART_HOME=$(dirname $(find -name metadata.cartridge | head -1))
-
 echo "CART_HOME=${CART_HOME}" > ${WORKSPACE}/carthome.properties

 # Check if the user has enabled Gerrit Code reviewing


Please can we remove all Gerrit references.

RobertNorthard · 2017-02-28T14:28:01Z

projects/jobs/jobs.groovy

+
+def workspace = envVarProperty.getProperty('WORKSPACE')
+def projectFolderName = envVarProperty.getProperty('PROJECT_NAME')
+def cartridgeFolder = envVarProperty.getProperty('CARTRIDGE_FOLDER')


If CARTRIDGE_FOLDER is not provided we will get a missing property exception. I propose we do something similar to the SCM_NAMESPACE below and default it to '' if it's not provided.

dsingh07 · 2017-02-28T14:57:24Z

Thanks for the quick review @RobertNorthard

Have made those changes that you've requested.

RobertNorthard · 2017-02-28T17:08:28Z

Thanks @dsingh07.

@nickdgriffin @anton-kasperovich this requires one final look over (fresh pair of eyes) before we merge.

Signed-off-by: Robert A. Nothard <[email protected]>

Signed-off-by: Northard, Robert A <[email protected]>

dsingh07 · 2017-02-28T17:42:22Z

Commits have been squashed and is ready for a final round of testing from @nickdgriffin and @anton-kasperovich :)

RobertNorthard · 2017-03-01T20:30:16Z

projects/jobs/jobs.groovy

@@ -2,10 +2,20 @@
 def gerritBaseUrl = "ssh://jenkins@gerrit:29418"
 def cartridgeBaseUrl = gerritBaseUrl + "/cartridges"
 def platformToolsGitUrl = gerritBaseUrl + "/platform-management"
+def scmPropertiesPath = "${PLUGGABLE_SCM_PROVIDER_PROPERTIES_PATH}"


This variable looks like it's not used. Please can we remove.

Done! Thanks @RobertNorthard

RobertNorthard · 2017-03-08T12:33:48Z

Thanks @dsingh07 added a few comments in relation to error handling. After this PR is merged, we should also merge Accenture/adop-cartridge-skeleton#3 to demonstrate how to use the pluggable library.

RobertNorthard · 2017-03-08T15:59:36Z

projects/jobs/jobs.groovy

+
+  credentialInfo =  [CredentialsMatchers.firstOrNull(available_credentials, username_matcher).username,
+                  CredentialsMatchers.firstOrNull(available_credentials, username_matcher).password];
+


This will throw an unchecked exception (null pointer) if the credentials are not found.

java.lang.NullPointerException: Cannot get property 'username' on null object".

Please can we wrap the username/password extraction so if it throws a NPE a helpful warning message is displayed to the user and default the username/password to an empty string.

The job should not fail if credentials are not found beacuse:

the credentials are only required by the SCM create repo section of the cartridge loader

not all cartridges will utilise urls.txt as they are just using the SCM provider to generate the appropriate SCM get/triggers.

…arning message to indicate credentials have not been provided. Signed-off-by: Northard, Robert A <[email protected]>

@RobertNorthard

Changes made by @RobertNorthard

RobertNorthard · 2017-03-09T15:28:04Z

LGTM. Thanks everyone for the hard work.

@dsingh07 @nickdgriffin @anton-kasperovich @BuleGeorge @SachinKSingh28 @marisbahtins @IrmantasM

nickdgriffin reviewed Feb 10, 2017

View reviewed changes

dsingh07 force-pushed the feature/pluggable branch from 4e00f6f to baeb703 Compare February 13, 2017 11:04

dsingh07 added the wip label Feb 22, 2017

dsingh07 self-assigned this Feb 22, 2017

dsingh07 force-pushed the feature/pluggable branch 2 times, most recently from 4f48dbd to f3ebf7d Compare February 23, 2017 17:54

dsingh07 removed their assignment Feb 23, 2017

nickdgriffin reviewed Feb 23, 2017

View reviewed changes

dsingh07 force-pushed the feature/pluggable branch 4 times, most recently from 00741ee to 333de5f Compare February 27, 2017 17:54

dsingh07 removed the wip label Feb 27, 2017

RobertNorthard suggested changes Feb 28, 2017

View reviewed changes

RobertNorthard reviewed Feb 28, 2017

View reviewed changes

dsingh07 force-pushed the feature/pluggable branch 2 times, most recently from 885e3a0 to 86f3e22 Compare February 28, 2017 15:48

RobertNorthard approved these changes Feb 28, 2017

View reviewed changes

dsingh07 force-pushed the feature/pluggable branch from 86f3e22 to 975bdb6 Compare February 28, 2017 17:32

dharmanshu.singh and others added 6 commits February 28, 2017 17:38

Updating platform jobs for pluggable scm

642450b

Adding library setup to job runner + resolving method bug

73ac8a6

Update Load_Cartridge for Pluggable SCM BitBucket

571eea7

Support load cartridge SCM repo create on Jenkins slaves.

09d5a06

Signed-off-by: Robert A. Nothard <[email protected]>

Use secret file storage location and general tidy.

ff23717

Signed-off-by: Northard, Robert A <[email protected]>

Support running load cartridge on a slave.

4b1aec8

Updating load_cartridge to support bitbucket scm changes

b56ab4a

dsingh07 force-pushed the feature/pluggable branch from 975bdb6 to b56ab4a Compare February 28, 2017 17:39

dsingh07 added the enhancement label Feb 28, 2017

RobertNorthard suggested changes Mar 1, 2017

View reviewed changes

Removing unused variable

715cc2c

RobertNorthard approved these changes Mar 2, 2017

View reviewed changes

RobertNorthard mentioned this pull request Mar 2, 2017

Run generate workspace and project on slaves. #42

Open

Adding exception to ios and windows slaves

2bc79b5

RobertNorthard previously requested changes Mar 8, 2017

View reviewed changes

Catch unchecked null pointer exception by providing the user with a w…

b5ccea7

…arning message to indicate credentials have not been provided. Signed-off-by: Northard, Robert A <[email protected]>

dsingh07 force-pushed the feature/pluggable branch from d8af25d to b5ccea7 Compare March 9, 2017 14:35

RobertNorthard merged commit b90f071 into Accenture:master Mar 9, 2017

RobertNorthard mentioned this pull request Mar 9, 2017

Added pluggable SCM support. Accenture/adop-cartridge-skeleton#3

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updating platform jobs for pluggable scm #30

Updating platform jobs for pluggable scm #30

dsingh07 commented Jan 24, 2017 •

edited by RobertNorthard

Loading

nickdgriffin Feb 10, 2017

nickdgriffin Feb 10, 2017

dsingh07 commented Feb 13, 2017

dsingh07 commented Feb 22, 2017

nickdgriffin Feb 23, 2017

RobertNorthard Feb 28, 2017

RobertNorthard Feb 28, 2017

RobertNorthard Feb 28, 2017 •

edited

Loading

dsingh07 commented Feb 28, 2017

RobertNorthard commented Feb 28, 2017 •

edited

Loading

dsingh07 commented Feb 28, 2017

RobertNorthard Mar 1, 2017

dsingh07 Mar 2, 2017

RobertNorthard commented Mar 8, 2017 •

edited

Loading

RobertNorthard Mar 8, 2017 •

edited

Loading

RobertNorthard commented Mar 9, 2017 •

edited

Loading


		credentialInfo = [CredentialsMatchers.firstOrNull(available_credentials, username_matcher).username,
		CredentialsMatchers.firstOrNull(available_credentials, username_matcher).password];

Updating platform jobs for pluggable scm #30

Updating platform jobs for pluggable scm #30

Conversation

dsingh07 commented Jan 24, 2017 • edited by RobertNorthard Loading

nickdgriffin Feb 10, 2017

Choose a reason for hiding this comment

nickdgriffin Feb 10, 2017

Choose a reason for hiding this comment

dsingh07 commented Feb 13, 2017

dsingh07 commented Feb 22, 2017

nickdgriffin Feb 23, 2017

Choose a reason for hiding this comment

RobertNorthard Feb 28, 2017

Choose a reason for hiding this comment

RobertNorthard Feb 28, 2017

Choose a reason for hiding this comment

RobertNorthard Feb 28, 2017 • edited Loading

Choose a reason for hiding this comment

dsingh07 commented Feb 28, 2017

RobertNorthard commented Feb 28, 2017 • edited Loading

dsingh07 commented Feb 28, 2017

RobertNorthard Mar 1, 2017

Choose a reason for hiding this comment

dsingh07 Mar 2, 2017

Choose a reason for hiding this comment

RobertNorthard commented Mar 8, 2017 • edited Loading

RobertNorthard Mar 8, 2017 • edited Loading

Choose a reason for hiding this comment

RobertNorthard commented Mar 9, 2017 • edited Loading

dsingh07 commented Jan 24, 2017 •

edited by RobertNorthard

Loading

RobertNorthard Feb 28, 2017 •

edited

Loading

RobertNorthard commented Feb 28, 2017 •

edited

Loading

RobertNorthard commented Mar 8, 2017 •

edited

Loading

RobertNorthard Mar 8, 2017 •

edited

Loading

RobertNorthard commented Mar 9, 2017 •

edited

Loading