tests: udpate the github workflow to run tests suggested by spread-fi…

…lter tool (#14519)

* tests: udpate the github workflow to run tests suggested by spread-filter tool

Through this change we will run the tests suggested by the spread-filter
tool following the rules located in tests/lib/spread/rules which defined
for each system.

For nested tests, this ruleset will make sure that:
. Always that the nested.sh lib or the assertions dir are updated, all
the nested test are executed
 . When an specific nested test is updated, then this is executed
. If the "Run Nested" label is added to the PR, all the nested tests
will be executed as well.
 . In master branch, all nested test are executed
 . In other scenarios, nested tests are not executed.

Also, in master branch, all the tests are executed (the tests defined in
the matrix for each system).

* Fix the step "Setup run tests variable"

* add missing params to track changed files

* Fix issue when the RUN_TESTS has just spaces

This produced issues for example running all the nested tests when it is
not needed.

.github/labeler.yml

@@ -3,11 +3,6 @@
 # - https://github.com/actions/labeler/issues/112
 # - https://github.com/actions/labeler/issues/104
 
-# Add 'Run nested -auto-' label to either any change on nested lib or nested test
-Run nested -auto-:
-  - tests/lib/nested.sh
-  - tests/nested/**/*
-
 # Add 'Needs Documentation -auto-' label to indicate a change needs changes in the docs
 Needs Documentation -auto-:
   - cmd/snap/**/*"

.github/workflows/spread-tests.yaml

@@ -67,7 +67,12 @@ jobs:
 
           # Make sure the test results dirs are created
           # This step has to be after the cache is restored
-          mkdir -p "$TEST_RESULTS_DIR"        
+          mkdir -p "$TEST_RESULTS_DIR"
+
+    - name: Prepare nested env vars
+      if: contains(github.event.pull_request.labels.*.name, 'Run nested') && startsWith(matrix.group, 'nested-')
+      run: |
+          echo "RUN_NESTED=true" >> "$GITHUB_ENV"
 
     - name: Get changed files
       id: changed-files
@@ -102,20 +107,29 @@ jobs:
           if [ -n "$FAILED_TESTS" ]; then
               RUN_TESTS="$FAILED_TESTS"
           else
+              CHANGES_PARAM=""
+              for CHANGE in $CHANGED_FILES; do
+                  CHANGES_PARAM="$CHANGES_PARAM -c $CHANGE"
+              done
               for SYSTEM in ${{ inputs.systems }}; do
-                  for TESTS in ${{ inputs.tasks }}; do
-                    RUN_TESTS="$RUN_TESTS ${{ inputs.backend }}:$SYSTEM:$TESTS"
-                  done
-                  CHANGES_PARAM=""
-                  for CHANGE in $CHANGED_FILES; do
-                    CHANGES_PARAM="$CHANGES_PARAM -c $CHANGE"
-                  done
-                  SUGGESTED_TESTS="$SUGGESTED_TESTS $(./tests/lib/external/snapd-testing-tools/utils/spread-filter -r ./tests/lib/spread/rules/${{ inputs.rules }}.yaml -p "${{ inputs.backend }}:$SYSTEM" $CHANGES_PARAM)"
+                  # Configure parameters to run tests based on current changes
+                  # The tests are just filtered when the change is a PR
+                  # When 'Run Nested' label is added in a PR, all the nested tests have to be executed
+                  if [ -z "${{ github.event.number }}" ] || [ "$RUN_NESTED" = 'true' ]; then
+                      for TESTS in ${{ inputs.tasks }}; do
+                          RUN_TESTS="$RUN_TESTS ${{ inputs.backend }}:$SYSTEM:$TESTS"
+                      done
+                  else
+                      NEW_TESTS="$(./tests/lib/external/snapd-testing-tools/utils/spread-filter -r ./tests/lib/spread/rules/${{ inputs.rules }}.yaml -p "${{ inputs.backend }}:$SYSTEM" $CHANGES_PARAM)"
+                      if [ -z "$RUN_TESTS" ]; then
+                          RUN_TESTS="$NEW_TESTS"
+                      else
+                          RUN_TESTS="$RUN_TESTS $NEW_TESTS"
+                      fi
+                  fi
               done
           fi
           echo RUN_TESTS="$RUN_TESTS"  >> $GITHUB_ENV
-          echo "Suggested tests by spread-filter tool"
-          echo "$SUGGESTED_TESTS"
 
     - name: Setup grafana parameters
       if: "!contains(github.event.pull_request.labels.*.name, 'Skip spread')"
@@ -159,7 +173,7 @@ jobs:
         done
 
     - name: Run spread tests
-      if: "!contains(github.event.pull_request.labels.*.name, 'Skip spread') && ( !startsWith(inputs.group, 'nested-') || contains(github.event.pull_request.labels.*.name, 'Run nested') )"
+      if: "!contains(github.event.pull_request.labels.*.name, 'Skip spread')"
       env:
           SPREAD_GOOGLE_KEY: ${{ secrets.SPREAD_GOOGLE_KEY }}
       run: |

tests/lib/spread/rules/nested.yaml

@@ -4,6 +4,12 @@ rules:
       - tests/nested/.*
     to: [$SELF]
 
-  rest:
-    from: [.*]
+  nestedlib:
+    from:
+      - tests/lib/nested.sh
+    to: [tests/nested/]
+
+  assertions:
+    from:
+      - tests/lib/assertions/.*
     to: [tests/nested/]