From 43d19c62db4b3f05f4873da25c79ce3c44ff42a0 Mon Sep 17 00:00:00 2001 From: Alejandro Alvarez Date: Tue, 28 May 2024 11:35:06 +0200 Subject: [PATCH] Dat 16824 (#212) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 🔧 (ephemeral-cloud-infra.yml): add GitHub Actions workflow for managing ephemeral cloud infrastructure using Terraform and Spacectl. Includes linting, formatting, initializing, validating, previewing, deploying, and destroying infrastructure based on user input action. * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): update branch reference from 'master' to 'DAT-16824' for checkout action to target the correct branch * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): update GITHUB_TOKEN secret to use BOT_TOKEN for installing spacectl to ensure correct permissions are set * 🔧 (ephemeral-cloud-infra.yml): update Terraform Format job to include ls -ltr command before running terraform fmt for debugging purposes * 🔧 (ephemeral-cloud-infra.yml): remove unnecessary working-directory field from Terraform Format job to run in the default directory * 🔧 (ephemeral-cloud-infra.yml): remove Terraform Format and Terraform Validate steps to streamline workflow and improve efficiency * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): add TF_VAR prefix to secrets for Terraform compatibility 🚀 (.github/workflows/ephemeral-cloud-infra.yml): update workflow to create and destroy ephemeral stack based on deploy action * 🔧 (ephemeral-cloud-infra.yml): remove unnecessary TF_VAR_ prefix from secrets to simplify configuration and improve readability * 🔧 (ephemeral-cloud-infra.yml): update workflow to create and destroy ephemeral infrastructure based on input action. Add support for deploying and destroying ephemeral infrastructure using spacectl commands. * 🔧 (ephemeral-cloud-infra.yml): update destroy command to use spacectl stack task for better control and visibility * 🔧 (ephemeral-cloud-infra.yml): add support for deploying or destroying the dynamodb table by introducing a new boolean input parameter 'dynamodb' with a default value of false 🔧 (ephemeral-cloud-infra.yml): update the 'terraform destroy' command to include tail option for better logging visibility * 🔧 (ephemeral-cloud-infra.yml): Update workflow inputs to include 'destroy' boolean flag for destroying ephemeral cloud infrastructure. Adjust workflow steps to conditionally execute based on the 'destroy' flag value. * 🔧 (ephemeral-cloud-infra.yml): Update workflow inputs to include a deploy option for deploying the ephemeral cloud infrastructure. Adjust job conditions to execute based on the deploy input value. * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): add support for specifying stack ID to destroy as an input parameter to improve flexibility and customization * 🔧 (ephemeral-cloud-infra.yml): add output for EPHEMERAL_STACK_ID to access stack ID in subsequent steps * 🔧 (ephemeral-cloud-infra.yml): update output variable name from EPHEMERAL_STACK_ID to stack_id for consistency 🔧 (ephemeral-cloud-infra.yml): update output variable assignment to use GITHUB_OUTPUT instead of set-output for proper workflow output handling * 📝 (.github/workflows/ephemeral-cloud-infra.yml): add output for stack_id to provide the stack ID of the ephemeral cloud infrastructure. * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): add TF_VAR_create_dynamodb variable to enable dynamic creation of DynamoDB resources during infrastructure deployment * 🔧 (ephemeral-cloud-infra.yml): add upload and download steps for Terraform state as artifacts to improve traceability and debugging capabilities * 🔧 (ephemeral-cloud-infra.yml): set TF_VAR_stack_id environment variable based on workflow outputs and inputs to ensure correct stack id is used for deployment and destruction of ephemeral infrastructure * 🔧 (ephemeral-cloud-infra.yml): add AWS credentials configuration step to assume a specific role for AWS operations in us-east-1 region * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): update AWS role ARN for production environment to AWS_PROD_GITHUB_OIDC_ROLE_ARN_BUILD_LOGIC secret * 🔧 (ephemeral-cloud-infra.yml): update permissions for job to improve security and access control 🔧 (ephemeral-cloud-infra.yml): update role-to-assume secret for AWS credentials to match the correct environment and application role * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): update AWS role ARN secret to match the correct role for the build logic process * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): update AWS_DEV_GITHUB_OIDC_ROLE_ARN_BUILD_LOGIC secret to AWS_DEV_GITHUB_OIDC_ROLE_ARN_INFRASTRUCTURE for clarity and consistency in role naming. * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): update AWS role ARN for building logic to ensure correct permissions are assumed * 🔧 (ephemeral-cloud-infra.yml): comment out TF_VAR_create_dynamodb to prevent unnecessary creation of DynamoDB table during workflow execution * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): add support for setting TF_VAR_create_dynamodb variable before deploying ephemeral infrastructure * 🔧 (ephemeral-cloud-infra.yml): remove commented out TF_VAR_create_dynamodb variable to clean up configuration file * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): add setting TF_VAR_stack_id variable to improve Terraform configuration clarity * 🔧 (ephemeral-cloud-infra.yml): refactor setting TF_VAR_stack_id to use the last part of the stack_id for better readability * 🐛 (ephemeral-cloud-infra.yml): fix the index used in the cut command to correctly extract the stack ID from the output of create_stack step * 📝 (.github/workflows/ephemeral-cloud-infra.yml): update liquibase-infrastructure repository ref to 'master' branch to fix workflow issue * 📝 (.github/workflows/ephemeral-cloud-infra.yml): update description for dynamodb option to clarify it deploys or destroys the dynamodb infrastructure * 🔧 (.github/workflows/ephemeral-cloud-infra.yml): remove unnecessary conversion to uppercase for ID variable * 🔧 (ephemeral-cloud-infra.yml): convert extracted ID to uppercase for consistency and improved readability * 📝 (ephemeral-cloud-infra.yml): Add ephemeral-cloud-infra workflow for creating/destroying test automation cloud infrastructure 📝 (README.md): Add description for ephemeral-cloud-infra.yml workflow in README table * 🔧 (ephemeral-cloud-infra.yml): upgrade actions/upload-artifact and actions/download-artifact to v4 for improved functionality and compatibility * 🔧 (ephemeral-cloud-infra.yml): add support for deploying or destroying documentdb infrastructure by setting TF_VAR_create_documentdb variable based on inputs.documentdb value * 📝 (.github/workflows/ephemeral-cloud-infra.yml): update checkout action to use DAT-16824 branch instead of master for liquibase-infrastructure repository * 📝 (ephemeral-cloud-infra.yml): update checkout ref from DAT-16824 to master to ensure the latest changes are used in the workflow * 📝 (.github/workflows/ephemeral-cloud-infra.yml): update checkout action to use specific branch DAT-16824 instead of master for liquibase-infrastructure repository * 🔧 (ephemeral-cloud-infra.yml): remove unnecessary conversion to uppercase for ID variable * 🔧 (ephemeral-cloud-infra.yml): convert extracted ID to uppercase for consistency and improved readability * 🔧 (ephemeral-cloud-infra.yml): remove unnecessary conversion to uppercase for ID variable * 🔧 (ephemeral-cloud-infra.yml): update terraform destroy command to include flags for better performance and efficiency --- .github/workflows/ephemeral-cloud-infra.yml | 23 ++++++++++++++------- README.md | 1 + 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ephemeral-cloud-infra.yml b/.github/workflows/ephemeral-cloud-infra.yml index 478bf396..14cd5863 100644 --- a/.github/workflows/ephemeral-cloud-infra.yml +++ b/.github/workflows/ephemeral-cloud-infra.yml @@ -12,10 +12,15 @@ on: required: false type: boolean dynamodb: - description: 'Deploy or destroy the dynamodb table' + description: 'Deploy or destroy the dynamodb infrastructure' required: false type: boolean default: false + documentdb: + description: 'Deploy or destroy the documentdb infrastructure' + required: false + type: boolean + default: false stack_id: description: 'The stack ID to destroy' required: false @@ -36,8 +41,8 @@ jobs: contents: read id-token: write outputs: - stack_id: ${{ steps.create_stack.outputs.stack_id }} - resources_id: ${{ steps.create_infra.outputs.resources_id }} + stack_id: ${{ steps.create_stack.outputs.stack_id }} # Used to reference the stack created in the ephemeral infra + resources_id: ${{ steps.create_infra.outputs.resources_id }} # Used to reference the resources created in the ephemeral infra env: SPACELIFT_API_KEY_ENDPOINT: ${{ secrets.SPACELIFT_API_KEY_ENDPOINT }} SPACELIFT_API_KEY_ID: ${{ secrets.SPACELIFT_API_KEY_ID }} @@ -47,7 +52,7 @@ jobs: - name: Checkout liquibase-infrastructure uses: actions/checkout@v4 with: - ref: master + ref: DAT-16824 repository: liquibase/liquibase-infrastructure token: ${{ secrets.BOT_TOKEN }} @@ -78,7 +83,7 @@ jobs: - name: Upload Terraform state as artifact if: ${{ inputs.deploy }} - uses: actions/upload-artifact@v3 + uses: actions/upload-artifact@v4 with: name: terraform-state path: test-automation-ephemeral/stack/terraform.tfstate @@ -88,23 +93,25 @@ jobs: working-directory: test-automation-ephemeral/infra if: ${{ inputs.deploy }} run: | - ID=$(echo ${{ steps.create_stack.outputs.stack_id }} | cut -d '-' -f 5 | tr '[:lower:]' '[:upper:]') + ID=$(echo ${{ steps.create_stack.outputs.stack_id }} | cut -d '-' -f 5) echo "resources_id=$ID" >> "$GITHUB_OUTPUT" spacectl stack environment setvar --id $EPHEMERAL_STACK_ID TF_VAR_stack_id $ID spacectl stack environment setvar --id $EPHEMERAL_STACK_ID TF_VAR_create_dynamodb ${{ inputs.dynamodb }} + spacectl stack environment setvar --id $EPHEMERAL_STACK_ID TF_VAR_create_documentdb ${{ inputs.documentdb }} spacectl stack deploy --id $EPHEMERAL_STACK_ID --auto-confirm - name: Destroy ephemeral infra + continue-on-error: true env: TF_VAR_stack_id: ${{ inputs.stack_id }} if: ${{ inputs.destroy }} working-directory: test-automation-ephemeral/infra run: | - spacectl stack task --id ${{ inputs.stack_id }} --tail "terraform destroy -auto-approve" + spacectl stack task --id ${{ inputs.stack_id }} --tail "terraform destroy -refresh=false -parallelism=10 -auto-approve" - name: Download Terraform state if: ${{ inputs.destroy }} - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4 with: name: terraform-state path: test-automation-ephemeral/stack diff --git a/README.md b/README.md index fdfa7e74..deba7113 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,7 @@ Please review the below table of reusable workflows and their descriptions: | `build-artifact.yml` | Runs maven build and saves artifacts | | `codeql.yml` | Runs CodeQL scanning | | `create-release.yml` | Runs Release Drafter to auto create draft release notes | +| `ephemeral-cloud-infra.yml` | Creates/Destroys test automation cloud infrastructure | | `extension-attach-artifact-release.yml` | Attaches a tested artifact to the draft release. Receives a `zip` input to upload generated zip files | | `extension-release-published.yml` | Publishes a release to Maven Central | | `extension-update-version.yml` | Updates release and development `pom.xml` versions |