From cefdd3ce45a7a27f45fa59960ed14566b9d23ae8 Mon Sep 17 00:00:00 2001 From: Ray Harris Date: Tue, 16 Nov 2021 09:18:27 -0600 Subject: [PATCH] merge upstream (#3) * Let build-harness know what OS we're on (#195) * Add Slash Command Dispatch GitHub Actions template (#197) * Update Slash Command Dispatch GitHub Actions templates * Update Slash Command Dispatch GitHub Actions templates * Update Slash Command Dispatch GitHub Actions template * suppress detachedhead advise on versioned tag install (#199) Thanks! Makes sense... * Add discourse badges (#200) * fix gomplate install * rebuild readme * Update README.md (#201) Co-authored-by: osterman Co-authored-by: Cloud Posse Bot (CI/CD) Co-authored-by: osterman * Fixes #149, tf/lint on mac xargs does not support --no-run-if-empty (#206) * fix dispatch template paths (#211) * fix dispatch template paths * remove duplicate target * Use github actions to build and push (#213) * Use github actions to build and push * Update readme * rename to docker * rename to docker * update descriptions * build-and-push * Update readme * more renaming * Update .github/workflows/build-and-push.yml Co-authored-by: Andriy Knysh Co-authored-by: Andriy Knysh * fix issue template (#215) * add test workflow (#212) * add test workflow * bump versions * add test command * rename to chatops * drop slash command (#216) * Update codeowners (#217) * Fix terraform docs (#218) * build image on releases (#219) * Fix assert-set (#220) * Fix assert-set * Update description * use native make to test variables * reenable deps-dev * use native make error handling * add yq (#222) * Changes to support README.md to passing super-linter (#230) * MD changes to pass linting * Additional changes * Built readme * CR changes * Create ignore file by default * Change description to better match docs/deps * Change linter ignore to according type * Fix MD040/fenced-code-language lint error * Ignore just part of violating MD file * Ignore MD041 for whole file targets.md * Ignore linting all docs/*.md files * Update README template to pass lint and add gh action superlinter (#232) * Update README template to pass lint In PR against `terraform-example-module` I ran into [issue with superlint](https://github.com/cloudposse/terraform-example-module/pull/3/checks?check_run_id=928328375) > ERROR:[README.md:240 MD046/code-block-style Code block style [Expected: fenced; Actual: indented]] Here is documentation about it: https://github.com/DavidAnson/markdownlint/blob/master/doc/Rules.md#md046---code-block-style * Additional fix for another lint failure * Convert licenses to code and add superlint to gh actions * Fix readme * Remove ending white characters' * add auto-release action (#238) * refine auto release (#239) * Expand auto-releasers configuration (#240) * use labels * fix auto labels * Update Release Template (#241) * Update chatops (#236) * update chatops permissions * update chatops * Update .github/workflows/chatops.yml Co-authored-by: Nuru * Update .github/workflows/chatops.yml Co-authored-by: Nuru * update slash-command-dispatch Co-authored-by: Nuru * Fix module upgrade script (#243) * Fix module upgrade script * update deps * Add context.tf to CODEOWNERS (#245) * Fixing superlint terraform docs check for inherited markdown files (#244) * Update auto-release and docker workflow to push release tags (#249) * terraform/migrate-to-registry Makefile target added (#258) * terraform/migrate-to-registry Makefile target added * target renamed * readme rebuilded * markdown linter rules fix to avoid raising errors for lines longer then 400 chars * full linter config applied * terraform format for examples as well * markdownlint disabled for contributors table * regex fix * readme fix * readme fix * linter rules removed, no need for them any more * new target to remove upper bound for terraform core version constraint (#261) * Add support for Terraform 0.14 (#262) * fix broken interpolation (#265) * Add commands to help create pull requests (#266) * Add renovate config to Terraform modules (#267) * Support more automation (#268) * RUNNER_DOCKER_IMAGE was missing for 'builder' target (#269) * [auto-context] Open PR (as cloudpossebot) when context.tf changes (#270) * feat: add dynamic update the lists of related and references (#273) * Update README.md Template with BridgeCrew Compliance Badges (#256) * Update README.md Template with BridgeCrew Compliance Badges * template fixes * Security & Compliance section shown only for terraform modules Co-authored-by: Erik Osterman (CEO @ Cloud Posse) * approvers now allowed to review all *.tf files (#277) * approvers now allowed to review all *.tf files * README.yaml added for approvers * mergify rule to close PRs without changes (#280) * mergify rule to close PRs without changes * Update templates/terraform/.github/mergify.yml Co-authored-by: Erik Osterman (CEO @ Cloud Posse) * nightly build added before midnight to prepare build-harness images before most of nightly build fires (#281) * integration of generate-related-references target (#282) * remove generate-related-references target from auto-format (#283) * auto-release respect 'no-release' label (#279) * Adds auto-init capability using the bootstrap include directive (#272) * bugfix (#285) * Add `make` target to run pre-commit hooks (#286) * Add support for "deprecated" flag/section in README (#288) * Better formatting of combined release notes (#289) * Better formatting of combined release notes * add conditional to related section or README template (#290) * add conditional * fix formatting * update readme * Auto Format Co-authored-by: cloudpossebot <11232728+cloudpossebot@users.noreply.github.com> * Add Terraform versions 0.15 and 1.x (#293) * Enable formatting of modules requiring TF 0.15 or TF 1. (#294) * Create a draft release rather than no release with `no-release` label (#296) * update AWS CLI (#297) * Make 1.x the default Terraform version (#299) * remove target * remove .github directory Co-authored-by: David Schmidt <51931019+schmidtd@users.noreply.github.com> Co-authored-by: Andriy Knysh Co-authored-by: Hans Kristian Moen Co-authored-by: Erik Osterman Co-authored-by: Cloud Posse Bot (CI/CD) Co-authored-by: osterman Co-authored-by: Matt Gowie Co-authored-by: marcin Co-authored-by: Nuru Co-authored-by: Maxim Mironenko Co-authored-by: Matt Calhoun Co-authored-by: Vladimir Co-authored-by: Loren Gordon Co-authored-by: cloudpossebot <11232728+cloudpossebot@users.noreply.github.com> --- .editorconfig | 3 + .github/CODEOWNERS | 4 - .github/ISSUE_TEMPLATE/bug_report.md | 37 --- .github/ISSUE_TEMPLATE/config.yml | 18 -- .github/ISSUE_TEMPLATE/feature_request.md | 36 --- .github/ISSUE_TEMPLATE/question.md | 0 .github/PULL_REQUEST_TEMPLATE.md | 13 -- .github/auto-assign.yml | 16 -- .github/auto-label.yml | 24 -- .github/workflows/auto-assign.yml | 13 -- .github/workflows/auto-greet.yml | 23 -- .github/workflows/auto-label.yml | 11 - .github/workflows/auto-readme.yml | 43 ---- .travis.yml | 28 --- Dockerfile | 49 +++- Dockerfile.slim | 54 +++++ Makefile | 15 +- Makefile.helpers | 18 +- README.md | 177 ++++++++------ README.yaml | 24 +- bin/generate_related_references.py | 113 +++++++++ bin/install.sh | 2 +- bin/terraform-docs.awk | 90 -------- bin/terraform-docs.sh | 14 -- bin/upgrade_terraform_modules.sh | 2 +- docs/auto-init.md | 42 ++++ docs/extensions.md | 2 + docs/targets.md | 11 +- modules/aws/Makefile | 2 +- modules/docs/Makefile | 8 +- modules/github/Makefile.init | 41 ++-- modules/go/Makefile | 2 +- modules/make/Makefile | 9 +- modules/packages/Makefile | 24 +- modules/pre-commit/Makefile | 5 + modules/readme/Makefile | 5 +- modules/terraform/Makefile | 86 ++++++- templates/.github/CODEOWNERS | 23 +- .../.github/ISSUE_TEMPLATE/feature_request.md | 4 +- templates/.github/auto-assign.yml | 16 -- templates/.github/auto-label.yml | 2 - templates/.github/auto-release.yml | 54 +++++ templates/.github/workflows/auto-assign.yml | 13 -- templates/.github/workflows/auto-greet.yml | 23 -- templates/.github/workflows/auto-label.yml | 11 - templates/.github/workflows/auto-readme.yml | 43 ---- templates/.github/workflows/auto-release.yml | 26 +++ .../workflows/slash-command-dispatch.yml | 20 -- .../.github/workflows/validate-codeowners.yml | 27 +++ templates/Makefile.build-harness | 166 ++++++++++++- templates/README.md.gotmpl | 218 ++++++++++++------ templates/terraform/.github/mergify.yml | 65 ++++++ templates/terraform/.github/renovate.json | 12 + .../.github/workflows/auto-context.yml | 57 +++++ .../.github/workflows/auto-format.yml | 88 +++++++ .../terraform/.github/workflows/chatops.yml | 37 +++ 56 files changed, 1255 insertions(+), 714 deletions(-) delete mode 100644 .github/CODEOWNERS delete mode 100644 .github/ISSUE_TEMPLATE/bug_report.md delete mode 100644 .github/ISSUE_TEMPLATE/config.yml delete mode 100644 .github/ISSUE_TEMPLATE/feature_request.md delete mode 100644 .github/ISSUE_TEMPLATE/question.md delete mode 100644 .github/PULL_REQUEST_TEMPLATE.md delete mode 100644 .github/auto-assign.yml delete mode 100644 .github/auto-label.yml delete mode 100644 .github/workflows/auto-assign.yml delete mode 100644 .github/workflows/auto-greet.yml delete mode 100644 .github/workflows/auto-label.yml delete mode 100644 .github/workflows/auto-readme.yml delete mode 100644 .travis.yml create mode 100644 Dockerfile.slim create mode 100755 bin/generate_related_references.py delete mode 100644 bin/terraform-docs.awk delete mode 100755 bin/terraform-docs.sh create mode 100644 docs/auto-init.md create mode 100644 modules/pre-commit/Makefile delete mode 100644 templates/.github/auto-assign.yml delete mode 100644 templates/.github/auto-label.yml create mode 100644 templates/.github/auto-release.yml delete mode 100644 templates/.github/workflows/auto-assign.yml delete mode 100644 templates/.github/workflows/auto-greet.yml delete mode 100644 templates/.github/workflows/auto-label.yml delete mode 100644 templates/.github/workflows/auto-readme.yml create mode 100644 templates/.github/workflows/auto-release.yml delete mode 100644 templates/.github/workflows/slash-command-dispatch.yml create mode 100644 templates/.github/workflows/validate-codeowners.yml create mode 100644 templates/terraform/.github/mergify.yml create mode 100644 templates/terraform/.github/renovate.json create mode 100644 templates/terraform/.github/workflows/auto-context.yml create mode 100644 templates/terraform/.github/workflows/auto-format.yml create mode 100644 templates/terraform/.github/workflows/chatops.yml diff --git a/.editorconfig b/.editorconfig index 7928ca498..719fca446 100644 --- a/.editorconfig +++ b/.editorconfig @@ -1,3 +1,6 @@ +[*] +insert_final_newline = true + # Override for Makefile [{Makefile, makefile, GNUmakefile}] indent_style = tab diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS deleted file mode 100644 index 41c1baad5..000000000 --- a/.github/CODEOWNERS +++ /dev/null @@ -1,4 +0,0 @@ -# Use this file to define individuals or teams that are responsible for code in a repository. -# Read more: - -* @cloudposse/engineering \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md deleted file mode 100644 index f3df96b5d..000000000 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -name: Bug report -about: Create a report to help us improve -title: '' -labels: 'bug' -assignees: '' - ---- - -Found a bug? Maybe our [Slack Community](https://slack.cloudposse.com) can help. - -[![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) - -## Describe the Bug -A clear and concise description of what the bug is. - -## Expected Behavior -A clear and concise description of what you expected to happen. - -## Steps to Reproduce -Steps to reproduce the behavior: -1. Go to '...' -2. Run '....' -3. Enter '....' -4. See error - -## Screenshots -If applicable, add screenshots or logs to help explain your problem. - -## Environment (please complete the following information): - -Anything that will help us triage the bug will help. Here are some ideas: - - OS: [e.g. Linux, OSX, WSL, etc] - - Version [e.g. 10.15] - -## Additional Context -Add any other context about the problem here. \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml deleted file mode 100644 index 76ae6d67a..000000000 --- a/.github/ISSUE_TEMPLATE/config.yml +++ /dev/null @@ -1,18 +0,0 @@ -blank_issues_enabled: false - -contact_links: - - - name: Community Slack Team - url: https://cloudposse.com/slack/ - about: |- - Please ask and answer questions here. - - - name: Office Hours - url: https://cloudposse.com/office-hours/ - about: |- - Join us every Wednesday for FREE Office Hours (lunch & learn). - - - name: DevOps Accelerator Program - url: https://cloudposse.com/accelerate/ - about: |- - Own your infrastructure in record time. We build it. You drive it. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md deleted file mode 100644 index ecc9eb622..000000000 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -name: Feature Request -about: Suggest an idea for this project -title: '' -labels: 'feature request' -assignees: '' - ---- - -Have a question? Please checkout our [Slack Community](https://slack.cloudposse.com) in the `#geodesic` channel or visit our [Slack Archive](https://archive.sweetops.com/geodesic/). - -[![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) - -## Describe the Feature - -A clear and concise description of what the bug is. - -## Expected Behavior - -A clear and concise description of what you expected to happen. - -## Use Case - -Is your feature request related to a problem/challenge you are trying to solve? Please provide some additional context of why this feature or capability will be valuable. - -## Describe Ideal Solution - -A clear and concise description of what you want to happen. If you don't know, that's okay. - -## Alternatives Considered - -Explain what alternative solutions or features you've considered. - -## Additional Context - -Add any other context or screenshots about the feature request here. \ No newline at end of file diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md deleted file mode 100644 index e69de29bb..000000000 diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md deleted file mode 100644 index 4b8f32df3..000000000 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ /dev/null @@ -1,13 +0,0 @@ -## what -* Describe high-level what changed as a result of these commits (i.e. in plain-english, what do these changes mean?) -* Use bullet points to be concise and to the point. - -## why -* Provide the justifications for the changes (e.g. business case). -* Describe why these changes were made (e.g. why do these commits fix the problem?) -* Use bullet points to be concise and to the point. - -## references -* Link to any supporting github issues or helpful documentation to add some context (e.g. stackoverflow). -* Use `closes #123`, if this PR closes a GitHub issue `#123` - diff --git a/.github/auto-assign.yml b/.github/auto-assign.yml deleted file mode 100644 index 24fc72db3..000000000 --- a/.github/auto-assign.yml +++ /dev/null @@ -1,16 +0,0 @@ -# Set to true to add reviewers to pull requests -addReviewers: true - -# Set to author to set PR creator as assignee -addAssignees: author - -# A list of reviewers to be added to pull requests (GitHub user name) -# Teams are not supported (wontfix): https://github.com/kentaro-m/auto-assign/issues/39 -reviewers: - - aknysh - - osterman - - goruha - -# A number of reviewers added to the pull request -# Set 0 to add all the reviewers (default: 0) -numberOfReviewers: 0 diff --git a/.github/auto-label.yml b/.github/auto-label.yml deleted file mode 100644 index c7b151ca7..000000000 --- a/.github/auto-label.yml +++ /dev/null @@ -1,24 +0,0 @@ -modules/aws: modules/aws/** -modules/bash: modules/bash/** -modules/chamber: modules/chamber/** -modules/codefresh: modules/codefresh/** -modules/completion: modules/completion/** -modules/compose: modules/compose/** -modules/docker: modules/docker/** -modules/docs: modules/docs/** -modules/geodesic: modules/geodesic/** -modules/git: modules/git/** -modules/github: modules/github/** -modules/gitleaks: modules/gitleaks/** -modules/go: modules/go/** -modules/helm: modules/helm/** -modules/helmfile: modules/helmfile/** -modules/jenkins: modules/jenkins/** -modules/make: modules/make/** -modules/packages: modules/packages/** -modules/readme: modules/readme/** -modules/semver: modules/semver/** -modules/slack: modules/slack/** -modules/template: modules/template/** -modules/terraform: modules/terraform/** -modules/travis: modules/travis/** diff --git a/.github/workflows/auto-assign.yml b/.github/workflows/auto-assign.yml deleted file mode 100644 index dba9e4f6f..000000000 --- a/.github/workflows/auto-assign.yml +++ /dev/null @@ -1,13 +0,0 @@ -name: auto-assign -on: - pull_request: - types: [opened, reopened] - -jobs: - reviewers: - runs-on: ubuntu-latest - steps: - - uses: cloudposse/actions/github/auto-assign@0.5.0 - with: - repo-token: "${{ secrets.GITHUB_TOKEN }}" - configuration-path: '.github/auto-assign.yml' diff --git a/.github/workflows/auto-greet.yml b/.github/workflows/auto-greet.yml deleted file mode 100644 index 284bd70d9..000000000 --- a/.github/workflows/auto-greet.yml +++ /dev/null @@ -1,23 +0,0 @@ -name: auto-greet - -on: [pull_request, issues] - -jobs: - comment: - runs-on: ubuntu-latest - steps: - - uses: actions/first-interaction@v1 - with: - repo-token: ${{ secrets.GITHUB_TOKEN }} - issue-message: |- - Thank you for reporting the issue! If you haven't already [joined our slack community](https://slack.sweetops.com), then we invite you to do so. - - This is a great place to get help and ask questions from our AMAZING community. - - [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) - pr-message: |- - Thank you for submitting this PR! If you haven't already [joined our slack community](https://slack.sweetops.com), then we invite you to do so. - - We receive an overwhelming number of contributions. By joining our slack, we'll be able to review your PR faster. - - [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) diff --git a/.github/workflows/auto-label.yml b/.github/workflows/auto-label.yml deleted file mode 100644 index 6c8047e2e..000000000 --- a/.github/workflows/auto-label.yml +++ /dev/null @@ -1,11 +0,0 @@ -name: auto-label -on: [pull_request] - -jobs: - components: - runs-on: ubuntu-latest - steps: - - uses: actions/labeler@v2.1.0 - with: - repo-token: "${{ secrets.GITHUB_TOKEN }}" - configuration-path: '.github/auto-label.yml' diff --git a/.github/workflows/auto-readme.yml b/.github/workflows/auto-readme.yml deleted file mode 100644 index 48bccff66..000000000 --- a/.github/workflows/auto-readme.yml +++ /dev/null @@ -1,43 +0,0 @@ -name: 'auto-readme' - -on: - # Open a PR to update README on merge to master (as necessary) - push: - branches: - - master - - # Open a PR to update README for all PRs (as necessary) - pull_request: - types: [opened, synchronize, reopened] - - # Update README nightly - schedule: - - cron: '0 0 * * *' - -jobs: - update: - runs-on: ubuntu-latest - steps: - - name: checkout - uses: actions/checkout@v1 - - name: readme/update - shell: bash - env: - GITHUB_TOKEN: "${{ secrets.GITHUB_BOT_TOKEN }}" - run: | - make init - make readme/deps - make readme - - name: Create Pull Request - uses: cloudposse/actions/github/create-pull-request@0.4.0 - with: - labels: automated pr - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_BOT_TOKEN }} - COMMIT_MESSAGE: Update README.md - PULL_REQUEST_TITLE: Automatic Update of README.md - PULL_REQUEST_BODY: |- - This is an auto-generated PR which updates the `README.md` from the `README.yaml` - using the [`cloudposse/build-harness`](https://github.com/cloudposse/build-harness). - PULL_REQUEST_BRANCH: github-actions/auto-readme - BRANCH_SUFFIX: none diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index 26b90114d..000000000 --- a/.travis.yml +++ /dev/null @@ -1,28 +0,0 @@ -language: go -go: - - 1.9.x - -addons: - apt: - packages: - - git - - make - - curl - -services: -- docker - -install: -- make docker/login -- make go/deps-build -- make go/deps-dev -- make readme/deps - -script: -- make bash/lint -- make make/lint -- make terraform/install -- make docker/build - -after_success: -- make travis/docker-tag-and-push diff --git a/Dockerfile b/Dockerfile index eef01964d..b73ee422c 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM golang:1.13.4-alpine3.10 +FROM golang:1.15.11-alpine3.13 LABEL maintainer="Cloud Posse " LABEL "com.github.actions.name"="Build Harness" @@ -6,22 +6,36 @@ LABEL "com.github.actions.description"="Run any build-harness make target" LABEL "com.github.actions.icon"="tool" LABEL "com.github.actions.color"="blue" -RUN apk update && \ - apk --update add \ +RUN apk --update --no-cache add \ bash \ ca-certificates \ coreutils \ curl \ git \ gettext \ + go \ grep \ + groff \ jq \ libc6-compat \ make \ - py-pip && \ + perl \ + python3-dev \ + py-pip \ + py3-ruamel.yaml && \ + python3 -m pip install --upgrade pip setuptools wheel && \ + pip3 install --no-cache-dir \ + PyYAML==5.4.1 \ + awscli==1.20.28 \ + boto==2.49.0 \ + boto3==1.18.28 \ + iteration-utilities==0.11.0 \ + pre-commit \ + PyGithub==1.54.1 && \ git config --global advice.detachedHead false -RUN curl -sSL https://apk.cloudposse.com/install.sh | bash +SHELL ["/bin/bash", "-o", "pipefail", "-c"] +RUN curl -fsSL --retry 3 https://apk.cloudposse.com/install.sh | bash ## Install as packages @@ -29,17 +43,34 @@ RUN curl -sSL https://apk.cloudposse.com/install.sh | bash ## So can not be curl binary RUN apk --update --no-cache add \ chamber@cloudposse \ + gomplate@cloudposse \ helm@cloudposse \ helmfile@cloudposse \ - codefresh@cloudposse + codefresh@cloudposse \ + terraform-0.11@cloudposse terraform-0.12@cloudposse \ + terraform-0.13@cloudposse terraform-0.14@cloudposse \ + terraform-0.15@cloudposse terraform-1@cloudposse \ + terraform-config-inspect@cloudposse \ + terraform-docs@cloudposse \ + vert@cloudposse \ + yq@cloudposse && \ + sed -i /PATH=/d /etc/profile + +# Use Terraform 0.13 by default +ARG DEFAULT_TERRAFORM_VERSION=1 +RUN update-alternatives --set terraform /usr/share/terraform/$DEFAULT_TERRAFORM_VERSION/bin/terraform && \ + mkdir -p /build-harness/vendor && \ + cp -p /usr/share/terraform/$DEFAULT_TERRAFORM_VERSION/bin/terraform /build-harness/vendor/terraform -ADD ./ /build-harness/ +COPY ./ /build-harness/ ENV INSTALL_PATH /usr/local/bin WORKDIR /build-harness -RUN make -s template/deps aws/install +ARG PACKAGES_PREFER_HOST=true +RUN make -s bash/lint make/lint +RUN make -s template/deps readme/deps +RUN make -s go/deps-build go/deps-dev ENTRYPOINT ["/usr/bin/make"] - diff --git a/Dockerfile.slim b/Dockerfile.slim new file mode 100644 index 000000000..8bd30bd1f --- /dev/null +++ b/Dockerfile.slim @@ -0,0 +1,54 @@ +FROM alpine:3.13 +LABEL maintainer="Cloud Posse " + +LABEL "com.github.actions.name"="Build Harness" +LABEL "com.github.actions.description"="Run any build-harness make target" +LABEL "com.github.actions.icon"="tool" +LABEL "com.github.actions.color"="blue" + +RUN apk --no-cache add \ + bash \ + ca-certificates \ + coreutils \ + curl \ + git \ + gettext \ + grep \ + jq \ + libc6-compat \ + make + +RUN git config --global advice.detachedHead false + +SHELL ["/bin/bash", "-o", "pipefail", "-c"] +RUN curl -fsSL --retry 3 https://apk.cloudposse.com/install.sh | bash + +## Install as packages + +RUN apk --no-cache add \ + gomplate@cloudposse \ + terraform-0.12@cloudposse terraform-0.13@cloudposse terraform-0.14@cloudposse \ + terraform-0.15@cloudposse terraform-1@cloudposse \ + terraform-config-inspect@cloudposse \ + terraform-docs@cloudposse \ + vert@cloudposse + +RUN sed -i /PATH=/d /etc/profile + +# Use Terraform 0.13 by default +ARG DEFAULT_TERRAFORM_VERSION=0.13 +RUN update-alternatives --set terraform /usr/share/terraform/$DEFAULT_TERRAFORM_VERSION/bin/terraform && \ + mkdir -p /build-harness/vendor && \ + ln -s /usr/share/terraform/$DEFAULT_TERRAFORM_VERSION/bin/terraform /build-harness/vendor/terraform + +COPY ./ /build-harness/ + +ENV INSTALL_PATH /usr/local/bin + +WORKDIR /build-harness + +ENV PACKAGES_PREFER_HOST=true +RUN make -s bash/lint make/lint +RUN make -s template/deps readme/deps + +ENTRYPOINT ["/usr/bin/make"] diff --git a/Makefile b/Makefile index cd693ed0d..bc72d7f89 100644 --- a/Makefile +++ b/Makefile @@ -1,12 +1,22 @@ +# templates/Makefile.build-harness includes this Makefile +# and this Makefile includes templates/Makefile.build-harness +# to support different modes of invocation. Use a guard variable +# to prevent infinite recursive includes +ifeq ($(BUILD_HARNESS_TOP_LEVEL_MAKEFILE_GUARD),) +BUILD_HARNESS_TOP_LEVEL_MAKEFILE_GUARD := included + export OS ?= $(shell uname -s | tr '[:upper:]' '[:lower:]') export BUILD_HARNESS_PATH ?= $(shell 'pwd') export BUILD_HARNESS_EXTENSIONS_PATH ?= $(BUILD_HARNESS_PATH)/../build-harness-extensions -export BUILD_HARNESS_OS ?= $(shell uname -s | tr '[:upper:]' '[:lower:]') +export BUILD_HARNESS_OS ?= $(OS) export BUILD_HARNESS_ARCH ?= $(shell uname -m | sed 's/x86_64/amd64/g') export SELF ?= $(MAKE) export PATH := $(BUILD_HARNESS_PATH)/vendor:$(PATH) export DOCKER_BUILD_FLAGS ?= +# Forces auto-init off to avoid invoking the macro on recursive $(MAKE) +export BUILD_HARNESS_AUTO_INIT := false + # Debug should not be defaulted to a value because some cli consider any value as `true` (e.g. helm) export DEBUG ?= @@ -27,6 +37,7 @@ endif include $(BUILD_HARNESS_PATH)/Makefile.* include $(BUILD_HARNESS_PATH)/modules/*/bootstrap.Makefile* include $(BUILD_HARNESS_PATH)/modules/*/Makefile* +include $(BUILD_HARNESS_PATH)/templates/Makefile.build-harness # Don't fail if there are no build harness extensions # Wildcard conditions is to fixes `make[1]: *** No rule to make target` error ifneq ($(wildcard $(BUILD_HARNESS_EXTENSIONS_PATH)/modules/*/Makefile*),) @@ -41,3 +52,5 @@ ifndef TRANSLATE_COLON_NOTATION %: @$(SELF) -s $(subst :,/,$@) TRANSLATE_COLON_NOTATION=false endif + +endif diff --git a/Makefile.helpers b/Makefile.helpers index 2e02bfe4a..e4fcdcf4f 100644 --- a/Makefile.helpers +++ b/Makefile.helpers @@ -13,17 +13,27 @@ green = $(shell echo -e '\x1b[32;01m$1\x1b[0m') yellow = $(shell echo -e '\x1b[33;01m$1\x1b[0m') red = $(shell echo -e '\x1b[33;31m$1\x1b[0m') - -# Ensures that a variable is defined +# Ensures that a variable is defined and non-empty define assert-set - @[ -n "$($1)" ] || (echo "$(1) not defined in $(@)"; exit 1) + @$(if $($(1)),,$(error $(1) not defined in $(@))) endef # Ensures that a variable is undefined define assert-unset - @[ -z "$($1)" ] || (echo "$(1) should not be defined in $(@)"; exit 1) + @$(if $($1),$(error $(1) should not be defined in $(@)),) endef +test/assert-set: + $(call assert-set,PATH) + @echo assert-set PASS + +test/assert-unset: + $(call assert-unset,JKAHSDKJAHSDJKHASKD) + @echo assert-unset PASS + +test/assert: test/assert-set test/assert-unset + @exit 0 + default:: $(DEFAULT_HELP_TARGET) @exit 0 diff --git a/README.md b/README.md index 5a48e121a..2301402ba 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,23 @@ - +# Build Harness [![Build Status](https://github.com/cloudposse/build-harness/workflows/docker/badge.svg?branch=master)](https://github.com/cloudposse/build-harness/actions?query=workflow%3Adocker) [![Latest Release](https://img.shields.io/github/release/cloudposse/build-harness.svg)](https://github.com/cloudposse/build-harness/releases/latest) [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) [![Discourse Forum](https://img.shields.io/discourse/https/ask.sweetops.com/posts.svg)](https://ask.sweetops.com/) + +[![README Header][readme_header_img]][readme_header_link] +[![Cloud Posse][logo]](https://cpco.io/homepage) + -[![README Header][readme_header_img]][readme_header_link] - -[![Cloud Posse][logo]](https://cpco.io/homepage) - -# Build Harness [![Build Status](https://travis-ci.org/cloudposse/build-harness.svg?branch=master)](https://travis-ci.org/cloudposse/build-harness) [![Latest Release](https://img.shields.io/github/release/cloudposse/build-harness.svg)](https://github.com/cloudposse/build-harness/releases/latest) [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) - +--> This `build-harness` is a collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more. It's designed to work with CI/CD systems such as GitHub Actions, Codefresh, Travis CI, CircleCI and Jenkins. - --- -This project is part of our comprehensive ["SweetOps"](https://cpco.io/sweetops) approach towards DevOps. +This project is part of our comprehensive ["SweetOps"](https://cpco.io/sweetops) approach towards DevOps. [][share_email] [][share_googleplus] [][share_facebook] @@ -73,7 +55,6 @@ It's 100% Open Source and licensed under the [APACHE2](LICENSE). - ## Screenshots @@ -82,6 +63,9 @@ It's 100% Open Source and licensed under the [APACHE2](LICENSE). + + + ## Usage @@ -104,9 +88,9 @@ Run `make help` for a list of available targets. The `build-harness` is compatible with [GitHub Actions](https://github.com/features/actions). -Here's an example of running `make readme/lint` +Here's an example of running `make readme/lint` -``` +```yaml name: build-harness/readme/lint on: [pull_request] jobs: @@ -139,8 +123,9 @@ Here are some real world examples: + ## Makefile Targets -``` +```text Available targets: aws/install Install aws cli bundle @@ -148,6 +133,7 @@ Available targets: bash/lint Lint all bash scripts chamber/install Install chamber chamber/shell Start a chamber shell with secrets exported to the environment + clean Clean build-harness codefresh/export DEPRECATED!!! Export codefresh additional envvars codefresh/notify/slack/build Send notification from codefresh to slack using "build" template codefresh/notify/slack/deploy Send notification from codefresh to slack using "deploy" template @@ -236,6 +222,7 @@ Available targets: help Help screen help/all Display help for all targets help/short This help short screen + init Init build-harness jenkins/run-job-with-tag Run a Jenkins Job with $(TAG) make/lint Lint all makefiles packages/delete Delete packages @@ -254,32 +241,81 @@ Available targets: slack/notify/deploy Send notification to slack using "deploy" template template/build Create $OUT file by building it from $IN template file template/deps Install dependencies + terraform/bump-tf-12-min-version Rewrite versions.tf to bump modules with minimum core version of '0.12.x' to '>= 0.12.26' terraform/get-modules Ensure all modules can be fetched terraform/get-plugins Ensure all plugins can be fetched terraform/install Install terraform terraform/lint Lint check Terraform - terraform/upgrade-modules Upgrade all terraform module sources + terraform/loosen-constraints and convert "~>" constraints to ">=". + terraform/rewrite-required-providers Rewrite versions.tf to update existing configuration to add an explicit source attribute for each provider + terraform/upgrade-modules This target has not been upgraded to handle registry format terraform/validate Basic terraform sanity check travis/docker-login Login into docker hub travis/docker-tag-and-push Tag & Push according Travis environment variables ``` + + ## Extending `build-harness` with targets from another repo It is possible to extend the `build-harness` with targets and entire modules of your own, without having to fork or modify `build-harness` itself. This might be useful if, for example, you wanted to maintain some tooling that was specific to your environment that didn't have enough general applicability to be part of the main project. This makes it so you don't necessarily need to fork `build-harness` itself - you can place a repo defined by the environment variable `BUILD_HARNESS_EXTENSIONS_PATH` (a filesystem peer of `build-harness` named `build-harness-extensions` by default) and populate it with tools in the same `Makefile` within `module` structure as `build-harness` has. Modules will be combined and available with a unified `make` command. + + +## Using the "auto-init" feature + +Typically, the `build-harness` project requires running `make init` before any of the Makefile targets can be invoked. The `init` target will "install" the `build-harness` project and "include" the `Makefile` from the `build-harness` project. + +Alternatively, the "auto-init" feature can automatically run the `init` logic for you to install the `build-harness` and help keep the install up-to-date. This feature is enabled using the env or Makefile variable `BUILD_HARNESS_AUTO_INIT=true`. By default, this feature is disabled; to enable it, you must set the variable yourself. +**Note:** The "auto-init" feature is a convenience for running `make` interactively. Regardless of your setting of `BUILD_HARNESS_AUTO_INIT`, "auto-init" will be disabled if `make` is running inside a Docker container. Scripts and automation should continue to call `make init` explicitly. +```make +BUILD_HARNESS_AUTO_INIT = true -## Share the Love +-include $(shell curl -sSL -o .build-harness "https://git.io/build-harness"; echo .build-harness) +``` -Like this project? Please give it a ★ on [our GitHub](https://github.com/cloudposse/build-harness)! (it helps us **a lot**) +The "auto-init" feature will _also_ keep the install up-to-date. It will check the value of `BUILD_HARNESS_BRANCH`, get the commit ID, compare that to the current checkout, and update the clone if they differ. A useful side-effect is that it becomes easy to pin to versions of the `build-harness` from your own project, and let the `build-harness` update itself as you update the pin: + +```make +BUILD_HARNESS_AUTO_INIT = true +BUILD_HARNESS_BRANCH = {TAG} + +-include $(shell curl -sSL -o .build-harness "https://git.io/build-harness"; echo .build-harness) +``` + +Now when you run `make` the project will update itself to use the version specified by the `BUILD_HARNESS_BRANCH` value: + +```sh +$ make help +Removing existing build-harness +Cloning https://github.com/cloudposse/build-harness.git#{TAG}... +Cloning into 'build-harness'... +remote: Enumerating objects: 143, done. +remote: Counting objects: 100% (143/143), done. +remote: Compressing objects: 100% (118/118), done. +remote: Total 143 (delta 7), reused 71 (delta 3), pack-reused 0 +Receiving objects: 100% (143/143), 85.57 KiB | 2.09 MiB/s, done. +Resolving deltas: 100% (7/7), done. +Available targets: + + aws/install Install aws cli bundle +``` + + + + +## Share the Love + +Like this project? Please give it a ★ on [our GitHub](https://github.com/cloudposse/build-harness)! (it helps us **a lot**) Are you using this project or any of our other projects? Consider [leaving a testimonial][testimonial]. =) + ## Related Projects Check out these related projects. @@ -288,18 +324,16 @@ Check out these related projects. - [Dev Harness](https://github.com/cloudposse/dev) - Cloud Posse Local Development Harness - - ## References -For additional context, refer to some of these links. +For additional context, refer to some of these links. - [Wikipedia - Test Harness](https://en.wikipedia.org/wiki/Test_harness) - The `build-harness` is similar in concept to a "Test Harness" ## Help -**Got a question?** We got answers. +**Got a question?** We got answers. File a GitHub [issue](https://github.com/cloudposse/build-harness/issues), send us an [email][email] or join our [Slack Community][slack]. @@ -308,7 +342,7 @@ File a GitHub [issue](https://github.com/cloudposse/build-harness/issues), send ## DevOps Accelerator for Startups -We are a [**DevOps Accelerator**][commercial_support]. We'll help you build your cloud infrastructure from the ground up so you can own it. Then we'll show you how to operate it and stick around for as long as you need us. +We are a [**DevOps Accelerator**][commercial_support]. We'll help you build your cloud infrastructure from the ground up so you can own it. Then we'll show you how to operate it and stick around for as long as you need us. [![Learn More](https://img.shields.io/badge/learn%20more-success.svg?style=for-the-badge)][commercial_support] @@ -331,13 +365,17 @@ We deliver 10x the value for a fraction of the cost of a full-time engineer. Our Join our [Open Source Community][slack] on Slack. It's **FREE** for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally *sweet* infrastructure. +## Discourse Forums + +Participate in our [Discourse Forums][discourse]. Here you'll find answers to commonly asked questions. Most questions will be related to the enormous number of projects we support on our GitHub. Come here to collaborate on answers, find solutions, and get ideas about the products and services we value. It only takes a minute to get started! Just sign in with SSO using your GitHub account. + ## Newsletter -Sign up for [our newsletter][newsletter] that covers everything on our technology radar. Receive updates on what we're up to on GitHub as well as awesome new projects we discover. +Sign up for [our newsletter][newsletter] that covers everything on our technology radar. Receive updates on what we're up to on GitHub as well as awesome new projects we discover. ## Office Hours -[Join us every Wednesday via Zoom][office_hours] for our weekly "Lunch & Learn" sessions. It's **FREE** for everyone! +[Join us every Wednesday via Zoom][office_hours] for our weekly "Lunch & Learn" sessions. It's **FREE** for everyone! [![zoom](https://img.cloudposse.com/fit-in/200x200/https://cloudposse.com/wp-content/uploads/2019/08/Powered-by-Zoom.png")][office_hours] @@ -365,34 +403,36 @@ In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow. ## Copyrights -Copyright © 2016-2020 [Cloud Posse, LLC](https://cloudposse.com) +Copyright © 2016-2021 [Cloud Posse, LLC](https://cloudposse.com) -## License +## License -[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) See [LICENSE](LICENSE) for full details. - Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - https://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. +```text +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +``` @@ -414,7 +454,7 @@ This project is maintained and funded by [Cloud Posse, LLC][website]. Like it? P We're a [DevOps Professional Services][hire] company based in Los Angeles, CA. We ❤️ [Open Source Software][we_love_open_source]. -We offer [paid support][commercial_support] on all of our projects. +We offer [paid support][commercial_support] on all of our projects. Check out [our other projects][github], [follow us on twitter][twitter], [apply for a job][jobs], or [hire us][hire] to help with your cloud strategy and implementation. @@ -422,8 +462,10 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply ### Contributors -| [![Erik Osterman][osterman_avatar]][osterman_homepage]
[Erik Osterman][osterman_homepage] | [![Igor Rodionov][goruha_avatar]][goruha_homepage]
[Igor Rodionov][goruha_homepage] | [![Andriy Knysh][aknysh_avatar]][aknysh_homepage]
[Andriy Knysh][aknysh_homepage] | [![Sarkis][sarkis_avatar]][sarkis_homepage]
[Sarkis][sarkis_homepage] | [![Alexander Babai][alebabai_avatar]][alebabai_homepage]
[Alexander Babai][alebabai_homepage] | [![Jon Boulle][jonboulle_avatar]][jonboulle_homepage]
[Jon Boulle][jonboulle_homepage] | -|---|---|---|---|---|---| + +| [![Erik Osterman][osterman_avatar]][osterman_homepage]
[Erik Osterman][osterman_homepage] | [![Igor Rodionov][goruha_avatar]][goruha_homepage]
[Igor Rodionov][goruha_homepage] | [![Andriy Knysh][aknysh_avatar]][aknysh_homepage]
[Andriy Knysh][aknysh_homepage] | [![Sarkis][sarkis_avatar]][sarkis_homepage]
[Sarkis][sarkis_homepage] | [![Alexander Babai][alebabai_avatar]][alebabai_homepage]
[Alexander Babai][alebabai_homepage] | [![Jon Boulle][jonboulle_avatar]][jonboulle_homepage]
[Jon Boulle][jonboulle_homepage] | [![Marcin Brański][3h4x_avatar]][3h4x_homepage]
[Marcin Brański][3h4x_homepage] | +|---|---|---|---|---|---|---| + [osterman_homepage]: https://github.com/osterman [osterman_avatar]: https://img.cloudposse.com/150x150/https://github.com/osterman.png @@ -437,6 +479,8 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply [alebabai_avatar]: https://img.cloudposse.com/150x150/https://github.com/alebabai.png [jonboulle_homepage]: https://github.com/jonboulle [jonboulle_avatar]: https://img.cloudposse.com/150x150/https://github.com/jonboulle.png + [3h4x_homepage]: https://github.com/3h4x + [3h4x_avatar]: https://img.cloudposse.com/150x150/https://github.com/3h4x.png [![README Footer][readme_footer_img]][readme_footer_link] [![Beacon][beacon]][website] @@ -453,6 +497,7 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply [testimonial]: https://cpco.io/leave-testimonial?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/build-harness&utm_content=testimonial [office_hours]: https://cloudposse.com/office-hours?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/build-harness&utm_content=office_hours [newsletter]: https://cpco.io/newsletter?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/build-harness&utm_content=newsletter + [discourse]: https://ask.sweetops.com/?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/build-harness&utm_content=discourse [email]: https://cpco.io/email?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/build-harness&utm_content=email [commercial_support]: https://cpco.io/commercial-support?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/build-harness&utm_content=commercial_support [we_love_open_source]: https://cpco.io/we-love-open-source?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/build-harness&utm_content=we_love_open_source diff --git a/README.yaml b/README.yaml index c155efcc2..f31bfceb9 100644 --- a/README.yaml +++ b/README.yaml @@ -1,11 +1,11 @@ --- -# +# # This is the canonical configuration for the `README.md` # Run `make readme` to rebuild the `README.md` # # Name of this project -name: Build Harness +name: Build Harness # Logo for this project #logo: docs/logo.png @@ -19,20 +19,23 @@ copyrights: url: "https://cloudposse.com" year: "2016" -# Canonical GitHub repo +# Canonical GitHub repo github_repo: cloudposse/build-harness # Badges to display badges: - name: "Build Status" - image: "https://travis-ci.org/cloudposse/build-harness.svg?branch=master" - url: "https://travis-ci.org/cloudposse/build-harness" + image: "https://github.com/cloudposse/build-harness/workflows/docker/badge.svg?branch=master" + url: "https://github.com/cloudposse/build-harness/actions?query=workflow%3Adocker" - name: "Latest Release" image: "https://img.shields.io/github/release/cloudposse/build-harness.svg" url: "https://github.com/cloudposse/build-harness/releases/latest" - name: "Slack Community" image: "https://slack.cloudposse.com/badge.svg" url: "https://slack.cloudposse.com" + - name: "Discourse Forum" + image: "https://img.shields.io/discourse/https/ask.sweetops.com/posts.svg" + url: "https://ask.sweetops.com/" related: - name: "Packages" @@ -53,7 +56,7 @@ screenshots: - name: "demo" description: "Example of using the `build-harness` to build a docker image" url: "https://cdn.rawgit.com/cloudposse/build-harness/master/docs/demo.svg" - + # Short description of this project description: |- This `build-harness` is a collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more. @@ -80,12 +83,12 @@ usage: |- **NOTE:** the `/` is interchangable with the `:` in target names ## GitHub Actions - + The `build-harness` is compatible with [GitHub Actions](https://github.com/features/actions). - Here's an example of running `make readme/lint` + Here's an example of running `make readme/lint` - ``` + ```yaml name: build-harness/readme/lint on: [pull_request] jobs: @@ -119,6 +122,7 @@ quickstart: |- include: - "docs/targets.md" - "docs/extensions.md" + - "docs/auto-init.md" # Contributors to this project contributors: @@ -134,3 +138,5 @@ contributors: github: "alebabai" - name: "Jon Boulle" github: "jonboulle" + - name: "Marcin Brański" + github: "3h4x" diff --git a/bin/generate_related_references.py b/bin/generate_related_references.py new file mode 100755 index 000000000..d59c5de14 --- /dev/null +++ b/bin/generate_related_references.py @@ -0,0 +1,113 @@ +#!/usr/bin/env python3 + +import json +import os +import subprocess + +import requests +from github import Github +from iteration_utilities import unique_everseen +from ruamel.yaml import YAML + +GH_TOKEN = os.environ["GITHUB_TOKEN"] +GH_ORG_NAME = os.getenv("GH_ORG_NAME", "cloudposse") +GH_SEARCH_PATTERN = os.getenv("GH_SEARCH_PATTERN", "terraform-") +TF_MODULE_PATH = os.getenv("TF_MODULE_PATH", ".") +TF_CONFIG_INSPECT_BINARY_PATH = os.getenv( + "TF_CONFIG_INSPECT_BINARY_PATH", "terraform-config-inspect" +) +TF_REGISTRY_URL = "https://registry.terraform.io" + +gh = Github(GH_TOKEN) +yaml = YAML(typ="rt") +yaml.default_flow_style = False +yaml.preserve_quotes = False + + +def parse_gh(): + gh_repos = [] + for repo in gh.get_organization(GH_ORG_NAME).get_repos(): + if GH_SEARCH_PATTERN in repo.name: + repo_object = {} + repo_object["name"] = repo.name + repo_object["description"] = repo.description + repo_object["url"] = repo.html_url + gh_repos.append(repo_object) + return gh_repos + + +def tf_config_inspect(): + output = json.loads( + subprocess.check_output( + [TF_CONFIG_INSPECT_BINARY_PATH, TF_MODULE_PATH, "--json"], + stderr=subprocess.STDOUT, + ) + ) + return output + + +def parse_tf_registry(src_data, src_type): + items = [] + src_item = "module_calls" + if src_type == "providers": + src_item = "required_providers" + + for k, v in src_data[src_item].items(): + item_object = {} + url = TF_REGISTRY_URL + "/v1/" + src_type + "/" + v["source"] + r = requests.get(url=url).json() + + if src_type == "providers": + name_pattern = "terraform-provider-{}".format(r["name"]) + else: + name_pattern = "terraform-{}-{}".format(r["provider"], r["name"]) + item_object["name"] = name_pattern + + if src_type == "providers": + # description on GitHub looks better than on terraform-registry + gh_repo_info = gh.get_repo("{}/{}".format(r["namespace"], name_pattern)) + item_object["description"] = gh_repo_info.description + item_object["url"] = TF_REGISTRY_URL + "/providers/{}/{}/latest".format( + r["namespace"], r["name"] + ) + else: + item_object["description"] = r["description"] + item_object["url"] = r["source"] + + items.append(item_object) + return items + + +if __name__ == "__main__": + related_list = [] + reference_list = [] + + inspected_data = tf_config_inspect() + modules_list = parse_tf_registry(inspected_data, "modules") + providers_list = parse_tf_registry(inspected_data, "providers") + gh_repos_list = parse_gh() + + # this can be done in one line but it requires itertools + # and additional step to remove empty dicts + for m in unique_everseen(modules_list): + related_list.append(m) + for g in unique_everseen(gh_repos_list): + related_list.append(g) + for p in unique_everseen(providers_list): + reference_list.append(p) + + with open("{}/README.yaml".format(TF_MODULE_PATH)) as f: + readme = yaml.load(f) + + readme["related"] = related_list + + # ensure that "references" key is present and then insert data + if readme.get("references"): + readme["references"] = reference_list + else: + # create key "references" after the "related" + readme.insert(list(readme.keys()).index("related") + 1, "references", []) + readme["references"] = reference_list + + with open("{}/README.yaml".format(TF_MODULE_PATH), "w") as f: + yaml.dump(readme, f) diff --git a/bin/install.sh b/bin/install.sh index 1fef0edf3..9caaa9877 100755 --- a/bin/install.sh +++ b/bin/install.sh @@ -10,4 +10,4 @@ if [ "$BUILD_HARNESS_PROJECT" ] && [ -d "$BUILD_HARNESS_PROJECT" ]; then fi echo "Cloning ${GITHUB_REPO}#${BUILD_HARNESS_BRANCH}..." -git clone -b $BUILD_HARNESS_BRANCH $GITHUB_REPO +git clone -c advice.detachedHead=false --depth=1 -b $BUILD_HARNESS_BRANCH $GITHUB_REPO diff --git a/bin/terraform-docs.awk b/bin/terraform-docs.awk deleted file mode 100644 index bd6b2b7ff..000000000 --- a/bin/terraform-docs.awk +++ /dev/null @@ -1,90 +0,0 @@ -# This script converts Terraform 0.12 variables/outputs to something suitable for `terraform-docs` -# As of terraform-docs v0.6.0, HCL2 is not supported. This script is a *dirty hack* to get around it. -# https://github.com/segmentio/terraform-docs/ -# https://github.com/segmentio/terraform-docs/issues/62 - -{ - if ( $0 ~ /\{/ ) { - braceCnt++ - } - - if ( $0 ~ /\}/ ) { - braceCnt-- - } - - # [START] variable or output block started - if ($0 ~ /^[[:space:]]*(variable|output)[[:space:]][[:space:]]*"(.*?)"/) { - # Normalize the braceCnt (should be 1 now) - braceCnt = 1 - # [CLOSE] "default" block - if (blockDefCnt > 0) { - blockDefCnt = 0 - } - blockCnt++ - print $0 - } - - # [START] multiline default statement started - if (blockCnt > 0) { - if ($0 ~ /^[[:space:]][[:space:]]*(default)[[:space:]][[:space:]]*=/) { - if ($3 ~ "null") { - print " default = \"null\"" - } else { - print $0 - blockDefCnt++ - blockDefStart=1 - } - } - } - - # [PRINT] single line "description" - if (blockCnt > 0) { - if (blockDefCnt == 0) { - if ($0 ~ /^[[:space:]][[:space:]]*description[[:space:]][[:space:]]*=/) { - # [CLOSE] "default" block - if (blockDefCnt > 0) { - blockDefCnt = 0 - } - print $0 - } - } - } - - # [PRINT] single line "type" - if (blockCnt > 0) { - if ($0 ~ /^[[:space:]][[:space:]]*type[[:space:]][[:space:]]*=/ ) { - # [CLOSE] "default" block - if (blockDefCnt > 0) { - blockDefCnt = 0 - } - type=$3 - if (type ~ "object") { - print " type = \"object\"" - } else { - # legacy quoted types: "string", "list", and "map" - if ($3 ~ /^[[:space:]]*"(.*?)"[[:space:]]*$/) { - print " type = " $3 - } else { - print " type = \"" $3 "\"" - } - } - } - } - - # [CLOSE] variable/output block - if (blockCnt > 0) { - if (braceCnt == 0 && blockCnt > 0) { - blockCnt-- - print $0 - } - } - - # [PRINT] Multiline "default" statement - if (blockCnt > 0 && blockDefCnt > 0) { - if (blockDefStart == 1) { - blockDefStart = 0 - } else { - print $0 - } - } -} diff --git a/bin/terraform-docs.sh b/bin/terraform-docs.sh deleted file mode 100755 index 4e6be996d..000000000 --- a/bin/terraform-docs.sh +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/bash - -which awk 2>&1 >/dev/null || ( echo "awk not available"; exit 1) -which terraform 2>&1 >/dev/null || ( echo "terraform not available"; exit 1) -which terraform-docs 2>&1 >/dev/null || ( echo "terraform-docs not available"; exit 1) - -if [[ "`terraform version | head -1`" =~ 0\.12 ]]; then - TMP_FILE="$(mktemp /tmp/terraform-docs-XXXXXXXXXX)" - awk -f ${BUILD_HARNESS_PATH}/bin/terraform-docs.awk $2/*.tf > ${TMP_FILE} - terraform-docs $1 ${TMP_FILE} - rm -f ${TMP_FILE} -else - terraform-docs $1 $2 -fi diff --git a/bin/upgrade_terraform_modules.sh b/bin/upgrade_terraform_modules.sh index cd596682a..f7c27133a 100755 --- a/bin/upgrade_terraform_modules.sh +++ b/bin/upgrade_terraform_modules.sh @@ -18,7 +18,7 @@ function github_latest_release() { function upgrade_modules() { local file=$1 echo "Processing $file..." - for source in $(json2hcl -reverse < $file | jq -r '.module | .[][] | first | .source' 2>/dev/null); do + for source in $(grep -Po '^\s*source\s*=\s*"(.*?)"' -r .|cut -d'"' -f2|sort -u); do if [[ $source =~ github.com/ ]]; then echo "[GITHUB]: $source" if [[ $source =~ github.com/(.*?)/(.*?)\.git ]]; then diff --git a/docs/auto-init.md b/docs/auto-init.md new file mode 100644 index 000000000..81ed1e117 --- /dev/null +++ b/docs/auto-init.md @@ -0,0 +1,42 @@ + +## Using the "auto-init" feature + +Typically, the `build-harness` project requires running `make init` before any of the Makefile targets can be invoked. The `init` target will "install" the `build-harness` project and "include" the `Makefile` from the `build-harness` project. + +Alternatively, the "auto-init" feature can automatically run the `init` logic for you to install the `build-harness` and help keep the install up-to-date. This feature is enabled using the env or Makefile variable `BUILD_HARNESS_AUTO_INIT=true`. By default, this feature is disabled; to enable it, you must set the variable yourself. + +**Note:** The "auto-init" feature is a convenience for running `make` interactively. Regardless of your setting of `BUILD_HARNESS_AUTO_INIT`, "auto-init" will be disabled if `make` is running inside a Docker container. Scripts and automation should continue to call `make init` explicitly. + +```make +BUILD_HARNESS_AUTO_INIT = true + +-include $(shell curl -sSL -o .build-harness "https://git.io/build-harness"; echo .build-harness) +``` + +The "auto-init" feature will _also_ keep the install up-to-date. It will check the value of `BUILD_HARNESS_BRANCH`, get the commit ID, compare that to the current checkout, and update the clone if they differ. A useful side-effect is that it becomes easy to pin to versions of the `build-harness` from your own project, and let the `build-harness` update itself as you update the pin: + +```make +BUILD_HARNESS_AUTO_INIT = true +BUILD_HARNESS_BRANCH = {TAG} + +-include $(shell curl -sSL -o .build-harness "https://git.io/build-harness"; echo .build-harness) +``` + +Now when you run `make` the project will update itself to use the version specified by the `BUILD_HARNESS_BRANCH` value: + +```sh +$ make help +Removing existing build-harness +Cloning https://github.com/cloudposse/build-harness.git#{TAG}... +Cloning into 'build-harness'... +remote: Enumerating objects: 143, done. +remote: Counting objects: 100% (143/143), done. +remote: Compressing objects: 100% (118/118), done. +remote: Total 143 (delta 7), reused 71 (delta 3), pack-reused 0 +Receiving objects: 100% (143/143), 85.57 KiB | 2.09 MiB/s, done. +Resolving deltas: 100% (7/7), done. +Available targets: + + aws/install Install aws cli bundle +``` + diff --git a/docs/extensions.md b/docs/extensions.md index 93de2d0ef..c21645ac3 100644 --- a/docs/extensions.md +++ b/docs/extensions.md @@ -1,6 +1,8 @@ + ## Extending `build-harness` with targets from another repo It is possible to extend the `build-harness` with targets and entire modules of your own, without having to fork or modify `build-harness` itself. This might be useful if, for example, you wanted to maintain some tooling that was specific to your environment that didn't have enough general applicability to be part of the main project. This makes it so you don't necessarily need to fork `build-harness` itself - you can place a repo defined by the environment variable `BUILD_HARNESS_EXTENSIONS_PATH` (a filesystem peer of `build-harness` named `build-harness-extensions` by default) and populate it with tools in the same `Makefile` within `module` structure as `build-harness` has. Modules will be combined and available with a unified `make` command. + diff --git a/docs/targets.md b/docs/targets.md index c8e32d33a..9ba84410d 100644 --- a/docs/targets.md +++ b/docs/targets.md @@ -1,5 +1,6 @@ + ## Makefile Targets -``` +```text Available targets: aws/install Install aws cli bundle @@ -7,6 +8,7 @@ Available targets: bash/lint Lint all bash scripts chamber/install Install chamber chamber/shell Start a chamber shell with secrets exported to the environment + clean Clean build-harness codefresh/export DEPRECATED!!! Export codefresh additional envvars codefresh/notify/slack/build Send notification from codefresh to slack using "build" template codefresh/notify/slack/deploy Send notification from codefresh to slack using "deploy" template @@ -95,6 +97,7 @@ Available targets: help Help screen help/all Display help for all targets help/short This help short screen + init Init build-harness jenkins/run-job-with-tag Run a Jenkins Job with $(TAG) make/lint Lint all makefiles packages/delete Delete packages @@ -113,13 +116,17 @@ Available targets: slack/notify/deploy Send notification to slack using "deploy" template template/build Create $OUT file by building it from $IN template file template/deps Install dependencies + terraform/bump-tf-12-min-version Rewrite versions.tf to bump modules with minimum core version of '0.12.x' to '>= 0.12.26' terraform/get-modules Ensure all modules can be fetched terraform/get-plugins Ensure all plugins can be fetched terraform/install Install terraform terraform/lint Lint check Terraform - terraform/upgrade-modules Upgrade all terraform module sources + terraform/loosen-constraints and convert "~>" constraints to ">=". + terraform/rewrite-required-providers Rewrite versions.tf to update existing configuration to add an explicit source attribute for each provider + terraform/upgrade-modules This target has not been upgraded to handle registry format terraform/validate Basic terraform sanity check travis/docker-login Login into docker hub travis/docker-tag-and-push Tag & Push according Travis environment variables ``` + diff --git a/modules/aws/Makefile b/modules/aws/Makefile index de9cb8026..f024d95bc 100644 --- a/modules/aws/Makefile +++ b/modules/aws/Makefile @@ -1,4 +1,4 @@ -export AWSCLI_VERSION ?= 1.11.185 +export AWSCLI_VERSION ?= 1.20.28 WITH_AWS ?= aws-vault exec $(AWS_PROFILE) -- diff --git a/modules/docs/Makefile b/modules/docs/Makefile index 6e63a4c4d..de808b147 100644 --- a/modules/docs/Makefile +++ b/modules/docs/Makefile @@ -6,13 +6,17 @@ docs/deps:: ## Update `docs/targets.md` from `make help` docs/targets.md: docs/deps @( \ + echo ""; \ echo "## Makefile Targets"; \ - echo '```'; \ + echo '```text'; \ $(SELF) --no-print-directory --quiet --silent $(DEFAULT_HELP_TARGET) | sed $$'s,\x1b\\[[0-9;]*[a-zA-Z],,g'; \ echo '```'; \ + echo ""; \ ) > $@ .PHONY : docs/terraform.md ## Update `docs/terraform.md` from `terraform-docs` docs/terraform.md: docs/deps packages/install/terraform-docs - @$(BUILD_HARNESS_PATH)/bin/terraform-docs.sh md . > $@ + @echo "" > $@ + @terraform-docs md . >> $@ + @echo "" >> $@ diff --git a/modules/github/Makefile.init b/modules/github/Makefile.init index f91586dfb..ef1a207f5 100644 --- a/modules/github/Makefile.init +++ b/modules/github/Makefile.init @@ -1,34 +1,39 @@ GITHUB_TEMPLATES = \ .github/CODEOWNERS \ - .github/workflows/auto-greet.yml \ - .github/workflows/auto-assign.yml \ - .github/workflows/auto-label.yml \ - .github/workflows/auto-readme.yml \ - .github/workflows/slash-command-dispatch.yml \ - .github/ISSUE_TEMPLATE/config.yml \ .github/PULL_REQUEST_TEMPLATE.md \ + .github/ISSUE_TEMPLATE/config.yml \ .github/ISSUE_TEMPLATE/feature_request.md \ .github/ISSUE_TEMPLATE/bug_report.md \ - .github/ISSUE_TEMPLATE/question.md + .github/ISSUE_TEMPLATE/question.md \ + .github/auto-release.yml \ + .github/workflows/auto-release.yml \ + .github/workflows/validate-codeowners.yml + +# Install extra configuration for terraform projects +GITHUB_TERRAFORM_TEMPLATES = .github/workflows/chatops.yml \ + .github/workflows/auto-context.yml \ + .github/workflows/auto-format.yml \ + .github/mergify.yml \ + .github/renovate.json + $(GITHUB_TEMPLATES): $(addprefix $(BUILD_HARNESS_PATH)/templates/, $(GITHUB_TEMPLATES)) mkdir -p $(dir $@) cp $(BUILD_HARNESS_PATH)/templates/$@ $@ git ls-files --error-unmatch $@ 2>/dev/null || git add $@ -.github/auto-assign.yml:: # do not overwrite config by default +$(GITHUB_TERRAFORM_TEMPLATES): $(addprefix $(BUILD_HARNESS_PATH)/templates/terraform/, $(GITHUB_TERRAFORM_TEMPLATES)) mkdir -p $(dir $@) - cp $(BUILD_HARNESS_PATH)/templates/$@ $@ + cp $(BUILD_HARNESS_PATH)/templates/terraform/$@ $@ git ls-files --error-unmatch $@ 2>/dev/null || git add $@ -.github/auto-label.yml:: # do not overwrite config by default - mkdir -p $(dir $@) - cp $(BUILD_HARNESS_PATH)/templates/$@ $@ - git ls-files --error-unmatch $@ 2>/dev/null || git add $@ +github/init/context.tf: + curl -o context.tf -fsSL https://raw.githubusercontent.com/cloudposse/terraform-null-label/master/exports/context.tf + git ls-files --error-unmatch context.tf 2>/dev/null || git add context.tf + @if [[ -d examples/complete ]]; then \ + cp -p context.tf examples/complete/context.tf ; \ + fi -.github/slash-command-dispatch.yml:: # do not overwrite config by default - mkdir -p $(dir $@) - cp $(BUILD_HARNESS_PATH)/templates/$@ $@ - git ls-files --error-unmatch $@ 2>/dev/null || git add $@ +.PHONY: $(GITHUB_TEMPLATES) $(GITHUB_TERRAFORM_TEMPLATES) github/init/context.tf -github/init: $(GITHUB_TEMPLATES) .github/auto-assign.yml .github/auto-label.yml .github/slash-command-dispatch.yml +github/init: $(GITHUB_TEMPLATES) $(if $(wildcard *.tf),$(GITHUB_TERRAFORM_TEMPLATES) github/init/context.tf) diff --git a/modules/go/Makefile b/modules/go/Makefile index 10e90a9f5..4902323dc 100644 --- a/modules/go/Makefile +++ b/modules/go/Makefile @@ -1 +1 @@ -GO:= $(shell which go 2>/dev/null) +GO := $(shell which go 2>/dev/null) diff --git a/modules/make/Makefile b/modules/make/Makefile index b598cdbdc..b5779aaa2 100644 --- a/modules/make/Makefile +++ b/modules/make/Makefile @@ -3,6 +3,9 @@ make/lint: @LINT=true \ find . -type f -name '*Makefile*' \ -type f '!' -name '*.example' \ - -type f '!' -path './vendor/*' -exec \ - /bin/sh -c 'echo "==> {}">/dev/stderr; make --include-dir=modules/ --just-print --dry-run --recon --no-print-directory --quiet --silent -f {}' \; > /dev/null - @$(SELF) bash/lint + -type f '!' -path './modules/packages/*' \ + -type f '!' -path './vendor/*' \ + -type f '!' -path './templates/Makefile.build-harness' \ + -type f '!' -path './Makefile' \ + -exec \ + /bin/sh -c 'echo "==> {}">/dev/stderr; make --file=modules/packages/Makefile --include-dir=modules/ --just-print --dry-run --recon --no-print-directory --quiet --silent -f {}' \; > /dev/null diff --git a/modules/packages/Makefile b/modules/packages/Makefile index bf9cdff22..8fb31f9cd 100644 --- a/modules/packages/Makefile +++ b/modules/packages/Makefile @@ -1,6 +1,7 @@ export INSTALL_PATH ?= $(BUILD_HARNESS_PATH)/vendor -export PACKAGES_VERSION ?= 0.95.0 +export PACKAGES_VERSION ?= master export PACKAGES_PATH ?= $(BUILD_HARNESS_PATH)/vendor/packages +export PACKAGES_PREFER_HOST ?= false ## Delete packages packages/delete: @@ -13,18 +14,23 @@ packages/reinstall: packages/delete packages/install ## Install packages packages/install: @if [ ! -d $(PACKAGES_PATH) ]; then \ - echo "Installing packages $(PACKAGES_VERSION)..."; \ - rm -rf $(PACKAGES_PATH); \ - $(GIT) clone -c advice.detachedHead=false --depth=1 -b $(PACKAGES_VERSION) https://github.com/cloudposse/packages.git $(PACKAGES_PATH); \ - rm -rf $(PACKAGES_PATH)/.git; \ + echo "* Installing packages $(PACKAGES_VERSION)..."; \ + rm -rf $(PACKAGES_PATH); \ + $(GIT) clone -c advice.detachedHead=false --depth=1 -b $(PACKAGES_VERSION) https://github.com/cloudposse/packages.git $(PACKAGES_PATH); \ + rm -rf $(PACKAGES_PATH)/.git; \ fi ## Install package (e.g. helm, helmfile, kubectl) -packages/install/%: packages/install - @if [ ! -x $(INSTALL_PATH)/$(subst packages/install/,,$@) ]; then \ - $(MAKE) -C $(PACKAGES_PATH)/install $(subst packages/install/,,$@); \ +packages/install/%: + @binary="$*"; \ + if [[ -x "$(INSTALL_PATH)/$$binary" ]]; then \ + echo "* Package $$binary already installed"; \ + elif [[ "$(PACKAGES_PREFER_HOST)" == "true" ]] && installed=$$(command -v $* 2>/dev/null); then \ + echo Using "$*" from "$$installed" ; \ else \ - echo "Package $(subst packages/install/,,$@) already installed"; \ + $(MAKE) packages/install && \ + echo "* Installing $* to $(INSTALL_PATH)" && \ + $(MAKE) -C $(PACKAGES_PATH)/install "$*"; \ fi; ## Reinstall package (e.g. helm, helmfile, kubectl) diff --git a/modules/pre-commit/Makefile b/modules/pre-commit/Makefile new file mode 100644 index 000000000..891b6b25f --- /dev/null +++ b/modules/pre-commit/Makefile @@ -0,0 +1,5 @@ + +.PHONY: pre-commit/run + +pre-commit/run: + pre-commit run --all-files diff --git a/modules/readme/Makefile b/modules/readme/Makefile index 8371d6baf..4155d1347 100644 --- a/modules/readme/Makefile +++ b/modules/readme/Makefile @@ -28,7 +28,10 @@ readme/lint: @rm -f $(README_LINT) ## Create README.md by building it from README.yaml -readme/build: $(README_DEPS) +readme/build: readme/deps $(README_DEPS) @gomplate --file $(README_TEMPLATE_FILE) \ --out $(README_FILE) @echo "Generated $(README_FILE) from $(README_TEMPLATE_FILE) using data from $(README_TEMPLATE_YAML)" + +readme/generate-related-references: + @$(BUILD_HARNESS_PATH)/bin/generate_related_references.py diff --git a/modules/terraform/Makefile b/modules/terraform/Makefile index a159f3e7b..89e956efe 100644 --- a/modules/terraform/Makefile +++ b/modules/terraform/Makefile @@ -1,6 +1,6 @@ TMP ?= /tmp TERRAFORM ?= $(BUILD_HARNESS_PATH)/vendor/terraform -TERRAFORM_VERSION ?= 0.11.11 +TERRAFORM_VERSION ?= 0.12.26 TERRAFORM_URL ?= https://releases.hashicorp.com/terraform/$(TERRAFORM_VERSION)/terraform_$(TERRAFORM_VERSION)_$(OS)_$(BUILD_HARNESS_ARCH).zip ## Install terraform @@ -32,9 +32,89 @@ endif ## Lint check Terraform terraform/lint: +ifeq ($(OS), darwin) + @FAIL=`$(TERRAFORM) fmt -write=false | xargs -n 1 printf '\t- %s\n'`; \ + [ -z "$$FAIL" ] || (echo "Terraform configuration needs linting. Run '$(TERRAFORM) fmt'"; echo $$FAIL; exit 1) +else @FAIL=`$(TERRAFORM) fmt -write=false | xargs --no-run-if-empty -n 1 printf '\t- %s\n'`; \ [ -z "$$FAIL" ] || (echo "Terraform configuration needs linting. Run '$(TERRAFORM) fmt'"; echo $$FAIL; exit 1) +endif + +terraform/fmt: +ifeq ($(wildcard *.tf),) + @echo "* $@: No terraform files detected" +else + @{ command -v terraform-0.13 && \ + command -v terraform-0.14 && \ + command -v terraform-0.15 && \ + command -v terraform-1 \ + ; } >/dev/null || { echo "* $@ requires binaries named terraform-0.13, terraform-0.14, terraform-0.15, and terraform-1" && false; } + @ # vert exits non-zero if any of the versions are not acceptable, so `|| [[ -n "$VERSION" ]]` for a real error check + @TF012=0.12.29; \ + TF013=$$(terraform-0.13 version --json | jq -r .terraform_version); \ + TF014=$$(terraform-0.14 version --json | jq -r .terraform_version); \ + TF015=$$(terraform-0.15 version --json | jq -r .terraform_version); \ + TF1=$$(terraform-1 version --json | jq -r .terraform_version); \ + VERSION=$$(vert -s "$$(terraform-config-inspect --json . | jq -r '.required_core[]')" "$$TF012" "$$TF013" "$$TF014" "$$TF015" "$$TF1" | head -1) || [[ -n "$$VERSION" ]]; \ + VERSION=$${VERSION:0:4}; \ + [[ $$VERSION =~ ^1 ]] && VERSION=1 || true; \ + terraform-$${VERSION} fmt -recursive . +endif ## Upgrade all terraform module sources -terraform/upgrade-modules: packages/install/json2hcl - @$(BUILD_HARNESS_PATH)/bin/upgrade_terraform_modules.sh all +## This target has not been upgraded to handle registry format +terraform/upgrade-modules: + @echo "* $@ has been disabled because it has not been updated to handle Terraform registry syntax" + # @$(BUILD_HARNESS_PATH)/bin/upgrade_terraform_modules.sh all + +# Rewrite the *.tf files to use registry notation for modules sources +terraform/rewrite-module-source: TERRAFORM = terraform-0.13 +terraform/rewrite-module-source: + @sed -i -E 's,\s*source\s+=\s+"git::https://github.com/([^/]+)/terraform-([^-]+)-(.+).git\?ref=(tags/)?([0-9.]+)", source = "\1/\3/\2"\n version = "\5",g' $$(find . -type f -not -name context.tf -name '*.tf') + @$(TERRAFORM) fmt . + @$(TERRAFORM) fmt examples/complete + +terraform/rewrite-readme-source: TERRAFORM = terraform-0.13 +terraform/rewrite-readme-source: + @sed -i -E 's,^(\s*)source\s+=\s+"git::https://github.com/([^/]+)/terraform-([^-]+)-(.+).git\?ref=(tags/)?master",\1source = "\2/\4/\3"\n\1# Cloud Posse recommends pinning every module to a specific version\n\1# version = "x.x.x",g' README.yaml + +## Rewrite versions.tf to remove upper bound for terraform core version constraint (like this ">= 0.12.0, < 0.14.0") +## and convert "~>" constraints to ">=". +terraform/loosen-constraints: TERRAFORM = terraform-0.13 +terraform/loosen-constraints: + @for v in $$(find . -type f -name 'versions.tf'); do \ + DIR="$$(dirname $$v)"; \ + sed -i -E 's,\s\=\s*\"(.*[0-9.]+)(\,\s*<.*)", = "\1",g' "$$v" ; \ + sed -i -E 's,=\s*"\s*~>,= ">=,g' "$$v" ; \ + $(TERRAFORM) fmt "$$DIR" ; \ + done + +## Rewrite versions.tf to bump modules with minimum core version of '0.12.x' to '>= 0.12.26' +terraform/bump-tf-12-min-version: TERRAFORM = terraform-0.13 +terraform/bump-tf-12-min-version: terraform/loosen-constraints + @for v in $$(find . -type f -name 'versions.tf'); do \ + DIR="$$(dirname $$v)"; \ + PIN="$$(terraform-config-inspect --json "$$DIR" | jq -r '.required_core[]')" ; \ + if vert "$$PIN" 0.12.25 >/dev/null; then \ + echo updating "$$v" ; \ + sed -i -E 's,required_version\s*\=\s*\"(\~>|>\=)\s?(0\.12(\.\d+)?)\",required_version = ">= 0.12.26",g' "$$v" ; \ + $(TERRAFORM) fmt $$DIR ; \ + else \ + echo "$$v" already pinned to "$$PIN" ; \ + fi; \ + done + +## Rewrite versions.tf to update existing configuration to add an explicit source attribute for each provider +terraform/rewrite-required-providers: TERRAFORM = terraform-0.13 +terraform/rewrite-required-providers: terraform/loosen-constraints + @for v in $$(find . -type f -name 'versions.tf'); do \ + DIR="$$(dirname $$v)" ; \ + KEEP_VERSION=$$(grep -Eo 'required_version\s*=\s*"(.*)"' "$$v" | cut -d '"' -f2) ; \ + echo upgrading $$v ; \ + terraform-0.13 0.13upgrade -yes "$$DIR" >/dev/null ; \ + sed -i -E 's,(required_version\s*\=\s*\")(.*)(\"),\1'"$$KEEP_VERSION"'\3,g' "$$v" ; \ + done + +terraform/v14-rewrite: TERRAFORM = terraform-0.13 +terraform/v14-rewrite: terraform/loosen-constraints terraform/bump-tf-12-min-version terraform/rewrite-required-providers terraform/rewrite-module-source terraform/rewrite-readme-source + @{ [[ "$(TERRAFORM_FORCE_README)" != "true" ]] && git diff --no-patch --exit-code README.yaml; } || $(MAKE) readme diff --git a/templates/.github/CODEOWNERS b/templates/.github/CODEOWNERS index 41c1baad5..6f64b5a33 100644 --- a/templates/.github/CODEOWNERS +++ b/templates/.github/CODEOWNERS @@ -1,4 +1,25 @@ # Use this file to define individuals or teams that are responsible for code in a repository. # Read more: +# +# Order is important: the last matching pattern has the highest precedence -* @cloudposse/engineering \ No newline at end of file +# These owners will be the default owners for everything +* @cloudposse/engineering @cloudposse/contributors + +# Cloud Posse must review any changes to Makefiles +**/Makefile @cloudposse/engineering +**/Makefile.* @cloudposse/engineering + +# Cloud Posse must review any changes to GitHub actions +.github/* @cloudposse/engineering + +# Cloud Posse must review any changes to standard context definition, +# but some changes can be rubber-stamped. +**/*.tf @cloudposse/engineering @cloudposse/contributors @cloudposse/approvers +README.yaml @cloudposse/engineering @cloudposse/contributors @cloudposse/approvers +README.md @cloudposse/engineering @cloudposse/contributors @cloudposse/approvers +docs/*.md @cloudposse/engineering @cloudposse/contributors @cloudposse/approvers + +# Cloud Posse Admins must review all changes to CODEOWNERS or the mergify configuration +.github/mergify.yml @cloudposse/admins +.github/CODEOWNERS @cloudposse/admins diff --git a/templates/.github/ISSUE_TEMPLATE/feature_request.md b/templates/.github/ISSUE_TEMPLATE/feature_request.md index ecc9eb622..39a8686f1 100644 --- a/templates/.github/ISSUE_TEMPLATE/feature_request.md +++ b/templates/.github/ISSUE_TEMPLATE/feature_request.md @@ -7,7 +7,7 @@ assignees: '' --- -Have a question? Please checkout our [Slack Community](https://slack.cloudposse.com) in the `#geodesic` channel or visit our [Slack Archive](https://archive.sweetops.com/geodesic/). +Have a question? Please checkout our [Slack Community](https://slack.cloudposse.com) or visit our [Slack Archive](https://archive.sweetops.com/). [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) @@ -33,4 +33,4 @@ Explain what alternative solutions or features you've considered. ## Additional Context -Add any other context or screenshots about the feature request here. \ No newline at end of file +Add any other context or screenshots about the feature request here. diff --git a/templates/.github/auto-assign.yml b/templates/.github/auto-assign.yml deleted file mode 100644 index 24fc72db3..000000000 --- a/templates/.github/auto-assign.yml +++ /dev/null @@ -1,16 +0,0 @@ -# Set to true to add reviewers to pull requests -addReviewers: true - -# Set to author to set PR creator as assignee -addAssignees: author - -# A list of reviewers to be added to pull requests (GitHub user name) -# Teams are not supported (wontfix): https://github.com/kentaro-m/auto-assign/issues/39 -reviewers: - - aknysh - - osterman - - goruha - -# A number of reviewers added to the pull request -# Set 0 to add all the reviewers (default: 0) -numberOfReviewers: 0 diff --git a/templates/.github/auto-label.yml b/templates/.github/auto-label.yml deleted file mode 100644 index 4126ae481..000000000 --- a/templates/.github/auto-label.yml +++ /dev/null @@ -1,2 +0,0 @@ -README: -- README.md diff --git a/templates/.github/auto-release.yml b/templates/.github/auto-release.yml new file mode 100644 index 000000000..9976e1076 --- /dev/null +++ b/templates/.github/auto-release.yml @@ -0,0 +1,54 @@ +name-template: 'v$RESOLVED_VERSION' +tag-template: '$RESOLVED_VERSION' +version-template: '$MAJOR.$MINOR.$PATCH' +version-resolver: + major: + labels: + - 'major' + minor: + labels: + - 'minor' + - 'enhancement' + patch: + labels: + - 'auto-update' + - 'patch' + - 'fix' + - 'bugfix' + - 'bug' + - 'hotfix' + - 'no-release' + default: 'minor' + +categories: +- title: '🚀 Enhancements' + labels: + - 'enhancement' + - 'patch' +- title: '🐛 Bug Fixes' + labels: + - 'fix' + - 'bugfix' + - 'bug' + - 'hotfix' +- title: '🤖 Automatic Updates' + labels: + - 'auto-update' + +change-template: | +
+ $TITLE @$AUTHOR (#$NUMBER) + + $BODY +
+ +template: | + $CHANGES + +replacers: +# Remove irrelevant information from Renovate bot +- search: '/(?<=---\s+)+^#.*(Renovate configuration|Configuration)(?:.|\n)*?This PR has been generated .*/gm' + replace: '' +# Remove Renovate bot banner image +- search: '/\[!\[[^\]]*Renovate\][^\]]*\](\([^)]*\))?\s*\n+/gm' + replace: '' diff --git a/templates/.github/workflows/auto-assign.yml b/templates/.github/workflows/auto-assign.yml deleted file mode 100644 index dba9e4f6f..000000000 --- a/templates/.github/workflows/auto-assign.yml +++ /dev/null @@ -1,13 +0,0 @@ -name: auto-assign -on: - pull_request: - types: [opened, reopened] - -jobs: - reviewers: - runs-on: ubuntu-latest - steps: - - uses: cloudposse/actions/github/auto-assign@0.5.0 - with: - repo-token: "${{ secrets.GITHUB_TOKEN }}" - configuration-path: '.github/auto-assign.yml' diff --git a/templates/.github/workflows/auto-greet.yml b/templates/.github/workflows/auto-greet.yml deleted file mode 100644 index 284bd70d9..000000000 --- a/templates/.github/workflows/auto-greet.yml +++ /dev/null @@ -1,23 +0,0 @@ -name: auto-greet - -on: [pull_request, issues] - -jobs: - comment: - runs-on: ubuntu-latest - steps: - - uses: actions/first-interaction@v1 - with: - repo-token: ${{ secrets.GITHUB_TOKEN }} - issue-message: |- - Thank you for reporting the issue! If you haven't already [joined our slack community](https://slack.sweetops.com), then we invite you to do so. - - This is a great place to get help and ask questions from our AMAZING community. - - [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) - pr-message: |- - Thank you for submitting this PR! If you haven't already [joined our slack community](https://slack.sweetops.com), then we invite you to do so. - - We receive an overwhelming number of contributions. By joining our slack, we'll be able to review your PR faster. - - [![Slack Community](https://slack.cloudposse.com/badge.svg)](https://slack.cloudposse.com) diff --git a/templates/.github/workflows/auto-label.yml b/templates/.github/workflows/auto-label.yml deleted file mode 100644 index 6c8047e2e..000000000 --- a/templates/.github/workflows/auto-label.yml +++ /dev/null @@ -1,11 +0,0 @@ -name: auto-label -on: [pull_request] - -jobs: - components: - runs-on: ubuntu-latest - steps: - - uses: actions/labeler@v2.1.0 - with: - repo-token: "${{ secrets.GITHUB_TOKEN }}" - configuration-path: '.github/auto-label.yml' diff --git a/templates/.github/workflows/auto-readme.yml b/templates/.github/workflows/auto-readme.yml deleted file mode 100644 index 48bccff66..000000000 --- a/templates/.github/workflows/auto-readme.yml +++ /dev/null @@ -1,43 +0,0 @@ -name: 'auto-readme' - -on: - # Open a PR to update README on merge to master (as necessary) - push: - branches: - - master - - # Open a PR to update README for all PRs (as necessary) - pull_request: - types: [opened, synchronize, reopened] - - # Update README nightly - schedule: - - cron: '0 0 * * *' - -jobs: - update: - runs-on: ubuntu-latest - steps: - - name: checkout - uses: actions/checkout@v1 - - name: readme/update - shell: bash - env: - GITHUB_TOKEN: "${{ secrets.GITHUB_BOT_TOKEN }}" - run: | - make init - make readme/deps - make readme - - name: Create Pull Request - uses: cloudposse/actions/github/create-pull-request@0.4.0 - with: - labels: automated pr - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_BOT_TOKEN }} - COMMIT_MESSAGE: Update README.md - PULL_REQUEST_TITLE: Automatic Update of README.md - PULL_REQUEST_BODY: |- - This is an auto-generated PR which updates the `README.md` from the `README.yaml` - using the [`cloudposse/build-harness`](https://github.com/cloudposse/build-harness). - PULL_REQUEST_BRANCH: github-actions/auto-readme - BRANCH_SUFFIX: none diff --git a/templates/.github/workflows/auto-release.yml b/templates/.github/workflows/auto-release.yml new file mode 100644 index 000000000..3a38fae08 --- /dev/null +++ b/templates/.github/workflows/auto-release.yml @@ -0,0 +1,26 @@ +name: auto-release + +on: + push: + branches: + - main + - master + - production + +jobs: + publish: + runs-on: ubuntu-latest + steps: + # Get PR from merged commit to master + - uses: actions-ecosystem/action-get-merged-pull-request@v1 + id: get-merged-pull-request + with: + github_token: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} + # Drafts your next Release notes as Pull Requests are merged into "main" + - uses: release-drafter/release-drafter@v5 + with: + publish: ${{ !contains(steps.get-merged-pull-request.outputs.labels, 'no-release') }} + prerelease: false + config-name: auto-release.yml + env: + GITHUB_TOKEN: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} diff --git a/templates/.github/workflows/slash-command-dispatch.yml b/templates/.github/workflows/slash-command-dispatch.yml deleted file mode 100644 index ebcef28c7..000000000 --- a/templates/.github/workflows/slash-command-dispatch.yml +++ /dev/null @@ -1,20 +0,0 @@ -name: Slash Command Dispatch -on: - issue_comment: - types: [created] - -jobs: - slashCommandDispatch: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - - name: Slash Command Dispatch - uses: cloudposse/actions/github/slash-command-dispatch@0.9.0 - with: - token: ${{ secrets.GITHUB_BOT_TOKEN }} - reaction-token: ${{ secrets.GITHUB_TOKEN }} - repository: cloudposse/actions - commands: rebuild-readme - permission: none - issue-type: pull-request diff --git a/templates/.github/workflows/validate-codeowners.yml b/templates/.github/workflows/validate-codeowners.yml new file mode 100644 index 000000000..c5193b625 --- /dev/null +++ b/templates/.github/workflows/validate-codeowners.yml @@ -0,0 +1,27 @@ +name: Validate Codeowners +on: + workflow_dispatch: + + pull_request: + +jobs: + validate-codeowners: + runs-on: ubuntu-latest + steps: + - name: "Checkout source code at current commit" + uses: actions/checkout@v2 + - uses: mszostok/codeowners-validator@v0.5.0 + if: github.event.pull_request.head.repo.full_name == github.repository + name: "Full check of CODEOWNERS" + with: + # For now, remove "files" check to allow CODEOWNERS to specify non-existent + # files so we can use the same CODEOWNERS file for Terraform and non-Terraform repos + # checks: "files,syntax,owners,duppatterns" + checks: "syntax,owners,duppatterns" + # GitHub access token is required only if the `owners` check is enabled + github_access_token: "${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }}" + - uses: mszostok/codeowners-validator@v0.5.0 + if: github.event.pull_request.head.repo.full_name != github.repository + name: "Syntax check of CODEOWNERS" + with: + checks: "syntax,duppatterns" diff --git a/templates/Makefile.build-harness b/templates/Makefile.build-harness index 5834360f3..04a6cf506 100644 --- a/templates/Makefile.build-harness +++ b/templates/Makefile.build-harness @@ -1,19 +1,173 @@ +# +# This is a shim installed automatically by the build-harness +# https://github.com/cloudposse/build-harness +# + +# templates/Makefile.build-harness includes this Makefile +# and this Makefile includes templates/Makefile.build-harness +# to support different modes of invocation. Use a guard variable +# to prevent infinite recursive includes +ifeq ($(BUILD_HARNESS_TEMPLATES_MAKEFILE_GUARD),) +BUILD_HARNESS_TEMPLATES_MAKEFILE_GUARD := included + export SHELL = /bin/bash +export PWD = $(shell pwd) export BUILD_HARNESS_ORG ?= cloudposse export BUILD_HARNESS_PROJECT ?= build-harness +export BUILD_HARNESS_DOCKER_IMAGE ?= $(BUILD_HARNESS_ORG)/$(BUILD_HARNESS_PROJECT) export BUILD_HARNESS_BRANCH ?= master -export BUILD_HARNESS_PATH ?= $(shell until [ -d "$(BUILD_HARNESS_PROJECT)" ] || [ "`pwd`" == '/' ]; do cd ..; done; pwd)/$(BUILD_HARNESS_PROJECT) --include $(BUILD_HARNESS_PATH)/Makefile +export BUILD_HARNESS_CLONE_URL ?= https://github.com/$(BUILD_HARNESS_ORG)/$(BUILD_HARNESS_PROJECT).git + +# Resolves BUILD_HARNESS_PATH to BUILD_HARNESS_PATH_LOCAL when BUILD_HARNESS_PATH does not exist +BUILD_HARNESS_PATH ?= $(shell until [ -d "$(BUILD_HARNESS_PROJECT)" ] || [ "`pwd`" == '/' ]; do cd ..; done; pwd)/$(BUILD_HARNESS_PROJECT) +BUILD_HARNESS_PATH_LOCAL := $(PWD)/$(BUILD_HARNESS_PROJECT) +export BUILD_HARNESS_PATH := $(or $(wildcard $(BUILD_HARNESS_PATH)),$(BUILD_HARNESS_PATH_LOCAL)) +# It is kind of expensive to figure out the Docker SHA tag, so we just define the command here, and only call it when needed +# With the ":=" syntax, it stores the current value of BUILD_HARNESS_PATH, so this has to come after that has been set with ":=" +export BUILD_HARNESS_DOCKER_SHA_TAG_CMD := git -C "$(BUILD_HARNESS_PATH)" log -n 1 --format=sha-%h 2>/dev/null || echo latest + +# Toggles the auto-init feature +BUILD_HARNESS_AUTO_INIT ?= false + +# Macro to clone/install BUILD_HARNESS_PROJECT +define harness_install +curl --retry 5 --fail --silent --retry-delay 1 \ + https://raw.githubusercontent.com/$(BUILD_HARNESS_ORG)/$(BUILD_HARNESS_PROJECT)/$(BUILD_HARNESS_BRANCH)/bin/install.sh | \ + bash -s "$(BUILD_HARNESS_ORG)" "$(BUILD_HARNESS_PROJECT)" "$(BUILD_HARNESS_BRANCH)" +endef + +# Macro to auto-init the BUILD_HARNESS_PROJECT with the `include` directive +# Tests if BUILD_HARNESS_PROJECT does not yet exist, or if it does exist but the +# checkout does not match BUILD_HARNESS_BRANCH +define harness_auto_init +if [[ \ + -f "/build-harness/Makefile" || -f "/$(BUILD_HARNESS_PROJECT)/Makefile" \ +]]; then \ + echo "[.build-harness]: In $(BUILD_HARNESS_PROJECT) docker container, skipping auto-init" ;\ +elif [[ \ + grep -q docker /proc/1/cgroup 2>/dev/null \ +]]; then \ + echo "[.build-harness]: In unknown docker container, skipping auto-init" ;\ +elif [[ \ + "$(BUILD_HARNESS_PATH)" != "$(BUILD_HARNESS_PATH_LOCAL)" && \ + -f "$(BUILD_HARNESS_PATH)/Makefile" \ +]]; then \ + echo "[.build-harness]: Using external $(BUILD_HARNESS_PATH), skipping auto-init" ;\ +elif [[ \ + "$(BUILD_HARNESS_PATH)" == "$(BUILD_HARNESS_PATH_LOCAL)" && \ + -f "$(BUILD_HARNESS_PATH)/Makefile" && \ + "$$(git -C '$(BUILD_HARNESS_PATH_LOCAL)' ls-remote '$(BUILD_HARNESS_CLONE_URL)' '$(BUILD_HARNESS_BRANCH)' | cut -f1)" == "$$(git -C '$(BUILD_HARNESS_PATH_LOCAL)' rev-parse HEAD)" \ +]]; then \ + echo "[.build-harness]: Clone of $(BUILD_HARNESS_PROJECT) is up-to-date, skipping auto-init" ;\ +else \ + $(harness_install) ;\ +fi +endef + +-include $(if $(findstring true,$(BUILD_HARNESS_AUTO_INIT)),$(shell $(harness_auto_init) >&2)) $(BUILD_HARNESS_PATH)/Makefile .PHONY : init ## Init build-harness init:: - @curl --retry 5 --fail --silent --retry-delay 1 https://raw.githubusercontent.com/$(BUILD_HARNESS_ORG)/$(BUILD_HARNESS_PROJECT)/$(BUILD_HARNESS_BRANCH)/bin/install.sh | \ - bash -s "$(BUILD_HARNESS_ORG)" "$(BUILD_HARNESS_PROJECT)" "$(BUILD_HARNESS_BRANCH)" + @ $(harness_install) .PHONY : clean ## Clean build-harness clean:: @[ "$(BUILD_HARNESS_PATH)" == '/' ] || \ - [ "$(BUILD_HARNESS_PATH)" == '.' ] || \ - echo rm -rf $(BUILD_HARNESS_PATH) + [ "$(BUILD_HARNESS_PATH)" == '.' ] || \ + [ "$(BUILD_HARNESS_PATH)" == '/$(BUILD_HARNESS_PROJECT)' ] || \ + echo rm -rf $(BUILD_HARNESS_PATH) + +.PHONY: build-harness/shell builder build-harness/shell/pull builder/pull builder/build + +build-harness/shell/pull builder/pull builder/build: BUILD_HARNESS_DOCKER_SHA_TAG ?= $(shell $(BUILD_HARNESS_DOCKER_SHA_TAG_CMD)) +build-harness/shell/pull builder/pull: + docker pull $(BUILD_HARNESS_DOCKER_IMAGE):$(BUILD_HARNESS_DOCKER_SHA_TAG) + @[[ "$(BUILD_HARNESS_DOCKER_SHA_TAG)" == "latest" ]] || docker pull $(BUILD_HARNESS_DOCKER_IMAGE):latest + +builder/build: export DOCKER_IMAGE_NAME = $(BUILD_HARNESS_DOCKER_IMAGE):$(BUILD_HARNESS_DOCKER_SHA_TAG) +builder/build: + @$(MAKE) --no-print-directory docker/build + +DEFAULT_DOCKER_ENVS := AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN TERM AWS_PROFILE AWS_REGION \ + AWS_DEFAULT_PROFILE AWS_DEFAULT_REGION +EXTRA_DOCKER_ENVS ?= AWS_CONFIG_FILE AWS_SHARED_CREDENTIALS_FILE +DOCKER_ENVS ?= $(DEFAULT_DOCKER_ENVS) $(EXTRA_DOCKER_ENVS) + +## Start a shell inside of the `build-harness` docker container with `make build-harness/shell` or `make builder` +## Run `make` targets inside the build-harness shell by setting `TARGETS` or `TARGET`, e.g. +## make builder TARGETS="github/init readme" +build-harness/shell builder tester: MOUNT_HOME ?= $(shell [ -d "$$HOME" ] && printf -- "-e HOME -v \"%s\":\"%s\"" "$$HOME" "$$HOME") +build-harness/shell builder tester: TARGETS ?= $(TARGET) +build-harness/shell builder tester: ARGS := $(if $(TARGETS),$(TARGETS),-l || true) +build-harness/shell builder tester: ENTRYPOINT := $(if $(TARGETS),/usr/bin/make,/bin/bash) +build-harness/shell builder pr/pre-commit: RUNNER_DOCKER_TAG ?= $(shell $(BUILD_HARNESS_DOCKER_SHA_TAG_CMD)) +build-harness/shell builder pr/pre-commit: RUNNER_DOCKER_IMAGE ?= $(BUILD_HARNESS_DOCKER_IMAGE) +build-harness/shell builder: build-harness/runner + @exit 0 + +.PHONY: build-harness/shell-slim builder-slim pr/auto-format pr/auto-format/host pr/readme pr/readme/host pr/pre-commit tf14-upgrade + +build-harness/shell-slim builder-slim pr/auto-format pr/readme tf14-upgrade: RUNNER_DOCKER_IMAGE ?= $(BUILD_HARNESS_DOCKER_IMAGE) + +build-harness/shell-slim builder-slim tf14-upgrade pr/auto-format pr/readme: RUNNER_DOCKER_SHA_TAG ?= $(shell $(BUILD_HARNESS_DOCKER_SHA_TAG_CMD)) +build-harness/shell-slim builder-slim tf14-upgrade pr/auto-format pr/readme: RUNNER_DOCKER_TAG ?= \ + $(shell docker inspect --type=image $(RUNNER_DOCKER_IMAGE):$(RUNNER_DOCKER_SHA_TAG) >/dev/null 2>&1 && \ + echo "$(RUNNER_DOCKER_SHA_TAG) " || echo "slim-$(RUNNER_DOCKER_SHA_TAG)") + +build-harness/shell-slim builder-slim: TARGETS ?= $(TARGET) +build-harness/shell-slim builder-slim: ARGS := $(if $(TARGETS),$(TARGETS),-l || true) +build-harness/shell-slim builder-slim: ENTRYPOINT := $(if $(TARGETS),/usr/bin/make,/bin/bash) +build-harness/shell-slim builder-slim: build-harness/runner + +pr/auto-format pr/readme pr/pre-commit tf14-upgrade : ENTRYPOINT := /usr/bin/make + +pr/auto-format pr/auto-format/host: ARGS := terraform/fmt readme +pr/readme pr/readme/host: ARGS := readme/deps readme +pr/auto-format pr/readme: build-harness/runner +pr/auto-format/host pr/readme/host: + $(MAKE) $(ARGS) + +pr/pre-commit: ARGS := pre-commit/run +pr/pre-commit: build-harness/runner + +tf14-upgrade: export TERRAFORM_FORCE_README := true +tf14-upgrade: ARGS := github/init terraform/v14-rewrite +tf14-upgrade: build-harness/runner + +.PHONY: tester tester/pull + +tester tester/pull: TEST_HARNESS_DOCKER_IMAGE ?= cloudposse/test-harness +tester tester/pull: TEST_HARNESS_DOCKER_TAG ?= latest +tester: RUNNER_DOCKER_IMAGE ?= $(TEST_HARNESS_DOCKER_IMAGE) +tester: RUNNER_DOCKER_TAG ?= $(TEST_HARNESS_DOCKER_TAG) +tester: build-harness/runner + +tester/pull: + docker pull $(TEST_HARNESS_DOCKER_IMAGE):$(TEST_HARNESS_DOCKER_TAG) + + +.PHONY: build-harness/runner + +build-harness/runner: + $(info Starting $(RUNNER_DOCKER_IMAGE):$(RUNNER_DOCKER_TAG)) + docker run --name build-harness \ + --rm -it \ + -e PACKAGES_PREFER_HOST=true \ + $(addprefix -e ,$(DOCKER_ENVS)) \ + $(MOUNT_HOME) \ + -v $(CURDIR):/opt \ + --workdir /opt \ + --entrypoint $(ENTRYPOINT) \ + $(RUNNER_DOCKER_IMAGE):$(RUNNER_DOCKER_TAG) $(ARGS) + +.PHONY: reset-owner +reset-owner: + @if [[ -n $$(find . -xdev -user 0 -print) ]]; then \ + printf "\n* To reset ownership on files, run:\n sudo find . -xdev -user 0 -exec chown $$USER {} \;\n\n" ; \ + else \ + printf "\n* No root-owned files found\n\n" ; \ + fi + +endif diff --git a/templates/README.md.gotmpl b/templates/README.md.gotmpl index 4882eacb6..d4c431667 100644 --- a/templates/README.md.gotmpl +++ b/templates/README.md.gotmpl @@ -1,24 +1,35 @@ - +{{- if $deprecated }} +# (deprecated) {{(ds "config").name}}{{ if gt (len (ds "config").name) 23 }}{{ print "\n\n" }}{{ end }} +{{- else }} +# {{(ds "config").name}}{{ if gt (len (ds "config").name) 34 }}{{ print "\n\n" }}{{ end }} +{{- end }} +{{- if $deprecated }}[![deprecated](https://img.shields.io/badge/lifecycle-deprecated-critical)](#deprecated){{ end }} + {{- if has (ds "config") "badges" }} + {{- range $badge := (ds "config").badges }} + {{- printf " [![%s](%s)](%s)" $badge.name $badge.image $badge.url }} + {{- end }} +{{- end }} + +[![README Header][readme_header_img]][readme_header_link] +[![Cloud Posse][logo]](https://cpco.io/homepage) + -{{- defineDatasource "config" .Env.README_YAML | regexp.Replace ".*" "" -}} -{{- defineDatasource "includes" .Env.README_INCLUDES | regexp.Replace ".*" "" }} -[![README Header][readme_header_img]][readme_header_link] - -[![Cloud Posse][logo]](https://cpco.io/homepage) - -# {{(ds "config").name}}{{ if gt (len (ds "config").name) 34 }}{{ print "\n\n" }}{{ end }}{{ if has (ds "config") "badges" }}{{- range $badge := (ds "config").badges -}}{{ printf " [![%s](%s)](%s)" $badge.name $badge.image $badge.url }}{{ end }}{{ end }} - +--> {{ if has (ds "config") "logo" }} ![{{(ds "config").name}}]({{ (ds "config").logo }}) {{- end -}} +{{- if $deprecated }} +## Deprecated + +{{ if has (ds "config").deprecated "notice" }} + {{- (ds "config").deprecated.notice }} +{{- else }} + This module is no longer actively maintained +{{- end }} +{{- if (file.Exists "main.tf") }} + +We literally have [*hundreds of other terraform modules*][terraform_modules] that are Open Source and well-maintained. Check them out! +{{- end }} + {{ if has (ds "config") "description" }} +### Historical Description + {{(ds "config").description }} -{{ end }} +{{- end }} +{{- else }} +{{- if has (ds "config") "description" }} +{{ (ds "config").description }} + +{{- end }} +{{- end }} --- -This project is part of our comprehensive ["SweetOps"](https://cpco.io/sweetops) approach towards DevOps. +This project {{ if $deprecated }}was{{ else }}is{{ end }} part of our comprehensive ["SweetOps"](https://cpco.io/sweetops) approach towards DevOps. [][share_email] [][share_googleplus] [][share_facebook] @@ -86,10 +100,10 @@ It's 100% Open Source and licensed under the [Internet Systems Consortium](LICEN It's 100% Open Source and licensed under the [GNU General Public License](LICENSE). {{ end }} -{{ if (file.Exists "main.tf") }} -We literally have [*hundreds of terraform modules*][terraform_modules] that are Open Source and well-maintained. Check them out! -{{end}} - +{{ if not $deprecated }} +{{- if (file.Exists "main.tf") }} +We literally have [*hundreds of terraform modules*][terraform_modules] that are Open Source and well-maintained. Check them out! +{{- end }} {{ if has (ds "config") "screenshots" }} @@ -97,23 +111,57 @@ We literally have [*hundreds of terraform modules*][terraform_modules] that are {{ range $screenshot := (ds "config").screenshots }} {{ printf "![%s](%s)\n*%s*" $screenshot.name $screenshot.url $screenshot.description }}{{ end }} -{{ end }} +{{- end }} +{{- end }} + {{ if has (ds "config") "introduction" }} ## Introduction -{{ (ds "config").introduction -}} +{{ (ds "config").introduction }} {{ end }} + +{{ if (file.Exists "main.tf") }} +{{- $repo_encoded := (ds "config").github_repo | regexp.Replace "/" "%2F" -}} +## Security & Compliance [](https://bridgecrew.io/) + +Security scanning is graciously provided by Bridgecrew. Bridgecrew is the leading fully hosted, cloud-native solution providing continuous Terraform security and compliance. + +| Benchmark | Description | +|--------|---------------| +{{- if $deprecated }} +| [![deprecated](https://img.shields.io/badge/lifecycle-deprecated-critical)](#deprecated) | This project is no longer being maintained | +{{- end }} +| [![Infrastructure Security]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/general" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=INFRASTRUCTURE+SECURITY" $repo_encoded }}) | Infrastructure Security Compliance | +| [![CIS KUBERNETES]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/cis_kubernetes" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=CIS+KUBERNETES+V1.5" $repo_encoded }}) | Center for Internet Security, KUBERNETES Compliance | +| [![CIS AWS]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/cis_aws" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=CIS+AWS+V1.2" $repo_encoded }}) | Center for Internet Security, AWS Compliance | +| [![CIS AZURE]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/cis_azure" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=CIS+AZURE+V1.1" $repo_encoded }}) | Center for Internet Security, AZURE Compliance | +| [![PCI-DSS]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/pci" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=PCI-DSS+V3.2" $repo_encoded }}) | Payment Card Industry Data Security Standards Compliance | +| [![NIST-800-53]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/nist" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=NIST-800-53" $repo_encoded }}) | National Institute of Standards and Technology Compliance | +| [![ISO27001]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/iso" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=ISO27001" $repo_encoded }}) | Information Security Management System, ISO/IEC 27001 Compliance | +| [![SOC2]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/soc2" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=SOC2" $repo_encoded }})| Service Organization Control 2 Compliance | +| [![CIS GCP]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/cis_gcp" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=CIS+GCP+V1.1" $repo_encoded }}) | Center for Internet Security, GCP Compliance | +| [![HIPAA]({{ printf "https://www.bridgecrew.cloud/badges/github/%s/hipaa" (ds "config").github_repo}})]({{ printf "https://www.bridgecrew.cloud/link/badge?vcs=github&fullRepo=%s&benchmark=HIPAA" $repo_encoded }}) | Health Insurance Portability and Accountability Compliance | +{{ end }} + {{ if has (ds "config") "usage" }} ## Usage {{ if (file.Exists "main.tf") }} -**IMPORTANT:** The `master` branch is used in `source` just as an example. In your code, do not pin to `master` because there may be breaking changes between releases. -Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases]({{ printf "https://github.com/%s/releases" (ds "config").github_repo}}). +**IMPORTANT:** We do not pin modules to versions in our examples because of the +difficulty of keeping the versions in the documentation in sync with the latest released versions. +We highly recommend that in your code you pin the version to the exact version you are +using so that your infrastructure remains stable, and update versions in a +systematic way so that they do not catch you by surprise. + +Also, because of a bug in the Terraform registry ([hashicorp/terraform#21417](https://github.com/hashicorp/terraform/issues/21417)), +the registry shows many of our inputs as required when in fact they are optional. +The table below correctly indicates which inputs are required. {{end}} {{ (ds "config").usage -}} {{ end }} +{{ if not $deprecated -}} {{ if has (ds "config") "quickstart" -}} ## Quick Start @@ -133,32 +181,34 @@ Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest re {{- end }} {{ if has (ds "config") "related" }} -## Share the Love +## Share the Love -Like this project? Please give it a ★ on [our GitHub]({{ printf "https://github.com/%s" (ds "config").github_repo}})! (it helps us **a lot**) +Like this project? Please give it a ★ on [our GitHub]({{ printf "https://github.com/%s" (ds "config").github_repo}})! (it helps us **a lot**) Are you using this project or any of our other projects? Consider [leaving a testimonial][testimonial]. =) - +{{ end }} +{{ if has (ds "config") "related" }} ## Related Projects Check out these related projects. {{ range $related := (ds "config").related }} {{ printf "- [%s](%s) - %s" $related.name $related.url $related.description }}{{ end }} -{{ end}} +{{- end}} +{{- end}} {{ if has (ds "config") "references" }} ## References -For additional context, refer to some of these links. +For additional context, refer to some of these links. {{ range $reference := (ds "config").references }} {{ printf "- [%s](%s) - %s" $reference.name $reference.url $reference.description }}{{ end }} {{ end}} ## Help -**Got a question?** We got answers. +**Got a question?** We got answers. File a GitHub [issue]({{ printf "https://github.com/%s/issues" (ds "config").github_repo}}), send us an [email][email] or join our [Slack Community][slack]. @@ -167,7 +217,7 @@ File a GitHub [issue]({{ printf "https://github.com/%s/issues" (ds "config").git ## DevOps Accelerator for Startups -We are a [**DevOps Accelerator**][commercial_support]. We'll help you build your cloud infrastructure from the ground up so you can own it. Then we'll show you how to operate it and stick around for as long as you need us. +We are a [**DevOps Accelerator**][commercial_support]. We'll help you build your cloud infrastructure from the ground up so you can own it. Then we'll show you how to operate it and stick around for as long as you need us. [![Learn More](https://img.shields.io/badge/learn%20more-success.svg?style=for-the-badge)][commercial_support] @@ -190,16 +240,21 @@ We deliver 10x the value for a fraction of the cost of a full-time engineer. Our Join our [Open Source Community][slack] on Slack. It's **FREE** for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally *sweet* infrastructure. +## Discourse Forums + +Participate in our [Discourse Forums][discourse]. Here you'll find answers to commonly asked questions. Most questions will be related to the enormous number of projects we support on our GitHub. Come here to collaborate on answers, find solutions, and get ideas about the products and services we value. It only takes a minute to get started! Just sign in with SSO using your GitHub account. + ## Newsletter -Sign up for [our newsletter][newsletter] that covers everything on our technology radar. Receive updates on what we're up to on GitHub as well as awesome new projects we discover. +Sign up for [our newsletter][newsletter] that covers everything on our technology radar. Receive updates on what we're up to on GitHub as well as awesome new projects we discover. ## Office Hours -[Join us every Wednesday via Zoom][office_hours] for our weekly "Lunch & Learn" sessions. It's **FREE** for everyone! +[Join us every Wednesday via Zoom][office_hours] for our weekly "Lunch & Learn" sessions. It's **FREE** for everyone! [![zoom](https://img.cloudposse.com/fit-in/200x200/https://cloudposse.com/wp-content/uploads/2019/08/Powered-by-Zoom.png")][office_hours] +{{ if not $deprecated -}} ## Contributing ### Bug Reports & Feature Requests @@ -219,7 +274,7 @@ In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow. 5. Submit a **Pull Request** so that we can review your changes **NOTE:** Be sure to merge the latest changes from "upstream" before making a pull request! - +{{ end }} {{ if has (ds "config") "copyrights" }} ## Copyrights @@ -234,33 +289,35 @@ Copyright © 2017-{{ time.Now.Year }} [Cloud Posse, LLC](https://cpco.io/copyrig {{ end}} {{ if eq (ds "config").license "APACHE2" }} -## License +## License -[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) See [LICENSE](LICENSE) for full details. - Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - https://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. +```text +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +``` {{ end }} {{ if eq (ds "config").license "CC-BY-NC-SA-4.0" }} -## License +## License -[![License](https://img.shields.io/badge/License-CC%20BY%20NC%20SA%204.0-blue.svg)](https://creativecommons.org/licenses/by-nc-sa/4.0/) +[![License](https://img.shields.io/badge/License-CC%20BY%20NC%20SA%204.0-blue.svg)](https://creativecommons.org/licenses/by-nc-sa/4.0/) @@ -280,10 +337,11 @@ Distribution of the work or derivative of the work in any standard (paper) book {{ end }} {{ if eq (ds "config").license "MIT" }} -## License +## License [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +```text The MIT License (MIT) Permission is hereby granted, free of charge, to any person obtaining a copy @@ -305,13 +363,15 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Source: +``` {{ end }} {{ if eq (ds "config").license "ISC" }} -## License +## License [![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](https://opensource.org/licenses/ISC) +```text ISC License (ISC) Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. @@ -319,14 +379,16 @@ Permission to use, copy, modify, and/or distribute this software for any purpose THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. Source: +``` {{ end }} {{ if eq (ds "config").license "GPL3" }} -## License +## License [![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0) -GNU GENERAL PUBLIC LICENSE +```text +GNU GENERAL PUBLIC LICENSE Version 3, 29 June 2007 This program is free software: you can redistribute it and/or modify @@ -341,6 +403,7 @@ GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . +``` {{ end }} ## Trademarks @@ -355,7 +418,7 @@ This project is maintained and funded by [Cloud Posse, LLC][website]. Like it? P We're a [DevOps Professional Services][hire] company based in Los Angeles, CA. We ❤️ [Open Source Software][we_love_open_source]. -We offer [paid support][commercial_support] on all of our projects. +We offer [paid support][commercial_support] on all of our projects. Check out [our other projects][github], [follow us on twitter][twitter], [apply for a job][jobs], or [hire us][hire] to help with your cloud strategy and implementation. @@ -363,8 +426,10 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply {{ if has (datasource "config") "contributors" }} ### Contributors + | {{ range $contributor := (ds "config").contributors }}{{ printf " [![%s][%s_avatar]][%s_homepage]
[%s][%s_homepage] |" $contributor.name $contributor.github $contributor.github $contributor.name $contributor.github}}{{ end }} |{{- range $contributor := (ds "config").contributors -}}---|{{ end }} + {{ range $contributor := (ds "config").contributors -}} {{- if has $contributor "homepage" }} @@ -396,6 +461,7 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply [testimonial]: {{ printf $utm_link "https://cpco.io/leave-testimonial" "testimonial" }} [office_hours]: {{ printf $utm_link "https://cloudposse.com/office-hours" "office_hours" }} [newsletter]: {{ printf $utm_link "https://cpco.io/newsletter" "newsletter" }} + [discourse]: {{ printf $utm_link "https://ask.sweetops.com/" "discourse" }} [email]: {{ printf $utm_link "https://cpco.io/email" "email" }} [commercial_support]: {{ printf $utm_link "https://cpco.io/commercial-support" "commercial_support" }} [we_love_open_source]: {{ printf $utm_link "https://cpco.io/we-love-open-source" "we_love_open_source" }} diff --git a/templates/terraform/.github/mergify.yml b/templates/terraform/.github/mergify.yml new file mode 100644 index 000000000..ef15545ec --- /dev/null +++ b/templates/terraform/.github/mergify.yml @@ -0,0 +1,65 @@ +# https://docs.mergify.io/conditions.html +# https://docs.mergify.io/actions.html +pull_request_rules: +- name: "approve automated PRs that have passed checks" + conditions: + - "author~=^(cloudpossebot|renovate\\[bot\\])$" + - "base=master" + - "-closed" + - "head~=^(auto-update|renovate)/.*" + - "check-success=test/bats" + - "check-success=test/readme" + - "check-success=test/terratest" + - "check-success=validate-codeowners" + actions: + review: + type: "APPROVE" + bot_account: "cloudposse-mergebot" + message: "We've automatically approved this PR because the checks from the automated Pull Request have passed." + +- name: "merge automated PRs when approved and tests pass" + conditions: + - "author~=^(cloudpossebot|renovate\\[bot\\])$" + - "base=master" + - "-closed" + - "head~=^(auto-update|renovate)/.*" + - "check-success=test/bats" + - "check-success=test/readme" + - "check-success=test/terratest" + - "check-success=validate-codeowners" + - "#approved-reviews-by>=1" + - "#changes-requested-reviews-by=0" + - "#commented-reviews-by=0" + actions: + merge: + method: "squash" + +- name: "delete the head branch after merge" + conditions: + - "merged" + actions: + delete_head_branch: {} + +- name: "ask to resolve conflict" + conditions: + - "conflict" + - "-closed" + actions: + comment: + message: "This pull request is now in conflict. Could you fix it @{{author}}? 🙏" + +- name: "remove outdated reviews" + conditions: + - "base=master" + actions: + dismiss_reviews: + changes_requested: true + approved: true + message: "This Pull Request has been updated, so we're dismissing all reviews." + +- name: "close Pull Requests without files changed" + conditions: + - "#files=0" + actions: + close: + message: "This pull request has been automatically closed by Mergify because there are no longer any changes." diff --git a/templates/terraform/.github/renovate.json b/templates/terraform/.github/renovate.json new file mode 100644 index 000000000..ae4f0aa54 --- /dev/null +++ b/templates/terraform/.github/renovate.json @@ -0,0 +1,12 @@ +{ + "extends": [ + "config:base", + ":preserveSemverRanges" + ], + "labels": ["auto-update"], + "enabledManagers": ["terraform"], + "terraform": { + "ignorePaths": ["**/context.tf", "examples/**"] + } +} + diff --git a/templates/terraform/.github/workflows/auto-context.yml b/templates/terraform/.github/workflows/auto-context.yml new file mode 100644 index 000000000..ab979e0eb --- /dev/null +++ b/templates/terraform/.github/workflows/auto-context.yml @@ -0,0 +1,57 @@ +name: "auto-context" +on: + schedule: + # Update context.tf nightly + - cron: '0 3 * * *' + +jobs: + update: + if: github.event_name == 'schedule' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + + - name: Update context.tf + shell: bash + id: update + env: + GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}" + run: | + if [[ -f context.tf ]]; then + echo "Discovered existing context.tf! Fetching most recent version to see if there is an update." + curl -o context.tf -fsSL https://raw.githubusercontent.com/cloudposse/terraform-null-label/master/exports/context.tf + if git diff --no-patch --exit-code context.tf; then + echo "No changes detected! Exiting the job..." + else + echo "context.tf file has changed. Update examples and rebuild README.md." + make init + make github/init/context.tf + make readme/build + echo "::set-output name=create_pull_request::true" + fi + else + echo "This module has not yet been updated to support the context.tf pattern! Please update in order to support automatic updates." + fi + + - name: Create Pull Request + if: steps.update.outputs.create_pull_request == 'true' + uses: cloudposse/actions/github/create-pull-request@0.22.0 + with: + token: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} + committer: 'cloudpossebot <11232728+cloudpossebot@users.noreply.github.com>' + author: 'cloudpossebot <11232728+cloudpossebot@users.noreply.github.com>' + commit-message: Update context.tf from origin source + title: Update context.tf + body: |- + ## what + This is an auto-generated PR that updates the `context.tf` file to the latest version from `cloudposse/terraform-null-label` + + ## why + To support all the features of the `context` interface. + + branch: auto-update/context.tf + base: master + delete-branch: true + labels: | + auto-update + context diff --git a/templates/terraform/.github/workflows/auto-format.yml b/templates/terraform/.github/workflows/auto-format.yml new file mode 100644 index 000000000..375d0fd47 --- /dev/null +++ b/templates/terraform/.github/workflows/auto-format.yml @@ -0,0 +1,88 @@ +name: Auto Format +on: + pull_request_target: + types: [opened, synchronize] + +jobs: + auto-format: + runs-on: ubuntu-latest + container: cloudposse/build-harness:latest + steps: + # Checkout the pull request branch + # "An action in a workflow run can’t trigger a new workflow run. For example, if an action pushes code using + # the repository’s GITHUB_TOKEN, a new workflow will not run even when the repository contains + # a workflow configured to run when push events occur." + # However, using a personal access token will cause events to be triggered. + # We need that to ensure a status gets posted after the auto-format commit. + # We also want to trigger tests if the auto-format made no changes. + - uses: actions/checkout@v2 + if: github.event.pull_request.state == 'open' + name: Privileged Checkout + with: + token: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} + repository: ${{ github.event.pull_request.head.repo.full_name }} + # Check out the PR commit, not the merge commit + # Use `ref` instead of `sha` to enable pushing back to `ref` + ref: ${{ github.event.pull_request.head.ref }} + + # Do all the formatting stuff + - name: Auto Format + if: github.event.pull_request.state == 'open' + shell: bash + env: + GITHUB_TOKEN: "${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }}" + run: make BUILD_HARNESS_PATH=/build-harness PACKAGES_PREFER_HOST=true -f /build-harness/templates/Makefile.build-harness pr/auto-format/host + + # Commit changes (if any) to the PR branch + - name: Commit changes to the PR branch + if: github.event.pull_request.state == 'open' + shell: bash + id: commit + env: + SENDER: ${{ github.event.sender.login }} + run: | + set -x + output=$(git diff --name-only) + + if [ -n "$output" ]; then + echo "Changes detected. Pushing to the PR branch" + git config --global user.name 'cloudpossebot' + git config --global user.email '11232728+cloudpossebot@users.noreply.github.com' + git add -A + git commit -m "Auto Format" + # Prevent looping by not pushing changes in response to changes from cloudpossebot + [[ $SENDER == "cloudpossebot" ]] || git push + # Set status to fail, because the push should trigger another status check, + # and we use success to indicate the checks are finished. + printf "::set-output name=%s::%s\n" "changed" "true" + exit 1 + else + printf "::set-output name=%s::%s\n" "changed" "false" + echo "No changes detected" + fi + + - name: Auto Test + uses: cloudposse/actions/github/repository-dispatch@0.22.0 + # match users by ID because logins (user names) are inconsistent, + # for example in the REST API Renovate Bot is `renovate[bot]` but + # in GraphQL it is just `renovate`, plus there is a non-bot + # user `renovate` with ID 1832810. + # Mergify bot: 37929162 + # Renovate bot: 29139614 + # Cloudpossebot: 11232728 + # Need to use space separators to prevent "21" from matching "112144" + if: > + contains(' 37929162 29139614 11232728 ', format(' {0} ', github.event.pull_request.user.id)) + && steps.commit.outputs.changed == 'false' && github.event.pull_request.state == 'open' + with: + token: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} + repository: cloudposse/actions + event-type: test-command + client-payload: |- + { "slash_command":{"args": {"unnamed": {"all": "all", "arg1": "all"}}}, + "pull_request": ${{ toJSON(github.event.pull_request) }}, + "github":{"payload":{"repository": ${{ toJSON(github.event.repository) }}, + "comment": {"id": ""} + } + } + } diff --git a/templates/terraform/.github/workflows/chatops.yml b/templates/terraform/.github/workflows/chatops.yml new file mode 100644 index 000000000..4ddc06749 --- /dev/null +++ b/templates/terraform/.github/workflows/chatops.yml @@ -0,0 +1,37 @@ +name: chatops +on: + issue_comment: + types: [created] + +jobs: + default: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - name: "Handle common commands" + uses: cloudposse/actions/github/slash-command-dispatch@0.22.0 + with: + token: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} + reaction-token: ${{ secrets.GITHUB_TOKEN }} + repository: cloudposse/actions + commands: rebuild-readme, terraform-fmt + permission: triage + issue-type: pull-request + + test: + runs-on: ubuntu-latest + steps: + - name: "Checkout commit" + uses: actions/checkout@v2 + - name: "Run tests" + uses: cloudposse/actions/github/slash-command-dispatch@0.22.0 + with: + token: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} + reaction-token: ${{ secrets.GITHUB_TOKEN }} + repository: cloudposse/actions + commands: test + permission: triage + issue-type: pull-request + reactions: false + +