Skip to content

Latest commit

 

History

History

project_setup

README.md

Project Setup

We recommend you follow the directions below and use a skeleton for all new repositories.

The skeleton helper tool included in the scripts directory can quickly setup a new local repository. Once you've cloned and configured the repository to your local machine, it can be published to a repository created on GitHub.

For repositories created from skeletons, run setup-env and pre-commit install to set up your environment, enable linting and other tools to prevent new commits from immediately running into linting failures.

Once you've set up a repository, make sure to enable branch protection - see our branch protection guide for details.

Contents

Adding a skeleton configuration to an existing repository

To skeletonize an existing repository, please see the guide to skeletonize an existing repository.

Using the skeleton tool to start a new repository 💀🛠

We recommend that you start your project from one of the skeleton projects in this organization. The skeleton helper tool included in the scripts directory can quickly setup a new local repository. Once you've cloned and configured the repository to your local machine, you then publish it to a repository created on GitHub.

Selecting a skeleton

First, identify a suitable skeleton project to use as the starting point for your new repository. For a list of available skeletons, see the Skeleton List or use the following command:

./skeleton list
Available skeletons in cisagov:

skeleton-python-library
    A skeleton project for quickly getting a new cisagov Python library started.

skeleton-docker
    A skeleton project for quickly getting a new cisagov Docker container started.

skeleton-generic
    A generic skeleton project for quickly getting a new cisagov project started.

skeleton-tf-module
    A skeleton project for quickly getting a new cisagov Terraform module started.

skeleton-ansible-role
    A skeleton project for quickly getting a new cisagov Ansible role started.

skeleton-ansible-role-with-test-user
    A skeleton project for quickly getting a new cisagov Ansible role started when that role requires an AWS test user.

skeleton-packer
    A skeleton project for quickly getting a new cisagov packer project started.

skeleton-aws-lambda
    A skeleton project for quickly getting a new cisagov Python AWS Lambda started.

Cloning a selected skeleton

Next, use the skeleton tool to clone, rename, and prepare the contents of your new repository for publication. The tool will print out each command it is issuing and its result.

./skeleton clone [options] <parent-repo-name> <new-repo-name>

For example, to create a project based on skeleton-ansible-role named ansible-role-quantum-rng in your local ~/projects directory:

./skeleton clone --change-dir ~/projects skeleton-ansible-role ansible-role-quantum-rng

This command results in:

――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
💬 Clone an existing remote repository to the new name locally.
➤  git clone [email protected]:cisagov/skeleton-ansible-role.git ansible-role-quantum-rng
Cloning into 'ansible-role-quantum-rng'...
✅ success
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
💬 Disable pushing to the upstream (parent) repository.
➤  git remote set-url --push skeleton-ansible-role no_push
✅ success
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
💬 Add a new remote origin for the this repository.
➤  git remote add origin [email protected]:cisagov/ansible-role-quantum-rng.git
✅ success
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
💬 Search and replace repository name in source files.
➤  find . \( ! -regex '.*/\.git/.*' \) -type f -print0 | xargs -0 sed -i "" "s/skeleton-ansible-role/ansible-role-quantum-rng/g"
✅ success
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
💬 Stage modified files.
➤  git add --verbose .
add 'CONTRIBUTING.md'
add 'README.md'
add 'molecule/default/playbook.yml'
✅ success
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
💬 Commit staged files to the new repository.
➤  git commit --message "Rename repository references after clone."
[develop 565e041] Rename repository references after clone.
 3 files changed, 10 insertions(+), 10 deletions(-)
✅ success
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

The repository "skeleton-ansible-role" has been cloned and renamed to "ansible-role-quantum-rng".
Use the following commands to push the new repository to github:
    cd ~/projects/ansible-role-quantum-rng
    git push --set-upstream origin develop

Create and publish the GitHub repository

Once the skeleton tool has run, the next step is to publish to a GitHub remote repository.

To publish your new repository on GitHub, the remote must already exist. Create a new repository on GitHub with the same name as your new local repository. If you do not have permission, ask an administrator to create it for you.

Add the repository name and description, set the repository to public, and skip the rest of the options.

Next, publish your new repository to GitHub:

git push --set-upstream origin develop

Disabling squash merging

Click on the settings tab for your new repository and, in the "Options" section, make sure that "Allow squash merging" is unchecked.

Set up your environment and pre-commit

Follow the instructions in CONTRIBUTING.md on setting up pre-commit to run setup-env and enable the pre-commit hooks. If you have already set up the prerequisites, this involves:

# In the root directory of the repository
./setup-env

Create an initial pull request

You probably want to add code, documentation, and other items to your repository to customize it from the skeleton and make changes.

Create a new branch called first-commits and publish it:

git checkout -b first-commits
git push origin first-commits --set-upstream

Edit the existing files and add your own as needed, then commit your changes, push them, and create a pull request (PR) via GitHub or the command-line for your teammates to review.

GitHub only populates its list of status checks once a PR has been created so checks can run against changes. Status checks are extremely useful for quality control and automated testing, so we require these checks to pass before merging. This first PR will ensure your new repository is ready to go and give your teammates a chance to review your code before merging it.

If a status check doesn't apply to your new repository, leave it enabled anyway - it won't hurt anything.

Setting up branch protection

Once you've made your initial pull request, enable branch protection to enforce the codeowners approval requirements for pull requests.

Setting up type-specific configuration settings

Setting up Coveralls for Python projects

The README for your new Python project will be prepared with a Coveralls badge. To make the badge work properly, you'll need to add a repository secret.

  1. Visit Coveralls and go to Add Repos.
  2. Select your new repository and enable it. This will take you to a page with Python set up for Coveralls. The code block will have an entry for repo_token: <token>.
  3. Copy the repo_token value.
  4. On GitHub, visit your new repository's Settings -> Secrets page.
    • Note: If you don't have access to Settings, please contact an administrator to do this step for you.
  5. Add a New repository secret and name it COVERALLS_REPO_TOKEN with the value from Coveralls.

Ansible requirement file generation tool 🧻🛠

We have a plethora of ansible-roles in our organization. To facilitate the creation of a requirements.yml file used in an Ansible project, we have created the ansible-roles tool located in the scripts directory. The tool will output yml for all the current (non-archived) Ansible role repositories. A common usage of the tool is:

./ansible-roles > myproject/src/requirements.yml

This file will now contain definitions for all the Ansible roles. Edit the file, and remove any role that will not be required for your project.

Terraform IAM credentials to GitHub secrets 🔑‍👉🤫

When GitHub Actions workflows require credentials to run we provide them via secrets. This usually involves extracting the secrets from the Terraform state json output. Then some pointing, clicking, cutting and pasting on the repository's settings.

To simplify this task use the terraform-to-secrets tool located in the scripts directory. The tool will create secrets using your personal access token (PAT). Note: Your PAT needs to have the "repo" scope set.

Execute the tool from your GitHub project's terraform directory:

terraform-to-secrets 9f4ae878de917c7cf191b9861d3c1cf9224939f7
2020-02-22 15:50:36,059 INFO Using GitHub repository name: cisagov/ansible-role-dev-ssh-access
2020-02-22 15:50:36,060 INFO Searching Terraform state for IAM credentials.
2020-02-22 15:50:40,643 INFO Found credentials for user: test-ansible-role-dev-ssh-access
2020-02-22 15:50:40,643 INFO Creating GitHub API session using personal access token.
2020-02-22 15:50:40,644 INFO Requesting public key for repository cisagov/ansible-role-dev-ssh-access
2020-02-22 15:50:40,832 INFO Setting secrets for user: test-ansible-role-dev-ssh-access
2020-02-22 15:50:40,832 INFO Creating secret AWS_ACCESS_KEY_ID
2020-02-22 15:50:41,027 INFO Creating secret AWS_SECRET_ACCESS_KEY
2020-02-22 15:50:41,036 INFO Success!

Managing SSM parameters from files 🗂👉☁️

Use the ssm-param tool to copy files into SSM parameters in multiple regions simultaneously:

ssm-param cp *.pem /demo/
2019-08-27 16:45:58,651 INFO Found credentials in shared credentials file: ~/.aws/credentials
2019-08-27 16:45:58,744 INFO Putting parameter '/demo/dh4096.pem' in region 'us-east-1'
2019-08-27 16:45:58,981 INFO Putting parameter '/demo/dh4096.pem' in region 'us-east-2'
2019-08-27 16:45:59,327 INFO Putting parameter '/demo/dh4096.pem' in region 'us-west-1'
2019-08-27 16:45:59,887 INFO Putting parameter '/demo/dh4096.pem' in region 'us-west-2'
2019-08-27 16:46:00,363 INFO Putting parameter '/demo/private.pem' in region 'us-east-1'
2019-08-27 16:46:00,480 INFO Putting parameter '/demo/private.pem' in region 'us-east-2'
2019-08-27 16:46:00,640 INFO Putting parameter '/demo/private.pem' in region 'us-west-1'
2019-08-27 16:46:01,008 INFO Putting parameter '/demo/private.pem' in region 'us-west-2'
2019-08-27 16:46:01,414 INFO Putting parameter '/demo/public.pem' in region 'us-east-1'
2019-08-27 16:46:01,553 INFO Putting parameter '/demo/public.pem' in region 'us-east-2'
2019-08-27 16:46:01,718 INFO Putting parameter '/demo/public.pem' in region 'us-west-1'
2019-08-27 16:46:02,070 INFO Putting parameter '/demo/public.pem' in region 'us-west-2'

It can also delete parameters from multiple regions:

ssm-param rm /demo/dh4096.pem /demo/private.pem /demo/public.pem
2019-08-27 16:47:59,384 INFO Found credentials in shared credentials file: ~/.aws/credentials
2019-08-27 16:47:59,478 INFO Deleting parameter '/demo/dh4096.pem' in region 'us-east-1'
2019-08-27 16:47:59,715 INFO Deleting parameter '/demo/dh4096.pem' in region 'us-east-2'
2019-08-27 16:48:00,003 INFO Deleting parameter '/demo/dh4096.pem' in region 'us-west-1'
2019-08-27 16:48:00,523 INFO Deleting parameter '/demo/dh4096.pem' in region 'us-west-2'
2019-08-27 16:48:01,065 INFO Deleting parameter '/demo/private.pem' in region 'us-east-1'
2019-08-27 16:48:01,202 INFO Deleting parameter '/demo/private.pem' in region 'us-east-2'
2019-08-27 16:48:01,355 INFO Deleting parameter '/demo/private.pem' in region 'us-west-1'
2019-08-27 16:48:01,728 INFO Deleting parameter '/demo/private.pem' in region 'us-west-2'
2019-08-27 16:48:02,138 INFO Deleting parameter '/demo/public.pem' in region 'us-east-1'
2019-08-27 16:48:02,269 INFO Deleting parameter '/demo/public.pem' in region 'us-east-2'
2019-08-27 16:48:02,417 INFO Deleting parameter '/demo/public.pem' in region 'us-west-1'
2019-08-27 16:48:02,795 INFO Deleting parameter '/demo/public.pem' in region 'us-west-2'