Accommodate file encryption via ansible vault

[fullyint]

Moves some variables to group_vars/<environment>/vault.yml files.

To start using Ansible Vault:

1. copy .vault_pass.example to .vault_pass, edit the password, and probably chmod 600 .vault_pass
2. edit ansible.cfg:

- vault_password_file = .vault_pass.example
+ vault_password_file = .vault_pass

3. encrypt vault.yml files with command below (doesn't support fileglobs -- "ansible-vault encrypt" will not accept multiple files at once ansible/ansible#6241):

ansible-vault encrypt group_vars/all/vault.yml group_vars/development/vault.yml group_vars/staging/vault.yml group_vars/production/vault.yml

Preliminary discussion in #308 where I mentioned these items:

• encryption is optional
• there are scripts to help one avoid committing unencrypted password files

Would we want mysql_root_user: root vaulted? It is currently in group_vars/all/database.yml.

[swalkinshaw]

@fullyint no need for mysql_root_user to be vaulted.

[swalkinshaw]

Shouldn't this be set to .vault_pass? It's one less thing to change if users want it enabled.

[fullyint]

The playbook fails if vault_password_file refers to a missing file. The file .vault_pass will be missing by default because it is .gitignored. Users who don't want ansible vault would have to remove this line or create the .vault_pass file. I explain below why I didn't just omit this line and require vault users to add it.

To enable vault, we could have users create new stuff: add vault_password_file line to ansible.cfg and create the .vault_pass file. I chose the alternative of having users edit example versions of each (steps 1 and 2 in this PR's first comment) because I speculate that users would be more likely to try vault if they could edit example stuff instead of create new stuff.

An added bonus of having the examples present in the repo is that one can actually skip steps 1 and 2, and do only step 3 (run the ansible-vault encrypt command). That means one can test how easy it is to use ansible vault by just running that single command.

[swalkinshaw]

I had naively hoped that Ansible would just skip using vault_password_file if nothing was encrypted so that this could be set to .vault_pass.

[fullyint]

I originally assumed it would skip. That's what I would have preferred.

[swalkinshaw]

@fullyint let's move forward with this 👍

Can you work on a new doc section?

[mxxcon]

I'm sorry if this was discussed somewhere earlier, but why are you completely moving some variables into vaults?
I'm sure many of you already read https://www.reinteractive.net/posts/167-ansible-real-life-good-practices Why not use structure similar to shown there?
For example: in group_vars/production/wordpress_sites.yml you remove almost every single record related to database info.
Why not instead keep the current structure but assign variables to their vaulted counterparts:

      db_name: {{ vaulted_db_name }}
      db_user: {{ vaulted_db_user }}
      db_password: {{ vaulted_db_password }}

And then your vault.yml will contain

   vaulted_db_name: wp_myblog
   vaulted_db_user: wp_myblog_user
   vaulted_db_password: password123

This has the benefit of minimizing change to playbooks and keeps existence of variables more discoverable, while still getting the benefit of ansible-vault. And if a user decides not to use vault, they can just simply replace {{ }} values with plain text.

[austinpray]

I agree with the approach @mxxcon suggests. You should not have to look at two files to get the full picture.

[QWp6t]

👍

[fullyint]

@mxxcon I like that approach but I couldn't implement it directly because wordpress_sites is a hash.

There would be no problem with variables such as vaulted_db_name for a Trellis project with a single site. When there are multiple sites, that variable must somehow account for which site it corresponds to. The big challenge is how to loop over multiple sites in wordpress_sites when some values are vaulted.

You'll see how I approached the issue by searching this PR's diff for wordpress_sites_vault[item.key].

I'd be delighted if anyone would offer a better working implementation.

[mxxcon]

I see. I have not fully grokked trellis yet, so I'll defer to you guys since you are more familiar with its challenges. 😃

[fullyint]

Docs at roots/docs#5. I think this is ready. Thank you, @mxxcon, your comments inspired good revisions.

A Few Updates
I made two changes for better discoverability and to provide the full picture in wordpress_sites.yml.

1. I switched to using the structure @mxxcon suggested (pseudo leaf, recommended by ansible) for scalar variables (not used in loops), e.g., mail_password: "{{ vault_mail_password }}".

2. For the hash variables (used in loops), I added their names in comments where the variables would have appeared in the hash if left unencrypted. That way the unencrypted file shows the full picture and the variables still show up in searches.

wordpress_sites:
  example.com:
    ⋮
    env:
      wp_home: http://example.dev
      wp_siteurl: http://example.dev/wp
      wp_env: development
      # Define the following variables in group_vars/development/vault.yml
      # db_name:
      # db_user:
      # db_password:
      ⋮

One other change worth noting was that I removed the .vault_pass.example file and the related example default in ansible.cfg, changing my position on this decision.

Vault with Variables in a Hash
I believe I've used a good strategy for handling the encrypted variables from the wordpress_sites hash. I don't believe the pseudo leaf structure can be applied effectively to a hash that will be used in tasks that loop. Here are a few examples of failed or suboptimal implementations of the pseudo leaf structure to the wordpress_sites hash.

# 1 - Fail: If all sites use `vault_db_password`, they all have the same password
wordpress_sites:
  site-1.com:
    db_password: "{{ vault_db_password }}"
  site-2.com:
    db_password: "{{ vault_db_password }}"

# 2 - Suboptimal: Requiring users to edit variable names means more work and more risk for error
wordpress_sites:
  site-1.com:
    db_password: "{{ vault_db_password_1 }}"
  site-2.com:
    db_password: "{{ vault_db_password_2 }}"

# 3 - Fail: The `item.key` variable is only defined in the scope of the looping task and will be undefined in the more global scope of the example below
wordpress_sites:
  site-1.com:
    db_password: "{{ vault_wordpress_sites[item.key].env.db_password }}"
  site-2.com:
    db_password: "{{ vault_wordpress_sites[item.key].env.db_password }}"

[swalkinshaw]

admin_email shouldn't be in vault

[swalkinshaw]

admin_user as well?

[mxxcon]

@swalkinshaw why not? Do you not consider these to be sensitive?

Speaking of sensitive info, unless I'm misunderstanding this, is it proper for group_vars/all/vault.yml to contain vault_sudoer_passwords.admin variable? Shouldn't it be unique per environment(dev/stage/prod), if not even unique per host?

[swalkinshaw]

@mxxcon not really. Keep in mind all of this data should still be kept in private repos. I only consider "secrets" as applicable for vault.

I agree that vault_sudoer_passwords should probably be env dependent.

[fullyint]

As @swalkinshaw suggested, I removed a few vars from the vault:

• WP's admin_user and admin_email
• env vars: db_user and db_name

As @mxxcon suggested, I split out vault_sudoer_passwords into the separate group_vars/<environment>/vault.yml files.

I updated the docs PR to reflect the above changes.