Vault write support

[stuart-c]

This requires #177 to be merged first.

This PR adds write support to the Vault datasource, enabling use of dynamic secrets (such as database credentials).

[hairyhenderson]

Thanks @stuart-c! I like the idea. I'm not too sure about some of the design, but I'll take a closer look after #177 is merged.

[hairyhenderson]

I've been a little uneasy about the design and I just put my finger on it this morning.

$ gomplate -d vault=vault:/// -i 'otp={{(datasource "vault" "ssh/creds/test" "ip=10.1.2.3" "username=user").key}}'
otp=604a4bd5-7afd-30a2-d2d8-80c4aebc6183

The trouble with this is that this particular template can only be used with a vault datasource. It wouldn't allow us to use a JSON file instead without altering the template.

I think using the URL might be the key here, and maybe we can use query parameters to specify the ip and username. That'd allow usage like this:

$ gomplate -d 'vault=vault:///?ip=10.1.2.3&username=user' -i 'otp={{ (ds "vault" "ssh/creds/test").key }}'

@stuart-c what do you think?

[stuart-c]

The problem then is you'd end up with hundreds of -d entries on the command line with the resultant multiple Vault connections/authentications [would also break AWS EC2 auth unless we have nonce saving].

In terms of when I'd be looking to use this:

• Getting certificates using the PKI backend
• Generate database credentials (Cassandra, MongoDB, MSSQL, MySQL/MariaDB, PostgreSQL, etc.)
• Generate AWS & RabbitMQ credentials

[stuart-c]

Thinking about it a bit more, in general I don't think you can swap a Vault datasource for a HTTP or file based one - only Vault & Consul use args, which are an important part of the request.

The only situation it would work is if you were to have only a single Vault/Consul template entry for a specific key.

For example if you had the following template:

{{ (datasource "vault" "secret/key1").value }}
{{ (datasource "vault" "secret/key2").value }}

If you use Vault you'd get two different values (assuming the two secrets are different in Vault), but switching to HTTP or file they would be identical.

[hairyhenderson]

Thinking about it a bit more, in general I don't think you can swap a Vault datasource for a HTTP or file based one - only Vault & Consul use args, which are an important part of the request.

The only situation it would work is if you were to have only a single Vault/Consul template entry for a specific key.

Right - like the following:

$ gomplate -d 'secret=vault:///ssh/creds/test?ip=10.1.2.3&username=user' -i 'otp={{ (ds "secret").key }}'
...
$ gomplate -d 'secret=file:///run/secrets/secret.json' -i 'otp={{ (ds "secret").key }}'
...

It's important to me that datasources be more-or-less interchangeable in general. I have situations where I use the same template in one environment where Vault is used to store secrets, and another environment where we store them as Docker Swarm secrets - in which case they show up as files inside a container.

That being said, the latter use-case doesn't make too much sense for dynamic secrets because generally they're populated at deploy-time. In the near future, Swarm secrets will support pluggable back-ends (including Vault), so using gomplate to populate dynamic secrets will be less of a thing in that scenario. 🤔

There's no reason we couldn't simply support both methods of providing the additional arguments though, so why don't we start with what's in this PR and then I could add URL query-based args later on if and when the need arises. 🙂

[hairyhenderson]

One thing I would ask, though, is that instead of multiple args can we use a single arg, and specify multiple args as a comma-separated list?

Like this:

{{ datasource "vault" "ssh/creds/test" "ip=10.1.2.3,username=user" }}

Does this make sense?

[stuart-c]

Sounds good. I'll update the PR shortly

[hairyhenderson]

@stuart-c another way to approach this would be to only support URL query parameters, but allow them to be provided in the path to datasource:

{{ datasource "vault" "ssh/creds/test?ip=10.1.2.3&username=user" }}

This way there'd be no need to special-case based on number of args, and it'd allow specifying params both inside and outside the template.

IMO it's a little cleaner this way (at least, implementation-wise).

[stuart-c]

How's this @hairyhenderson?

[hairyhenderson]

Thanks - I'm good with this! 🎉