Isolation of deployed "secrets" between independent zones

[ghost]

Hi!
As in agreement with @lippserd below is the write up of our proposal for the isolated deployment of secrets to separate zones.

---

Let's assume we have a distributed monitoring environment with four satellites.

Furthermore, always two of these satellites monitor the infrastructure of two different, completely independent, real world entities, and as such shall be isolated from each other:

• EntityA is monitored by SatA1(Zone A_Int) and SatA2(Zone A_Cloud)
• EntityB is monitored by SatB1(Zone B_RZ_DE) and SatB2(Zone B_RZ_FI)

When monitoring services which need credentials, an example could be the SNMP Community, the correct mechanism to distribute the secrets to their respective icinga2 instances, seems to be assignment via Hosttemplates.

Current Behavior

When using the director-global zone for these Hosttemplates, they are distributed among the entire monitoring environment, thus information is leaked between the Entities.

Also when assigning a zone to the Hosttemplates, they get deployed only to that specific single zone.

This requires that for having them on each zone of an Entity, they must be duplicated N times for each zone of the Entity.
In our example it's not so bad as N=2, but in reality N can be dozens of zones and satellites.

Expected Behavior

A way to deploy "secret" custom variables to a limited subset of zones, without the need of duplicated configuration for each zone.

Possible Solutions

Proposal: On demand Hosttemplate deployment

Implicitly deploy a host template to a zone, always when it is directly or transitively associated with a Host belonging to this Zone.

Hosttemplates:

• generic-host in director-global zone
• entityA-secrets containing customvar snmp_community=SecretA
• entityA-host-secret imports generic-host && entityA-secret

Hosts:

• host.A_int.example.com imports entityA-host-secret and assigned to Zone A_Int.
• host.B_RZ_DE.example.com imports generic-host and assigned to Zone B_RZ_DE.

What should happen in this case, is that independently of the zone of the last two Hosttemplates:

• the deployment to Zone A_Int should include generic-host via director-global, but also entityA-secrets and entityA-host-secred by transitivity, as Host host.A_int.example.com is assigned to this Zone and inherits from these Hosttemplates.
• the deployment to Zone B_RZ_DE should include only the generic-host Template via director-global Zone.

There are obviously some details left to refine:

• When does a Template get deployed to director-global zone, and when on-demand deployed?
  Maybe a new zone-dropdown entry "on-demand"?
• How to leave a trace (e.g. for debugging) that a template got deployed via "on-demand" to a certain zone?
  Could a comment in the generated file referring to the mechanism be enough?

Evaluated but discarded: Hosttemplate Zone Inheritance

Explicitly deploy some host-templates to a "sub-tree" of the monitoring infrastructure.
This is by far the inferior solution, as it requires a specific setup of the monitoring infrastructure.

The basic idea is, that specific templates, are assigned to a Zone, but will be deployed recursively, to every child-zone in the distributed monitoring environment.
Example Zone Diagram:

Master -> EntityA(additional) -> A_Int
       |                      -> A_Cloud
       |
       -> EntityB(additional) -> B_RZ_DE
                              -> B_RZ_FI

If we now define a Hosttemplate containing secrets, we could assign it to e.g. EntityA zone, but flag it (it's not yet defined how) as to be "inherited" to also by A_Int and A_Cloud, where Hosts of these zones can import it.

As you can see, to take the advantage of this feature, we would have to introduce a new Zone for each Entity, which has the sole purpose of existing for modelling an entry-point to the sub-tree.

Additionaly there is a bigger mental overhead for admins, to track which templates are deployed where, with the danger of failed deployments, because of missing imported Hosttemplates.

The above solution, "just works", in the sense that whenever a template is directly or indirectly imported by a Host, it is automatically deployed to where it's needed by the director.

[lippserd]

Because template names are unique in Icinga, the Director cannot simply provide a template multiple times. However, this could be resolved using one of the following:

• Wrap templates that are not tied to a specific zone with if (! get_template(Host, "Name")) { template Host "Name" { ... } } and deploy them in the actual agent zones.
• Introduce something new that resolves and renders the template directly to the host definition.

[Thomas-Gelf]

Merged.