Basis for new inline injection feature

CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog for Vault Sidecar Injector
 
+## Release v5.1.1 - 2019-12-23
+
+**Added**
+
+- [VSI #18](https://github.com/Talend/vault-sidecar-injector/pull/18) - Basis for new inline injection feature
+
+**Fixed**
+
+- [VSI #16](https://github.com/Talend/vault-sidecar-injector/issues/16) - secrets-template with >1 templates that include range statement causes dest/template mismatch [Thanks @smurfralf]
+- [VSI #15](https://github.com/Talend/vault-sidecar-injector/issues/15) - Document requirement for configured certificates api [Thanks @drpebcak]
+
 ## Release v5.1.0 - 2019-12-09
 
 - [VSI #14](https://github.com/Talend/vault-sidecar-injector/pull/14) - Minor updates to Helm chart and documentation.

Makefile

@@ -1,5 +1,5 @@
-RELEASE_VERSION:=5.1.0
-VSI_VERSION:=5.0.0
+RELEASE_VERSION:=5.1.1
+VSI_VERSION:=5.0.1
 
 OWNER:=Talend
 REPO:=vault-sidecar-injector

README.md

@@ -95,7 +95,7 @@ Following annotations in requesting pods are supported:
 | `sidecar.vault.talend.org/secrets-destination` | O     | secrets   | "secrets.properties" | Comma-separated strings  | List of secrets filenames (without path), one per secrets path |
 | `sidecar.vault.talend.org/secrets-hook`        | O     | secrets   | | "true" / "on" / "yes" / "y" | If set, lifecycle hooks will be added to pod's container(s) to wait for secrets files |
 | `sidecar.vault.talend.org/secrets-path`        | O     | secrets   | "secret/<`com.talend.application` label>/<`com.talend.service` label>" | Comma-separated strings | List of secrets engines and path. If annotation not used, path is set from labels defined by `mutatingwebhook.annotations.appLabelKey`  and `mutatingwebhook.annotations.appServiceLabelKey` keys (refer to [configuration](https://github.com/Talend/vault-sidecar-injector/blob/master/README.md#configuration))      |
-| `sidecar.vault.talend.org/secrets-template`    | O     | secrets   | [Default template](https://github.com/Talend/vault-sidecar-injector/blob/master/README.md#default-template) | Comma-separated templates | Allow to override default template. Ignore `sidecar.vault.talend.org/secrets-path` annotation if set |
+| `sidecar.vault.talend.org/secrets-template`    | O     | secrets   | [Default template](https://github.com/Talend/vault-sidecar-injector/blob/master/README.md#default-template) | templates separated with `---` | Allow to override default template. Ignore `sidecar.vault.talend.org/secrets-path` annotation if set |
 | `sidecar.vault.talend.org/workload`   | O      | N/A               |  | "job" | Type of submitted workload |
 
 Upon successful injection, Vault Sidecar Injector will add annotation(s) to the requesting pods:
@@ -465,8 +465,8 @@ Several optional annotations to end up with:
 
 - Vault authentication using role `test-app-2` (value of `com.talend.application` label)
 - hook injected in application's container(s) to wait for secrets file availability
-- secrets fetched from Vault's path `secret/test-app-2/test-app-2-svc` using **several custom templates**
-- secrets to be stored into `/opt/talend/secrets/secrets.properties` and `/opt/talend/secrets/secrets2.properties`
+- secrets fetched from Vault's path `secret/test-app-2/test-app-2-svc` using **several custom templates** (use `---` as separation between them)
+- secrets to be stored into `/opt/talend/secrets/secrets.properties` (using first template) and `/opt/talend/secrets/secrets2.properties` (using second template)
 
 ```yaml
 apiVersion: apps/v1
@@ -490,7 +490,8 @@ spec:
           {{ if .Data.SECRET1 }}
           my_custom_key_name1={{ .Data.SECRET1 }}
           {{ end }}
-          {{ end }},
+          {{ end }}
+          ---
           {{ with secret "secret/test-app-2/test-app-2-svc" }}
           {{ if .Data.SECRET2 }}
           my_custom_key_name2={{ .Data.SECRET2 }}
@@ -827,7 +828,7 @@ The following table lists the configurable parameters of the `Vault Sidecar Inje
 | image.port       | Service main port    | 8443            |
 | image.pullPolicy   | Pull policy for docker image: IfNotPresent or Always       | IfNotPresent           |
 | image.serviceNameLabel   | Service Name. Must match label com.talend.service     | talend-vault-sidecar-injector      |
-| image.tag  | Version/tag of the docker image     | 5.0.0      |
+| image.tag  | Version/tag of the docker image     | 5.0.1      |
 | imageRegistry  | Image registry |   |
 | injectconfig.jobbabysitter.image.path   | Docker image path | everpeace/curl-jq |
 | injectconfig.jobbabysitter.image.pullPolicy | Pull policy for docker image: IfNotPresent or Always | IfNotPresent |
@@ -838,7 +839,7 @@ The following table lists the configurable parameters of the `Vault Sidecar Inje
 | injectconfig.jobbabysitter.resources.requests.memory | Job babysitter sidecar memory resource requests | 20Mi |
 | injectconfig.vault.image.path  | Docker image path  | vault |
 | injectconfig.vault.image.pullPolicy    | Pull policy for docker image: IfNotPresent or Always  | IfNotPresent   |
-| injectconfig.vault.image.tag  | Version/tag of the docker image | 1.3.0 |
+| injectconfig.vault.image.tag  | Version/tag of the docker image | 1.3.1 |
 | injectconfig.vault.loglevel                    | Vault log level: trace, debug, info, warn, err    | info    |
 | injectconfig.vault.resources.limits.cpu | Vault sidecar CPU resource limits | 50m |
 | injectconfig.vault.resources.limits.memory | Vault sidecar memory resource limits | 50Mi |

deploy/helm/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 name: vault-sidecar-injector
 description: A Helm chart for Talend Vault Sidecar Injector (OSS)
-version: 3.1.0
+version: 3.1.1
 icon: https://www.talend.com/wp-content/uploads/talend-logo.svg
 keywords:
    - Talend
@@ -14,4 +14,4 @@ sources:
 maintainers:
    - name: Talend
      email: [email protected]
-appVersion: 5.0.0
+appVersion: 5.0.1

deploy/helm/values.yaml

@@ -15,7 +15,7 @@ revisionHistoryLimit: 3  # revision history limit in tiller / helm / k8s
 
 image:
   path: "talend/vault-sidecar-injector"  # Docker image path
-  tag: "5.0.0"  # Version/tag of the docker image
+  tag: "5.0.1"  # Version/tag of the docker image
   pullPolicy: IfNotPresent  # Pull policy for docker images: IfNotPresent or Always
   port: 8443  # service main port exposed by the docker image
   metricsPort: 9000  # metricsPort defines the port exposed by the docker image for metrics collection
@@ -66,7 +66,7 @@ injectconfig:
   vault:
     image:
       path: "vault" # Docker image path
-      tag: "1.3.0" # Version/tag of the docker image
+      tag: "1.3.1" # Version/tag of the docker image
       pullPolicy: IfNotPresent # Pull policy for docker images: IfNotPresent or Always
     loglevel: info # Vault log level: trace, debug, info, warn, err
     resources:

deploy/samples/test-app2-deployment.yaml

@@ -19,7 +19,8 @@ spec:
           {{ if .Data.SECRET1 }}
           bob={{ .Data.SECRET1 }}
           {{ end }}
-          {{ end }},
+          {{ end }}
+          ---
           {{ with secret "secret/test2/test-app2-svc" }}
           {{ if .Data.SECRET2 }}
           alice={{ .Data.SECRET2 }}

pkg/config/load_test.go

@@ -51,18 +51,18 @@ func TestLoadConfig(t *testing.T) {
 	}{
 		{
 			inputLoaded{
-				"../../test/sidecarconfig.yaml",
-				"../../test/proxyconfig.hcl",
-				"../../test/tmplblock.hcl",
-				"../../test/tmpldefault.tmpl",
-				"../../test/podlifecyclehooks.yaml",
+				"../../test/config/sidecarconfig.yaml",
+				"../../test/config/proxyconfig.hcl",
+				"../../test/config/tmplblock.hcl",
+				"../../test/config/tmpldefault.tmpl",
+				"../../test/config/podlifecyclehooks.yaml",
 			},
 			expectedLoad{
-				"../../test/sidecarconfig.yaml.resolved",
+				"../../test/config/sidecarconfig.yaml.resolved",
 				proxyCfgFileResolved,
 				templateBlockResolved,
 				templateDefaultResolved,
-				"../../test/podlifecyclehooks.yaml.resolved",
+				"../../test/config/podlifecyclehooks.yaml.resolved",
 			},
 		},
 	}

pkg/webhook/constants.go

@@ -54,12 +54,11 @@ const (
 	templateTemplatesPlaceholder         = "<APPSVC_TEMPLATES>"
 	appSvcSecretsVolMountPathPlaceholder = "<APPSVC_SECRETS_VOL_MOUNTPATH>"
 
-	vaultK8sAuthMethod                = "kubernetes"         // Default auth method used by Vault Agent
-	appSvcSecretsVolName              = "secrets"            // Name of the volume shared between containers to store secrets file(s)
-	templateAppSvcDefaultDestination  = "secrets.properties" // Default secrets destination
-	k8sDefaultServiceAccountTokenPath = "/var/run/secrets/kubernetes.io/serviceaccount/token"
-	vaultDefaultSecretsEnginePath     = "secret" // Default path for Vault K/V Secrets Engine if no 'secrets-path' annotation
-	vaultProxyDefaultPort             = "8200"   // Default port to access local Vault proxy
+	vaultK8sAuthMethod               = "kubernetes"         // Default auth method used by Vault Agent
+	appSvcSecretsVolName             = "secrets"            // Name of the volume shared between containers to store secrets file(s)
+	templateAppSvcDefaultDestination = "secrets.properties" // Default secrets destination
+	vaultDefaultSecretsEnginePath    = "secret"             // Default path for Vault K/V Secrets Engine if no 'secrets-path' annotation
+	vaultProxyDefaultPort            = "8200"               // Default port to access local Vault proxy
 
 	//--- Job handling - Temporary mechanism until KEP https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/sidecarcontainers.md is implemented (and we migrate on appropriate version of k8s)
 	jobMonitoringContainerName     = "tvsi-job-babysitter" // Name of our specific sidecar container to inject in submitted jobs

pkg/webhook/inline_test.go

@@ -0,0 +1,181 @@
+// Copyright © 2019 Talend
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package webhook
+
+import (
+	"encoding/json"
+	"errors"
+	"io/ioutil"
+	"path/filepath"
+	"strings"
+	"talend/vault-sidecar-injector/pkg/config"
+	"testing"
+
+	"k8s.io/api/admission/v1beta1"
+	appsv1 "k8s.io/api/apps/v1"
+	authenticationv1 "k8s.io/api/authentication/v1"
+	batchv1 "k8s.io/api/batch/v1"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/klog"
+
+	"github.com/ghodss/yaml"
+	"github.com/stretchr/testify/assert"
+)
+
+type testResource struct {
+	manifest        string
+	workloadType    string
+	podTemplateSpec *corev1.PodTemplateSpec
+}
+
+func TestInlineInjector(t *testing.T) {
+	injectionCfg, err := config.Load(
+		config.WhSvrParameters{
+			0, 0, "", "",
+			"sidecar.vault.talend.org", "com.talend.application", "com.talend.service",
+			"../../test/config/sidecarconfig.yaml",
+			"../../test/config/proxyconfig.hcl",
+			"../../test/config/tmplblock.hcl",
+			"../../test/config/tmpldefault.tmpl",
+			"../../test/config/podlifecyclehooks.yaml",
+		},
+	)
+	if err != nil {
+		t.Errorf("Loading error \"%s\"", err)
+	}
+
+	// Create webhook instance
+	vaultInjector := New(injectionCfg, nil)
+
+	// Get all test workloads
+	workloads, err := filepath.Glob("../../test/workloads/*.yaml")
+	if err != nil {
+		t.Errorf("Fail listing files: %s", err)
+	}
+
+	// Loop on all test workloads: mutate and display JSON Patch structure
+	for _, workloadManifest := range workloads {
+		klog.Infof("Loading workload %s", workloadManifest)
+
+		ar, err := (&testResource{
+			manifest: workloadManifest,
+			workloadType: func(w string) string {
+				if strings.HasSuffix(w, "-deployment.yaml") {
+					return "deployment"
+				} else if strings.HasSuffix(w, "-job.yaml") {
+					return "job"
+				} else {
+					return ""
+				}
+			}(workloadManifest),
+		}).load()
+		if err != nil {
+			t.Errorf("Error creating AR \"%s\"", err)
+		}
+
+		// Mutate pod
+		resp := vaultInjector.mutate(ar)
+
+		assert.Equal(t, true, resp.Allowed)
+		assert.Nil(t, resp.Result)
+		assert.NotNil(t, resp.Patch)
+
+		var patch []patchOperation
+		if err = yaml.Unmarshal(resp.Patch, &patch); err != nil {
+			t.Errorf("JSON Patch unmarshal error \"%s\"", err)
+		}
+
+		klog.Infof("JSON Patch=%+v", patch)
+	}
+}
+
+func (tr *testResource) load() (*v1beta1.AdmissionReview, error) {
+	// TODO: Beware, there may be several resources. Only keep and mutate the ones with Vault Sidecar Injector's `sidecar.vault.talend.org/inject: "true"` annotation
+	data, err := ioutil.ReadFile(tr.manifest)
+	if err != nil {
+		return nil, err
+	}
+
+	if tr.workloadType == "deployment" {
+		resource := appsv1.Deployment{}
+		_, _, err = deserializer.Decode(data, nil, &resource)
+		if err != nil {
+			return nil, err
+		}
+
+		tr.podTemplateSpec = &resource.Spec.Template
+	} else if tr.workloadType == "job" {
+		resource := batchv1.Job{}
+		_, _, err = deserializer.Decode(data, nil, &resource)
+		if err != nil {
+			return nil, err
+		}
+
+		tr.podTemplateSpec = &resource.Spec.Template
+	} else {
+		return nil, errors.New("Worload not supported")
+	}
+
+	tr.addSATokenVolume()
+	return tr.createAdmissionReview()
+}
+
+func (tr *testResource) addSATokenVolume() {
+	// We expect to find serviceaccount token volume. It is dynamically added to the pod by the Service Account Admission Controller.
+	// Add it manually here to pass internal check.
+	saTokenVolumeMount := corev1.VolumeMount{
+		Name:      "default-token-1234",
+		ReadOnly:  true,
+		MountPath: k8sDefaultSATokenVolMountPath,
+	}
+
+	saTokenVolume := corev1.Volume{
+		Name: "default-token-1234",
+		VolumeSource: corev1.VolumeSource{
+			Secret: &corev1.SecretVolumeSource{
+				SecretName: "default-token",
+			},
+		},
+	}
+
+	tr.podTemplateSpec.Spec.Containers[0].VolumeMounts = append(tr.podTemplateSpec.Spec.Containers[0].VolumeMounts, saTokenVolumeMount)
+	tr.podTemplateSpec.Spec.Volumes = append(tr.podTemplateSpec.Spec.Volumes, saTokenVolume)
+}
+
+func (tr *testResource) createAdmissionReview() (*v1beta1.AdmissionReview, error) {
+	rawPod, err := json.Marshal(tr.podTemplateSpec)
+	if err != nil {
+		return nil, err
+	}
+
+	return &v1beta1.AdmissionReview{
+		Request: &v1beta1.AdmissionRequest{
+			Kind: metav1.GroupVersionKind{
+				Version: "v1",
+				Kind:    "Pod",
+			},
+			Namespace: tr.podTemplateSpec.GetNamespace(),
+			Operation: v1beta1.Create,
+			UserInfo: authenticationv1.UserInfo{
+				Username: "vault-sidecar-injector",
+			},
+			Object: runtime.RawExtension{
+				Raw: rawPod,
+			},
+		},
+	}, nil
+}

pkg/webhook/secrets.go

@@ -25,7 +25,7 @@ import (
 // Vault Sidecar Injector: Secrets Mode
 func (vaultInjector *VaultInjector) secretsMode(labels, annotations map[string]string) (string, error) {
 	secretsPath := strings.Split(annotations[vaultInjector.VaultInjectorAnnotationsFQ[vaultInjectorAnnotationSecretsPathKey]], ",")
-	secretsTemplate := strings.Split(annotations[vaultInjector.VaultInjectorAnnotationsFQ[vaultInjectorAnnotationSecretsTemplateKey]], ",")
+	secretsTemplate := strings.Split(annotations[vaultInjector.VaultInjectorAnnotationsFQ[vaultInjectorAnnotationSecretsTemplateKey]], "---")
 	templateDest := strings.Split(annotations[vaultInjector.VaultInjectorAnnotationsFQ[vaultInjectorAnnotationTemplateDestKey]], ",")
 	templateCmd := strings.Split(annotations[vaultInjector.VaultInjectorAnnotationsFQ[vaultInjectorAnnotationTemplateCmdKey]], ",")