Back to basics: Use all-in-one image instead of complicated deployment

Instead of enforcing complex deployment strategies such as having a dedicated endpoints image, we instead allow for an image with all the configuration backed-in. This allows for a simpler deployment as we would no longer require complex initContainers nor specialized images. This thus removes the usage of the endpoints copier and even the endpoints image option. Signed-off-by: Juan Antonio Osorio <[email protected]>
equinixmetal-helm · Jan 31, 2023 · 8d136f0 · 8d136f0
1 parent e6b087e
commit 8d136f0
Show file tree

Hide file tree

Showing 17 changed files with 338 additions and 407 deletions.
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
@@ -16,8 +16,8 @@ jobs:
         config:
           - name: 'Scenario: with default options'
             testdir: 'default-data'
-          - name: 'Scenario: with endpoints image'
-            testdir: 'with-endpoints-image'
+          - name: 'Scenario: with all-in-one image'
+            testdir: 'with-all-in-one-image'
           - name: 'Scenario: with extra mounts'
             testdir: 'with-extra-mounts'
           - name: 'Scenario: with templated annotations'
@@ -50,12 +50,12 @@ jobs:
           kubectl create namespace $KRAKEND_NS
 
       - name: 'Build image if needed'
-        if: matrix.config.testdir == 'with-endpoints-image'
+        if: matrix.config.testdir == 'with-all-in-one-image'
         run: |
           docker build -t krakend-test:latest -f tests/${{ matrix.config.testdir }}/Dockerfile tests/${{ matrix.config.testdir }}
 
       - name: 'Load image if needed'
-        if: matrix.config.testdir == 'with-endpoints-image'
+        if: matrix.config.testdir == 'with-all-in-one-image'
         run: |
           kind load docker-image --name chart-testing krakend-test:latest
 
@@ -86,9 +86,9 @@ jobs:
       - name: Test that response is correct
         run: |
           curl -s http://localhost:8081/test | jq -e '.version == "deficient"'
-        if: matrix.config.testdir != 'with-endpoints-image'
+        if: matrix.config.testdir != 'with-all-in-one-image'
 
       - name: Test that response is correct
         run: |
           curl -s http://localhost:8081/directories/1 | jq -e '.version == "flavorless"'
-        if: matrix.config.testdir == 'with-endpoints-image'
+        if: matrix.config.testdir == 'with-all-in-one-image'
diff --git a/README.md b/README.md
@@ -12,6 +12,20 @@ A Helm chart for Kubernetes
 
 ## Usage
 
+There are two main modes of operation:
+
+* configMap-based: the configuration is stored in a configMaps and mounted in the
+  container.
+
+* all-in-one image: the configuration is stored in the container image itself.
+
+Note that the all-in-one image method assumes that the configuration is stored
+in the `/etc/krakend-src/config` path. This is done to allow for mounting
+the license as describe in the section below.
+
+Both modes of operation enable the flexible configuration feature. For the all-in-one
+image, this is done so that the configuration can take environment variables into use.
+
 Note that for a further description on how to use partials, settings and templates,
 please refer to [the official krakend documentation](https://www.krakend.io/docs/configuration/flexible-config/).
 
@@ -34,27 +48,11 @@ please refer to [the official krakend documentation](https://www.krakend.io/docs
 | ingress.className | string | `""` | The class to use for the ingress |
 | ingress.enabled | bool | `false` | Specifies whether an ingress should be created |
 | ingress.hosts | list | `[{"host":"chart-example.local","paths":[{"path":"/","pathType":"ImplementationSpecific"}]}]` | The list of hosts to use for the ingress |
+| krakend.allInOneImage | bool | `false` | Whether the given krakend image to be used contains everything needed for the krakend service to run. If set to false, the image will be used as a base image and the configuration will be loaded from the configmap. Note that the all-in-one image assumes the krakend configuration is located at /etc/krakend-src/config/krakend.tmpl |
 | krakend.config | string, optional | `""` | If set, this key will contain the full configuration of the krakend service |
-| krakend.endpoints | object | `{"endpointsConfig":"[\n  {\n    \"endpoint\": \"/test\",\n    \"method\": \"GET\",\n    \"backend\": [\n        {\n            \"method\": \"GET\",\n            \"host\": [],\n            \"url_pattern\": \"/__debug/roots\",\n            \"encoding\": \"json\",\n            \"deny\": [\n                \"message\"\n            ]\n        }\n    ],\n    \"extra_config\": {\n        \"proxy\": {\n            \"static\": {\n                \"data\": {\n                    \"collection\": [\n                        {\n                            \"directories\": [\n                                \"Graceland\"\n                            ]\n                        }\n                    ],\n                    \"version\": \"deficient\"\n                },\n                \"strategy\": \"always\"\n            }\n        }\n    },\n    \"output_encoding\": \"json\"\n  }\n]","fromImage":false,"image":{"args":["-c","cp /endpoints.json /endpoints/endpoints.tmpl"],"command":["/bin/sh"],"pullPolicy":"IfNotPresent","registry":null,"repository":null,"resources":{"limits":{"cpu":"100m","memory":"128Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"tag":null}}` | Given that endpoitns is where most of the verbosity of the configuration is, it's possible to load them from a configmap or from an image. This section allows you to configure the relevant settings. |
-| krakend.endpoints.endpointsConfig | string | `"[\n  {\n    \"endpoint\": \"/test\",\n    \"method\": \"GET\",\n    \"backend\": [\n        {\n            \"method\": \"GET\",\n            \"host\": [],\n            \"url_pattern\": \"/__debug/roots\",\n            \"encoding\": \"json\",\n            \"deny\": [\n                \"message\"\n            ]\n        }\n    ],\n    \"extra_config\": {\n        \"proxy\": {\n            \"static\": {\n                \"data\": {\n                    \"collection\": [\n                        {\n                            \"directories\": [\n                                \"Graceland\"\n                            ]\n                        }\n                    ],\n                    \"version\": \"deficient\"\n                },\n                \"strategy\": \"always\"\n            }\n        }\n    },\n    \"output_encoding\": \"json\"\n  }\n]"` | If `fromImage` is set to false, the endpoints will be loaded from the partials configmap with this configuration |
-| krakend.endpoints.fromImage | bool | `false` | Use this flag to indicate that the endpoints should be loaded from the image instead of the configmap. This happens as an initContainer. |
-| krakend.endpoints.image | object | `{"args":["-c","cp /endpoints.json /endpoints/endpoints.tmpl"],"command":["/bin/sh"],"pullPolicy":"IfNotPresent","registry":null,"repository":null,"resources":{"limits":{"cpu":"100m","memory":"128Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"tag":null}` | This is the image to use to load the endpoints from. Note that the registry, repository and tag must be set. |
-| krakend.endpoints.image.args | list | `["-c","cp /endpoints.json /endpoints/endpoints.tmpl"]` | The arguments to use to load the endpoints from the image. |
-| krakend.endpoints.image.command | list | `["/bin/sh"]` | The command to use to load the endpoints from the image. |
-| krakend.endpoints.image.pullPolicy | string | `"IfNotPresent"` | The image pull policy to use for the endpoints loader |
-| krakend.endpoints.image.registry | string | `nil` | The image registry to use for the endpoints loader |
-| krakend.endpoints.image.repository | string | `nil` | The image repository to use for the endpoints loader Note that the image must contain a file named endpoints.json at the root of the image. |
-| krakend.endpoints.image.resources | object | `{"limits":{"cpu":"100m","memory":"128Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}` | The resources to use for the endpoints loader |
-| krakend.endpoints.image.tag | string | `nil` | The image tag to use for the endpoints loader |
-| krakend.env | array | `[{"name":"FC_ENABLE","value":"1"},{"name":"FC_SETTINGS","value":"/etc/krakend-src/settings"},{"name":"FC_PARTIALS","value":"/etc/krakend-src/partials"},{"name":"FC_TEMPLATES","value":"/etc/krakend-src/templates"}]` | The environment variables to use for the krakend container. The default is just the ones needed to enable flexible configuration. |
+| krakend.env | array | `[]` | The environment variables to use for the krakend container. The default is just the ones needed to enable flexible configuration. |
 | krakend.extraConfig | object | `{"router":{"@comment":"The health endpoint checks do not show in the logs","logger_skip_paths":["/__health"]}}` | The service-level "extra_config" settings. This will directly be translated to JSON and is added only if you're not fully overriding the configuration via the `config` setting. For more information, see https://www.krakend.io/docs/enterprise/configuration/ |
-| krakend.partials | Object | `{"input_headers.tmpl":"\"input_headers\": [\n  \"Content-Type\",\n  \"ClientId\"\n]","rate_limit_backend.tmpl":"\"qos/ratelimit/proxy\": {\n  \"max_rate\": 0.5,\n  \"capacity\": 1\n}"}` | The default configuration has a partials files that will be used to load several aspects of the configuration. If you want to include expra partials, add or remove them here. |
-| krakend.partialsCopierImage | object | `{"pullPolicy":"IfNotPresent","registry":"docker.io","repository":"library/alpine","resources":{"limits":{"cpu":"100m","memory":"128Mi"},"requests":{"cpu":"100m","memory":"128Mi"}},"tag":"3.17.1"}` | The default configuration has a partials file that will be used to load several aspects of the configuration. This is done through an initContainer that copies the partials to the /etc/krakend/partials folder. |
-| krakend.partialsCopierImage.pullPolicy | string | `"IfNotPresent"` | The image pull policy to use for the partials copier |
-| krakend.partialsCopierImage.registry | string | `"docker.io"` | The image registry to use for the partials copier |
-| krakend.partialsCopierImage.repository | string | `"library/alpine"` | The image repository to use for the partials copier |
-| krakend.partialsCopierImage.resources | object | `{"limits":{"cpu":"100m","memory":"128Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}` | The resources to use for the partials copier |
-| krakend.partialsCopierImage.tag | string | `"3.17.1"` | The image tag to use for the partials copier |
+| krakend.partials | Object | `{"endpoints.tmpl":"[\n  {\n    \"endpoint\": \"/test\",\n    \"method\": \"GET\",\n    \"backend\": [\n        {\n            \"method\": \"GET\",\n            \"host\": [],\n            \"url_pattern\": \"/__debug/roots\",\n            \"encoding\": \"json\",\n            \"deny\": [\n                \"message\"\n            ]\n        }\n    ],\n    \"extra_config\": {\n        \"proxy\": {\n            \"static\": {\n                \"data\": {\n                    \"collection\": [\n                        {\n                            \"directories\": [\n                                \"Graceland\"\n                            ]\n                        }\n                    ],\n                    \"version\": \"deficient\"\n                },\n                \"strategy\": \"always\"\n            }\n        }\n    },\n    \"output_encoding\": \"json\"\n  }\n]","input_headers.tmpl":"\"input_headers\": [\n  \"Content-Type\",\n  \"ClientId\"\n]","rate_limit_backend.tmpl":"\"qos/ratelimit/proxy\": {\n  \"max_rate\": 0.5,\n  \"capacity\": 1\n}"}` | The default configuration has a partials files that will be used to load several aspects of the configuration. If you want to include expra partials, add or remove them here. |
 | krakend.settings | object | `{"service.json":"{\n\t\"environment\": \"PRODUCTION\",\n\t\"default_host\": \"http://localhost:8080\",\n\t\"timeout\": \"3s\",\n\t\"cache_ttl\": \"3s\",\n\t\"output_encoding\": \"json\"\n}"}` | The default configuration has a settings files that will be used to load several aspects of the configuration. |
 | krakend.templates | object | `{}` | While default configuration does not take into use templates; you may want to add your own templates here. Note that you'd need to set a custom configuration file to use them. |
 | nameOverride | string | `""` |  |

diff --git a/README.md.gotmpl b/README.md.gotmpl
@@ -12,6 +12,20 @@ This is a helm chart that deploys a [Krakend](https://www.krakend.io/) instance.
 
 ## Usage
 
+There are two main modes of operation:
+
+* configMap-based: the configuration is stored in a configMaps and mounted in the
+  container.
+
+* all-in-one image: the configuration is stored in the container image itself.
+
+Note that the all-in-one image method assumes that the configuration is stored
+in the `/etc/krakend-src/config` path. This is done to allow for mounting
+the license as describe in the section below.
+
+Both modes of operation enable the flexible configuration feature. For the all-in-one
+image, this is done so that the configuration can take environment variables into use.
+
 Note that for a further description on how to use partials, settings and templates,
 please refer to [the official krakend documentation](https://www.krakend.io/docs/configuration/flexible-config/).
 

diff --git a/templates/_helpers.tpl b/templates/_helpers.tpl
@@ -68,7 +68,6 @@ Get krakend files directory
 {{- printf "/etc/krakend-src" -}}
 {{- end }}
 
-
 {{/*
 Get krakend config file directory
 */}}
@@ -80,7 +79,7 @@ Get krakend config file directory
 Get krakend config file name
 */}}
 {{- define "krakend.configFileName" -}}
-{{- printf "%s.tmpl" (include "krakend.fullname" .) -}}
+{{- printf "krakend.tmpl" -}}
 {{- end }}
 
 {{/*

diff --git a/templates/cm-config.yaml b/templates/cm-config.yaml
@@ -1,4 +1,4 @@
----
+{{- if not .Values.krakend.allInOneImage }}
 # This ConfigMap contains the initial configuration
 # file for the Krakend.io API Gateway.
 kind: ConfigMap
@@ -25,4 +25,5 @@ data:
         "endpoints": {{ include "endpoints.tmpl" }},
         "extra_config": `}}{{ .Values.krakend.extraConfig | toJson }}{{`
     }`}}
+{{- end }}
 {{- end }}
diff --git a/templates/cm-partials.yaml b/templates/cm-partials.yaml
@@ -1,3 +1,4 @@
+{{- if not .Values.krakend.allInOneImage }}
 kind: ConfigMap
 apiVersion: v1
 metadata:
@@ -7,7 +8,4 @@ data:
   {{ $key }}: |-
     {{ $value | nindent 4 }}
   {{- end }}
-  {{- if not .Values.krakend.endpoints.fromImage }}
-  endpoints.tmpl: |-
-    {{ .Values.krakend.endpoints.endpointsConfig | nindent 4 }}
   {{- end }}
diff --git a/templates/cm-settings.yaml b/templates/cm-settings.yaml
@@ -1,8 +1,10 @@
+{{- if not .Values.krakend.allInOneImage }}
 kind: ConfigMap
 apiVersion: v1
 metadata:
   name: {{ include "krakend.fullname" . }}-settings
 data:
   {{- range $key, $value := .Values.krakend.settings }}
   {{ $key }}: {{ $value | quote }}
-  {{- end }}
+  {{- end }}
+{{- end }}
diff --git a/templates/cm-templates.yaml b/templates/cm-templates.yaml
@@ -1,3 +1,4 @@
+{{- if not .Values.krakend.allInOneImage }}
 kind: ConfigMap
 apiVersion: v1
 metadata:
@@ -6,3 +7,4 @@ data:
   {{- range $key, $value := .Values.krakend.templates }}
   {{ $key }}: {{ $value | quote }}
   {{- end }}
+{{- end }}
diff --git a/templates/deployment.yaml b/templates/deployment.yaml
@@ -26,10 +26,12 @@ spec:
   template:
     metadata:
       annotations:
+        {{- if .Values.krakend.allInOneImage }}
         checksum/cm-config: {{ include (print $.Template.BasePath "/cm-config.yaml") . | sha256sum }}
         checksum/cm-partials: {{ include (print $.Template.BasePath "/cm-partials.yaml") . | sha256sum }}
         checksum/cm-settings: {{ include (print $.Template.BasePath "/cm-settings.yaml") . | sha256sum }}
         checksum/cm-templates: {{ include (print $.Template.BasePath "/cm-templates.yaml") . | sha256sum }}
+        {{- end }}
         {{- with .Values.podAnnotations }}
         {{- toYaml . | nindent 8 }}
         {{- end }}
@@ -43,55 +45,6 @@ spec:
       serviceAccountName: {{ include "krakend.serviceAccountName" . }}
       securityContext:
         {{- toYaml .Values.podSecurityContext | nindent 8 }}
-      initContainers:
-        - name: partials-copier
-          image: "{{ .Values.krakend.partialsCopierImage.registry }}/{{ .Values.krakend.partialsCopierImage.repository }}:{{ .Values.krakend.partialsCopierImage.tag | default .Chart.AppVersion }}"
-          imagePullPolicy: {{ .Values.krakend.partialsCopierImage.pullPolicy }}
-          securityContext:
-            allowPrivilegeEscalation: false
-            runAsNonRoot: true
-            runAsUser: 1000
-            runAsGroup: 1000
-            readOnlyRootFilesystem: true
-            capabilities:
-              drop:
-                - ALL
-          resources:
-            {{- toYaml .Values.krakend.partialsCopierImage.resources | nindent 12 }}
-          command:
-            - /bin/sh
-          args:
-            - "-c"
-            - "cp /partials/* /partials-but-really/"
-          volumeMounts:
-            - name: partials-but-really
-              mountPath: /partials-but-really
-            - name: partials
-              mountPath: /partials
-      {{- if .Values.krakend.endpoints.fromImage }}
-        - name: endpoints-copier
-          image: "{{ .Values.krakend.endpoints.image.registry }}{{ if .Values.krakend.endpoints.image.registry }}/{{ end }}{{ required "krakend.endpoints.image.repository is required" .Values.krakend.endpoints.image.repository }}:{{ required "krakend.endpoints.image.tag is required" .Values.krakend.endpoints.image.tag }}"
-          imagePullPolicy: {{ .Values.krakend.endpoints.image.pullPolicy }}
-          securityContext:
-            allowPrivilegeEscalation: false
-            readOnlyRootFilesystem: true
-            runAsNonRoot: true
-            runAsUser: 1000
-            runAsGroup: 1000
-            capabilities:
-              drop:
-                - ALL
-          resources:
-            {{- toYaml .Values.krakend.endpoints.image.resources | nindent 12 }}
-          command: {{ .Values.krakend.endpoints.image.command }}
-          {{- with .Values.krakend.endpoints.image.args }}
-          args:
-            {{- toYaml . | nindent 12 }}
-          {{- end }}
-          volumeMounts:
-            - name: partials-but-really
-              mountPath: /endpoints
-      {{- end }}
       containers:
         - name: {{ .Chart.Name }}
           securityContext:
@@ -108,6 +61,16 @@ spec:
           env:
             - name: KRAKEND_PORT
               value: {{ quote .Values.service.targetPort }}
+            - name: FC_ENABLE
+              value: '1'
+            {{- if not .Values.krakend.allInOneImage }}
+            - name: FC_SETTINGS
+              value: {{ include "krakend.settingsDir" . | quote }}
+            - name: FC_PARTIALS
+              value: {{ include "krakend.partialsDir" . | quote }}
+            - name: FC_TEMPLATES
+              value: {{ include "krakend.templatesDir" . | quote }}
+            {{- end }}
             {{- with .Values.krakend.env }}
             {{- toYaml . | nindent 12 }}
             {{- end }}
@@ -126,14 +89,16 @@ spec:
           volumeMounts:
             - name: tmp
               mountPath: /tmp
+            {{- if not .Values.krakend.allInOneImage }}
             - name: config
               mountPath: {{ include "krakend.configFileDir" . }}
-            - name: partials-but-really
+            - name: partials
               mountPath: {{ include "krakend.partialsDir" . }}
             - name: settings
               mountPath: {{ include "krakend.settingsDir" . }}
             - name: templates
               mountPath: {{ include "krakend.templatesDir" . }}
+            {{- end }}
             {{- with .Values.extraVolumeMounts }}
             {{- toYaml . | nindent 12 }}
             {{- end }}
@@ -154,6 +119,7 @@ spec:
       volumes:
         - name: tmp
           emptyDir: {}
+        {{- if not .Values.krakend.allInOneImage }}
         - name: config
           configMap:
             name: {{ include "krakend.fullname" . }}-config
@@ -166,8 +132,7 @@ spec:
         - name: templates
           configMap:
             name: {{ include "krakend.fullname" . }}-templates
-        - name: partials-but-really
-          emptyDir: {}
+        {{- end }}
         {{- with .Values.extraVolumes }}
         {{- toYaml . | nindent 8 }}
         {{- end }}
diff --git a/tests/with-all-in-one-image/Dockerfile b/tests/with-all-in-one-image/Dockerfile
@@ -0,0 +1,3 @@
+FROM docker.io/devopsfaith/krakend:2.1.4
+
+COPY krakend.tmpl /etc/krakend-src/config/krakend.tmpl
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		FROM docker.io/devopsfaith/krakend:2.1.4

		COPY krakend.tmpl /etc/krakend-src/config/krakend.tmpl