Adding Service annotations breaks workload

[wobes]

We are attempting to use GCE (Google Cloud Ingress controller on GKE) with the pingDevops helm chart. With GCE Ingress you need to add annotations to the service block to add in backend configurations for the GCE ingress controller. The issue is the way the yaml schema for the services and/or the way the template for the workload was written. It treats the annotations map as another service definition and thus the workload fails to deploy.

Here is an example.

Deploying the helm chart without annotations deployed on GKE works as expected.

service: ping-devops-pingfederate-admin.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    meta.helm.sh/release-name: ping-devops
    meta.helm.sh/release-namespace: ping-devops-review-9b2a503e

statefulset: ping-devops-pingfederate-admin.yaml

ports block

   name: pingfederate-admin
        ports:
        - containerPort: 7600
          name: clusterbind
          protocol: TCP
        - containerPort: 7700
          name: clusterfail
          protocol: TCP
        - containerPort: 9999
          name: https
          protocol: TCP

Now lets add annotations to the services by adding the required configurations to the values.yaml file.

values.yaml with annotations added to the service on the pingfederate-admin block as documented.

global:
  image:
    tag: "2405-11.3.7"
  ingress:
    enabled: true
    defaultDomain: shared-services.example.com
    defaultTlsSecret: poccert
    annotations:
      kubernetes.io/ingress.class: gce
      kubernetes.io/ingress.allow-http: "false"
pingfederate-admin:
  enabled: true
  workload:
    type: StatefulSet
  services:
    annotations: 
      cloud.google.com/app-protocols: '{"https":"HTTPS"}'
      cloud.google.com/backend-config: '{"ports": {"9999":"ping-fed-admin-backend-config"}}'

The workload (in this case a statefulset) fails as it is trying to add the annotation block as another port definition. Here is the output of the port section of the rendered yaml file for the statefulset ping-devops-pingfederate-admin.yaml

      name: pingfederate-admin
        ports:
        - containerPort: 0
          name: annotations
          protocol: TCP
        - containerPort: 7600
          name: clusterbind
          protocol: TCP
        - containerPort: 7700
          name: clusterfail
          protocol: TCP
        - containerPort: 9999
          name: https

As you can see it added name: annotations to the map of ports.

The good news is it did successfully add the annotation to the service.

apiVersion: v1
kind: Service
metadata:
  annotations:
    cloud.google.com/app-protocols: '{"https":"HTTPS"}'
    cloud.google.com/backend-config: '{"ports": {"9999":"ping-fed-admin-backend-config"}}'
    meta.helm.sh/release-name: ping-devops
    meta.helm.sh/release-namespace: ping-devops-review-9b2a503e

One fix (and probably the best approach) would be to modify the v.services schema like described in this issue: #159

however as a quick fix we forked this repo and added a conditional to the ports block that checks if the @serviceName != "annotations" on the charts/ping-devops/templates/pinglib/_workload.tpl file on line 144 {{- if ne $serviceName "annotations" }} .

       ports:
        {{- range $serviceName, $val := . }}
        {{- if ne $serviceName "annotations" }}
        {{- if kindIs "map" $val }}
        - containerPort: {{ $val.containerPort }}
          name: {{ $serviceName }}
        {{- end }}
        {{- end }}
        {{- end }}
        {{- end }}

After making this change, the statefulset deploys successfully and the GCE ingress works as expected. Note: The same problem will heppen for adding any of these maps to the services block:

  #  annotations: {}
  #  clusterServiceAnnotations: {}
  #  labels: {}
  #  clusterServiceLabels: {}

as the template is just looking for if kindIs "map".

See service Documentation here: https://github.com/pingidentity/helm-charts/blob/master/charts/ping-devops/values.yaml#L401-L421