Argo CD Istio deploy compatibility

[senare]

Summary

We are trying to make use of Argo CD and are deploying it together with Istio (service mesh).

However there are a few rough edges with using the provided manifest and deploying it without modifications, so I wonder if they couldn't be address 'upstreams' ?!

This is spec here => https://istio.io/docs/ops/deployment/requirements/

Also Included here for easy ref

Named service ports: Service ports must be named. The port name key/value pairs must have the following syntax: name: [-]. See Protocol Selection for more details.

Deployments with app and version labels: We recommend adding an explicit app label and version label to deployments. Add the labels to the deployment specification of pods deployed using the Kubernetes Deployment. The app and version labels add contextual information to the metrics and telemetry Istio collects.

The app label: Each deployment specification should have a distinct app label with a meaningful value. The app label is used to add contextual information in distributed tracing.

The version label: This label indicates the version of the application corresponding to the particular deployment.
Istio have some required conventions about the port naming i.e need to be on the form -suffix.

I would appreciate some feedback on how to handle this !

(Currently I work around it with copy&paste the manifest and do some minor edits, but its a bit hacky)

[alexec]

Would you like to raise a PR on the manifests to add these?

[senare]

I will make an attempt :)

But might not be the fastest, I need to figure out how to build and test etc ...

[martivo]

In my case enabling istio on argocd broke argo. If someone else ends up on this ticket because argocd won't work with istio. The reason is the same requirements - specifically service port naming. For example when using "--insecure" with argo-cd then you can not use "https" as the port name of argocd-server service, since istio will assume it is https. As a quick workaround I prefixed all the service port names in argocd with "tcp-" and now it is working.

The related errors in argocd were "tls: first record does not look like a TLS handshake".
I am not reporting it as a bug, since when this feature is done then the problem will go away by itself.

[vigohe]

My workaround was to enable "--insecure" at argocd-server and disable "TLS"

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-server-vts
  namespace: argocd
spec:
  gateways:
  - default/global-gateway
  hosts:
  - argocd.example.com
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

DestinationRule.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: argocd-server-dtrl
  namespace: argocd
spec:
  host: argocd-server.argocd.svc.cluster.local
  trafficPolicy:
    tls:
      mode: DISABLE

[nemo83]

I wrote about how to bootstrap a k8s cluster with ArgoCD 1.4.2 and Istio 1.5.1 using istio Operator in this blog post: https://nemo83.dev/posts/argocd-istio-operator-bootstrap/
Inside you'll find also link to github repo containing all the charts mentioned in the article as well the config updates to get ArgoCD served via Istio ingress gateway, the only thing missing is https, but if you know istio you should be able to add this missing bit yourself. I will blog about this later.

Please pay attention to:

1. this kustomization https://github.com/speedwing/eks-argocd-bootstrap/tree/master/argocd-bootstrap/argocd-istio-bootstrap
2. This chart where Gateway and Virtual services are configured: https://github.com/speedwing/eks-argocd-bootstrap/tree/master/applications/argocd-istio-app

Hope it helps

[L-U-C-K-Y]

Thanks for the workarounds!

I tried using SSL Passthrough of istio, but without success.
I'm not sure if I could pass-through multiple protocols over istio (gRPC/HTTPS) on the same port.

In my case enabling istio on argocd broke argo. If someone else ends up on this ticket because argocd won't work with istio. The reason is the same requirements - specifically service port naming. For example when using "--insecure" with argo-cd then you can not use "https" as the port name of argocd-server service, since istio will assume it is https. As a quick workaround I prefixed all the service port names in argocd with "tcp-" and now it is working.

The related errors in argocd were "tls: first record does not look like a TLS handshake".
I am not reporting it as a bug, since when this feature is done then the problem will go away by itself.

I would highly appreciate if this could get fixed by the argo team.

[Cajga]

Hi Guys,

I've created a kustomize setup to make the ArgoCD manifests fully compatible with Istio requirements (link in the description of the ticket). With this these are the steps to install ArgoCD on an Istio enabled k8s cluster with Istio TLS passthrough:

• wget -O argocd_install.yaml https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
• create the three files below (kustomizations.yaml, istio_patches.yaml and argocd-gateway.yaml)
• adjust the deployment versions in the istio_patches.yaml based on the image versions in the argocd_install.yaml
• kubectl create namespace argocd
• kubectl label namespace argocd istio-injection=enabled
• kubectl apply -k argocd_install.yaml
• lubectl apply -f argocd-gateway.yaml
• create a hosts file entry with argocd-dev.example.com and your external IP

The kustomize does the following changes:

• enables the following options on arg-server --repo-server argocd-repo-server:8081 --insecure
• adds app and version labels for Deployments
• fixes port names on Services (basically introduces http- prefix where it is needed)
• deletes port 443 from argocd-server service (we will use mTLS from istio anyway)
• puts every resource into the argocd namespace

kustomizations.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: argocd
resources:
  - argocd_install.yaml
patchesStrategicMerge:
  - istio_patches.yaml

istio_patches.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: server
    app.kubernetes.io/name: argocd-server
    app.kubernetes.io/part-of: argocd
    app: argocd-server
    version: v1.6.1
  name: argocd-server
spec:
  template:
    metadata:
      labels:
        app: argocd-server
        version: v1.6.1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: repo-server
    app.kubernetes.io/name: argocd-repo-server
    app.kubernetes.io/part-of: argocd
    app: argocd-repo-server
    version: v1.6.1
  name: argocd-repo-server
spec:
  template:
    metadata:
      labels:
        app: argocd-repo-server
        version: v1.6.1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: redis
    app.kubernetes.io/name: argocd-redis
    app.kubernetes.io/part-of: argocd
    app: argocd-redis
    version: v5.0.3
  name: argocd-redis
spec:
  template: 
    metadata:
      labels:
        app: argocd-redis
        version: v5.0.3
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: dex-server
    app.kubernetes.io/name: argocd-dex-server
    app.kubernetes.io/part-of: argocd
    app: argocd-dex-server
    version: v2.22.0
  name: argocd-dex-server
spec:
  template:
    metadata:
      labels:
        app: argocd-dex-server
        version: v2.22.0
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: application-controller
    app.kubernetes.io/name: argocd-application-controller
    app.kubernetes.io/part-of: argocd
    app: argocd-application-controller
    version: v1.6.1
  name: argocd-application-controller
spec:
  template:
    metadata:
      labels:
        app: argocd-application-controller
        version: v1.6.1
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-dex-server
spec:
  ports:
  - name: http
    port: 5556
    protocol: TCP
    targetPort: 5556
  - name: http-grpc
    port: 5557
    protocol: TCP
    targetPort: 5557
  - name: http-metrics
    port: 5558
    protocol: TCP
    targetPort: 5558

---
apiVersion: v1
kind: Service
metadata:
  name: argocd-metrics
spec:
  ports:
  - name: http-metrics
    port: 8082
    protocol: TCP
    targetPort: 8082
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-repo-server
spec:
  ports:
  - name: https-server
    port: 8081
    protocol: TCP
    targetPort: 8081
  - name: http-metrics
    port: 8084
    protocol: TCP
    targetPort: 8084
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-server-metrics
spec:
  ports:
  - name: http-metrics
    port: 8083
    protocol: TCP
    targetPort: 8083
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-server
spec:
  ports:
  - port: 80
    $patch: delete
  - name: https-argocd-server
    port: 443
    protocol: TCP
    targetPort: 8080

argocd-gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: argocd-gateway
  namespace: argocd
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    tls:
      httpsRedirect: true
    hosts:
    - "argocd-dev.example.com"
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: PASSTHROUGH
    hosts:
    - "argocd-dev.example.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd
  namespace: argocd
spec:
  hosts:
  - "argocd-dev.example.com"
  gateways:
  - argocd-gateway
  tls:
  - match:
    - port: 443
      sniHosts:
      - argocd-dev.example.com
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 443

[sp185503]

@Cajga Thanks for posting the fix. Side note, I had to name the port with http-, typical Istio stuff.

---
apiVersion: v1
kind: Service
metadata:
  name: argocd-server
spec:
  ports:
  - name: http-argocd-server
    port: 80
    protocol: TCP
    targetPort: 8080
  - port: 443
    $patch: delete

[Cajga]

@sp185503 well, I do rename it when it was necessary... Did I miss one?
According to the specs it is <protocol>[-<suffix>].
My understanding is that the -<suffix> part is not mandatory (it is not there in some of their examples).

[sp185503]

Honestly I've had so many issues where the name of the port was just http and Istio wouldn't forward traffic correctly. If you search for articles on this you'll find to fix this problem you need to add a suffix...I would be interested to hear other's input on this. Currently I'm running 1.5.x in my environments.

[Cajga]

@sp185503 I just asked if http is enough or we need the -suffix (I also mentioned that you had issues without) on istio slack and got the following answer from @howardjohn (Istio maintainer):

I have never heard of this issue, the code treats them the same (in theory at least)

Maybe you could elaborate a bit what did you see there and we can find out what was the problem.

For me, the kustomize above (after I applied my fixes) works well.

[sp185503]

In my case enabling istio on argocd broke argo. If someone else ends up on this ticket because argocd won't work with istio. The reason is the same requirements - specifically service port naming. For example when using "--insecure" with argo-cd then you can not use "https" as the port name of argocd-server service, since istio will assume it is https. As a quick workaround I prefixed all the service port names in argocd with "tcp-" and now it is working.

The related errors in argocd were "tls: first record does not look like a TLS handshake".
I am not reporting it as a bug, since when this feature is done then the problem will go away by itself.

After following @Cajga kustomize solution I was able to get Argocd installed with Istio however we were getting the following errors when trying to create a new application in Arogocd. Using thie tcp- prefix for all services ended up being the final solution. Here is a sample:

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: server
    app.kubernetes.io/name: argocd-server
    app.kubernetes.io/part-of: argocd
  name: argocd-server
  namespace: argocd
spec:
  ports:
  - name: tcp-argocdserver
    port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app.kubernetes.io/name: argocd-server
  type: ClusterIP

Previous error message:

Unable to create application: application spec is invalid: InvalidSpecError: Unable to get app details: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: authentication handshake failed: tls: first record does not look like a TLS handshake"

[Cajga]

@sp185503 I got that error when the agro-repo-server service had http instead of https in its port name. In the latest kustomize (see above the updated version) I fixed that:

apiVersion: v1
kind: Service
metadata:
  name: argocd-repo-server
spec:
  ports:
  - name: https-server
    port: 8081
    protocol: TCP
    targetPort: 8081
  - name: http-metrics
    port: 8084
    protocol: TCP
    targetPort: 8084

Please note that using tcp prefix will limit the possibilities that istio provides you.
I am working on a PR for get this into argocd manifests. Should be ready in few days.

[sp185503]

@Cajga I'll try that in a few minutes.

[Cajga]

Could someone please review the PR above?

[flickerfly]

Thanks for your work on this Cajga!

[PsychoSid]

This is pretty cool thanks. I took the install.yaml from @Cajga PR above and deployed on my test GKE clustee which uses Istio Ingress GW which normally does the TLS termination and has the certificates there. As this is PASSTHROUGH where is the best place to add a valid cert as going to https:// reports a bad cert ?

Cheers

[Cajga]

@PsychoSid I will create a PR for Istio ingress config options.
According to this page the certs are stored in this secret.

As a side note: you can easily make it work with Istio gateway termination (and together with mTLS STRICT mode you won't need https on the argocd-server). To make that work apply the following kustomization on the install.yaml:

# we enable insecure mode (accept http) for argocd as we use mTLS in Istio and TLS in TLS does not make too much sense
apiVersion: apps/v1
kind: Deployment
metadata:
  name: argocd-server
spec:
  template:
    spec:
      containers:
        - name: argocd-server
          command:
            - argocd-server
            - --staticassets
            - /shared/app
            - --repo-server
            - argocd-repo-server:8081
            - --insecure
---
# here we rename port 80 to get proper http metrics and delete the unused 443 port (we only use http as TLS is handled by Istio)
apiVersion: v1
kind: Service
metadata:
  name: argocd-server
spec:
  ports:
  - name: http-argocdserver
    port: 80
    protocol: TCP
    targetPort: 8080
  - port: 443
    $patch: delete

And use the following gateway and virtualservice config (obviously adjust the gateway to your needs):

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: argocd-gateway
  namespace: argocd
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    tls:
      httpsRedirect: true
    hosts:
    - "argocd-playground.example.com"
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: argocd-credential
    hosts:
    - "argocd-playground.example.com"
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: argocd
  namespace: argocd
spec:
  hosts:
  - "argocd-playground.example.com"
  gateways:
  - argocd-gateway
  http:
  - route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

[PsychoSid]

@Cajga above config's works like a charm with my certs handed by cert-manager/letsencypt. Many thanks.

Although the CLI now throws an error:-

"FATA[0000] rpc error: code = Internal desc = transport: received the unexpected content-type "text/plain; charset=utf-8"

If using plaintext get another error

$ argocd --insecure --plaintext login argocd.example.com 
Username: admin
Password:
FATA[0003] rpc error: code = Unavailable desc = transport is closing

And another edit but this may help others:)

It works if using

$ argocd --grpc-web login argocd.example.com
Username: admin
Password:
'admin' logged in successfully

So all seems well. Thanks again

[raonelakurti]

My workaround was to enable "--insecure" at argocd-server and disable "TLS"

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-server-vts
  namespace: argocd
spec:
  gateways:
  - default/global-gateway
  hosts:
  - argocd.example.com
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

DestinationRule.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: argocd-server-dtrl
  namespace: argocd
spec:
  host: argocd-server.argocd.svc.cluster.local
  trafficPolicy:
    tls:
      mode: DISABLE

I've tried accessing argocd through istio-ingress gateway.
For some reason, I'm not able access ArgoCD through UI when I have a prefix in the Virtual service. Without prefix I'm able to access it.

Gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
 name: argocd-gateway-template
 namespace: argocd
spec:
 selector:
   istio: ingressgateway
 servers:
 - hosts:
   - '*'
   port:
     name: http
     number: 80
     protocol: HTTP

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

[epacke]

I used a modified version of Cajgas great work above and wrote a guide on setting this up:
https://loadbalancing.se/2021/03/22/argocd-behind-istio-on-rancher/

Manifests can be found here:
https://github.com/epacke/argo-istio

Hopefully it helps someone else!

[vigohe]

My workaround was to enable "--insecure" at argocd-server and disable "TLS"
VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-server-vts
  namespace: argocd
spec:
  gateways:
  - default/global-gateway
  hosts:
  - argocd.example.com
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

DestinationRule.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: argocd-server-dtrl
  namespace: argocd
spec:
  host: argocd-server.argocd.svc.cluster.local
  trafficPolicy:
    tls:
      mode: DISABLE

I've tried accessing argocd through istio-ingress gateway.
For some reason, I'm not able access ArgoCD through UI when I have a prefix in the Virtual service. Without prefix I'm able to access it.

Gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
 name: argocd-gateway-template
 namespace: argocd
spec:
 selector:
   istio: ingressgateway
 servers:
 - hosts:
   - '*'
   port:
     name: http
     number: 80
     protocol: HTTP

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

@raonelakurti For prefix to work, you need to rewrite your match, because argocd is serving at root /

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    rewrite:
      uri: "/"
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

[raonelakurti]

My workaround was to enable "--insecure" at argocd-server and disable "TLS"
VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-server-vts
  namespace: argocd
spec:
  gateways:
  - default/global-gateway
  hosts:
  - argocd.example.com
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

DestinationRule.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: argocd-server-dtrl
  namespace: argocd
spec:
  host: argocd-server.argocd.svc.cluster.local
  trafficPolicy:
    tls:
      mode: DISABLE

I've tried accessing argocd through istio-ingress gateway.
For some reason, I'm not able access ArgoCD through UI when I have a prefix in the Virtual service. Without prefix I'm able to access it.

Gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
 name: argocd-gateway-template
 namespace: argocd
spec:
 selector:
   istio: ingressgateway
 servers:
 - hosts:
   - '*'
   port:
     name: http
     number: 80
     protocol: HTTP

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

@raonelakurti For prefix to work, you need to rewrite your match, because argocd is serving at root /

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    rewrite:
      uri: "/"
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

vigohe
I've installed argocd in --insecure mode. Argocd Deployment container configuration.

containers:
  - command:
  - argocd-server
  - --staticassets
  - /shared/app
  - --repo-server
  - argocd-repo-server:8081
  - --insecure

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    rewrite:
       uri: "/"
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

When I did the curl to the ingressgateway with suffix, I'm not able to see the argocd-ui login page,instead I
got the following response. When I accessed it through the browser, argocd login page is not loading.

<title>Argo CD</title>

<script type="text/javascript" src="main.d714f4605e74ef974014.js"></script>

[vigohe]

My workaround was to enable "--insecure" at argocd-server and disable "TLS"
VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-server-vts
  namespace: argocd
spec:
  gateways:
  - default/global-gateway
  hosts:
  - argocd.example.com
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

DestinationRule.yaml

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: argocd-server-dtrl
  namespace: argocd
spec:
  host: argocd-server.argocd.svc.cluster.local
  trafficPolicy:
    tls:
      mode: DISABLE

I've tried accessing argocd through istio-ingress gateway.
For some reason, I'm not able access ArgoCD through UI when I have a prefix in the Virtual service. Without prefix I'm able to access it.

Gateway.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
 name: argocd-gateway-template
 namespace: argocd
spec:
 selector:
   istio: ingressgateway
 servers:
 - hosts:
   - '*'
   port:
     name: http
     number: 80
     protocol: HTTP

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

@raonelakurti For prefix to work, you need to rewrite your match, because argocd is serving at root /

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    rewrite:
      uri: "/"
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

vigohe
I've installed argocd in --insecure mode. Argocd Deployment container configuration.

containers:
  - command:
  - argocd-server
  - --staticassets
  - /shared/app
  - --repo-server
  - argocd-repo-server:8081
  - --insecure

VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: argocd-vs
  namespace: argocd
spec:
  gateways:
  - argocd-gateway-template.argocd
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /argocd/
    rewrite:
       uri: "/"
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

When I did the curl to the ingressgateway with suffix, I'm not able to see the argocd-ui login page,instead I
got the following response. When I accessed it through the browser, argocd login page is not loading.

<title>Argo CD</title> <script type="text/javascript" src="main.d714f4605e74ef974014.js"></script>

@raonelakurti

I thought It would work like that... 😢

A better approach would be to to set the --rootpath and delete the rewrite suggestion.

spec:
  template:
    spec:
      name: argocd-server
      containers:
      - command:
        - /argocd-server
        - --staticassets
        - /shared/app
        - --repo-server
        - argocd-repo-server:8081
        - --rootpath
        - /argocd
        - --insecure

reference:
https://argoproj.github.io/argo-cd/operator-manual/ingress/#argocd-server-and-ui-root-path-v153

[raonelakurti]

@vigohe Thanks for your response. It worked after adding the root path. I really appreciate your time in answering my question.
One other question on RBAC with Argo CD. One of my teams were working on argocd project and incase if I have to enable RBAC with argo rollouts role? Is it possible? If yes which CRD I have to add in Kubernetes role?

[trathailoi]

Exellent work @Cajga !! Thank you so much !
Just a note here for others who are doing the same thing, I did change a little to make it work:

1. in the istio_patches.yaml file. From kind: Deployment to kind: StatefulSet for the name: argocd-application-controller resource thing.
2. change the file name from kustomizations.yaml to kustomization.yaml.

And then, it works perfectly.
Thank you again!

[raj1592]

Hello There,
I am facing issues when using "ArgoCD Server and UI Root Path(--rootpath)" provided with "*" hosts , I get 401 for all the below api calls:
/argocd/api/version
/argocd/api/v1/settings
/argocd/api/v1/session/userinfo
At the end of the login page I get this error
"Failed to load data, please try again."

below args are used for the server,
extraArgs:

• --rootpath
• /argocd
• --insecure

Below is the virtualservice manifest file

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: vs-argocd
spec:
hosts:

• "*"
  gateways:
• gateway
  http:
  • match:
    • uri:
      prefix: /argocd
      route:
      • destination:
        host: argocd-server.argocd.svc.cluster.local

Contrarily, it works if I pass in a hostname in hosts like below

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: vs-argocd-host
spec:
hosts:

• "argocd.example.com"
  gateways:
• gateway
  http:
  • match:
    • uri:
      prefix: /argocd
      route:
      • destination:
        host: argocd-server.argocd.svc.cluster.local

Could someone please help with this issue.

[leepereira]

error: got file 'argocd_install.yaml', but 'argocd_install.yaml' must be a directory to be a root

I am getting this error when i am trying to kubectl apply -k argocd_install.yaml

[flickerfly]

@leepereira You're probably putting it under the kustomizations: section instead of resources section.

[leepereira]

. ├── argocd-gateway.yaml ├── argocd_install.yaml ├── istio_patches.yaml └── kustomization.yaml

#cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: argocd
resources:
  - argocd_install.yaml
patchesStrategicMerge:
  - istio_patches.yaml

This is what I have on kustomization yaml,
@flickerfly : Are you suggesting that I create a kustomizations: section on this file.

I tried that and still getting the same error

Thanks for the help

[repakaanjan]

I wrote about how to bootstrap a k8s cluster with ArgoCD 1.4.2 and Istio 1.5.1 using istio Operator in this blog post: https://nemo83.dev/posts/argocd-istio-operator-bootstrap/ Inside you'll find also link to github repo containing all the charts mentioned in the article as well the config updates to get ArgoCD served via Istio ingress gateway, the only thing missing is https, but if you know istio you should be able to add this missing bit yourself. I will blog about this later.

Please pay attention to:

1. this kustomization https://github.com/speedwing/eks-argocd-bootstrap/tree/master/argocd-bootstrap/argocd-istio-bootstrap
2. This chart where Gateway and Virtual services are configured: https://github.com/speedwing/eks-argocd-bootstrap/tree/master/applications/argocd-istio-app

Hope it helps

@nemo83

I am trying to access the url:
https://nemo83.dev/posts/argocd-istio-operator-bootstrap/ and it's not working. can you please check or help me with some other URL.

[cippaciong]

@repakaanjan https://web.archive.org/web/20201126205749/https://nemo83.dev/posts/argocd-istio-operator-bootstrap/

[karlschriek]

@Cajga above config's works like a charm with my certs handed by cert-manager/letsencypt. Many thanks.

Although the CLI now throws an error:-

"FATA[0000] rpc error: code = Internal desc = transport: received the unexpected content-type "text/plain; charset=utf-8"

If using plaintext get another error

$ argocd --insecure --plaintext login argocd.example.com 
Username: admin
Password:
FATA[0003] rpc error: code = Unavailable desc = transport is closing

And another edit but this may help others:)

It works if using

$ argocd --grpc-web login argocd.example.com
Username: admin
Password:
'admin' logged in successfully

So all seems well. Thanks again

On what version of ArgoCD is this?

On v2.4.0, for me $ argocd --grpc-web login argocd.example.com produces the following error:

FATA[0006] rpc error: code = Unknown desc = POST https://argocd.mydomain.com:443/session.SessionService/Create failed with status code 403.

See also #10371

[Hronom]

Hello, is there solution?

How everyone use ArgoCD with Istio in prod? Since preferably for Kubrenetes cluster is to have one Ingress Controller it's better to adopt ArgoCD in order to be able to run with sidecar containers and so on...

[pre]

@Hronom argocd works out-of-the-box with Istio with the following changes:

• use --insecure for argocd-server and terminate TLS in Istio
• rename the metrics ports to tcp-metrics or Prometheus can't scrape with STRICT mTLS PeerAuthentication

[Hronom]

@pre it's not work out-of-the-box if you need to make changes :=)
You described not all changes that is needed to make it work correctly without errors and warnings.

Check this description from comments in this thread #2784 (comment)

So currently I can't use helm as is and I need to apply next kustomziation patch:

apiVersion: v1
kind: Service
metadata:
  name: argocd-applicationset-controller
spec:
  ports:
    - name: http-webhook
      port: 7000
      protocol: TCP
      targetPort: webhook
    - name: http-metrics
      port: 8080
      protocol: TCP
      targetPort: metrics
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-dex-server
spec:
  ports:
    - name: http
      port: 5556
      protocol: TCP
      targetPort: 5556
    - name: http-grpc
      port: 5557
      protocol: TCP
      targetPort: 5557
    - name: http-metrics
      port: 5558
      protocol: TCP
      targetPort: 5558
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-metrics
spec:
  ports:
    - name: http-metrics
      port: 8082
      protocol: TCP
      targetPort: 8082
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-notifications-controller-metrics
spec:
  ports:
    - name: http-metrics
      port: 9001
      protocol: TCP
      targetPort: 9001
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-repo-server
spec:
  ports:
    - name: https-server
      port: 8081
      protocol: TCP
      targetPort: 8081
    - name: http-metrics
      port: 8084
      protocol: TCP
      targetPort: 8084
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-server
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 8080
    - name: https
      $patch: delete
---
apiVersion: v1
kind: Service
metadata:
  name: argocd-server-metrics
spec:
  ports:
    - name: http-metrics
      port: 8083
      protocol: TCP
      targetPort: 8083

So if we properly set prefixes for ports - it will helps to switch to the helm without maintaining and applying kustomization patch with fixes.

There was started PR #3893 but so needed changes not reached final state...

It will be cool if fix at least with ports will be done.

And BTW this issues already 4 years... Istio pretty popular services mesh nowadays.

[pre]

@Hronom also the dex service port needs to be renamed from http to either https or tcp

#6183 (comment)

[Hronom]

@pre thank you for pushing this! Very appreciated!

Also adding here for the reference PR that was created long time ago but not merged #3893

[sailormurph]

I got this working via the helm chart, which I have locally:

#!/usr/bin/env bash
set -e
kubectl delete crd applications.argoproj.io || true
kubectl delete crd applicationsets.argoproj.io || true
kubectl delete crd appprojects.argoproj.io || true
kubectl delete clusterrole argocd-application-controller || true
kubectl delete clusterrole argocd-server || true
kubectl delete clusterrolebinding argocd-application-controller || true
kubectl delete clusterrolebinding argocd-server || true
kubectl delete namespace argocd || true
kubectl create namespace argocd
kubectl label namespace argocd istio-injection=enabled
helm install argocd ./argo-cd -n argocd --wait --timeout 10m \
  --create-namespace --debug \
  --set configs.params."server\.insecure"=true 
kubectl apply -f argocd-istio-virtualservice.yaml

argocd-istio-virtualservice.yaml:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: argocd
  namespace: argocd
spec:
  hosts:
  - "argocd.whatever.lol"
#Gateway handles TLS termination
  gateways:
  - istio-ingress/ingress-gateway1
  http:
  - name: argocd
    route:
    - destination:
        host: argocd-server.argocd.svc.cluster.local
        port:
          number: 80

[pre]

@sailormurph sure - that works until you enable STRICT mTLS. Then all ports which don’t have the protocol stop working, such as oauth callback of dex (should behttp-webhook) and metrics endpoints (should be http-metrics) and the https port of argo-server (needs to be removed).

[CodyLeeWhite]

I have been fighting this for the better part of a week. I have it mostly working except for

when trying to add a new repo
cwhite@CWhite-mac scripts % argocd repo add [email protected]:project/repo.git --ssh-private-key-path ~/.ssh/id_rsa --grpc-web FATA[0000] rpc error: code = Unimplemented desc =

im using the helm chart to install

configs:
  cm:
    dex.config: |
      connectors:
      - config:
          issuer: https://accounts.google.com
          clientID: xxxxxxx.apps.googleusercontent.com
          clientSecret: xxxxxxxxxx
        type: oidc
        id: google
        name: Google
    url: https://my.domain
  params:
    server.insecure: true
controller:
  extraArgs:
    - "--repo-server-plaintext"
    - "--repo-server=argocd-repo-server:15090"
repoServer:
  extraArgs:
    - "--disable-tls"
server:
  service:
    type: ClusterIP
  podAnnotations:
    sidecar.istio.io/rewriteAppHTTPProbers: "true"
  extraArgs:
    - "--repo-server-plaintext"
    - "--dex-server-plaintext"
    - "--repo-server=argocd-repo-server:15090"
    - "--dex-server=argocd-dex-server:15090"
dex:
  enabled: true
  extraArgs:
    - "--disable-tls"

[yevon]

Hi any news in this one? I'm not being able to make it as it seems that argocd tries to always redirect to https. Is it possible to disable argocd https -> http redirects in any way?

[epacke]

--insecure flag will do the trick. Read here:

https://argo-cd.readthedocs.io/en/stable/operator-manual/ingress/

[yevon]

--insecure flag will do the trick. Read here:

https://argo-cd.readthedocs.io/en/stable/operator-manual/ingress/

I tried it but it was ignoring me because the name of the parameter in the helm chart contains a dot, so i had to escape it as it wasn't applying the change in the correct place in the configmap configmap argocd-cmd-params-cm:

    helm:
      parameters:
      - name: "configs.params.server\\.insecure"
        value: "true"

I came across this issue and it resolved it:

argoproj/argo-helm#1780

Now it works with istio without major problems!