Port-forward timeout

[bakayolo]

When doing the port-forward to the argo-cd server, I am getting a lot of timeout and I have to restart the port-forward. It s very annoying since I am expecting my UI to be accessible all time through this port-forward.

Is this a known issue? any workaround (except exposing the endpoint)?

[jessesuen]

I don't think this is an argo-cd issue. Do you see errors in kubectl port-forward? Maybe this will help:

https://stackoverflow.com/questions/47484312/kubectl-port-forwarding-timeout-issue

[bakayolo]

Actually, it s only happening with argo-cd.

kubectl port-forward -n argocd svc/argocd-server 8081:80
Forwarding from 127.0.0.1:8081 -> 8080
Forwarding from [::1]:8081 -> 8080
Handling connection for 8081
Handling connection for 8081
E1205 14:32:13.727126   21412 portforward.go:303] error copying from remote stream to local connection: readfrom tcp6 [::1]:8081->[::1]:58157: write tcp6 [::1]:8081->[::1]:58157: write: broken pipe
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
E1205 14:33:07.313769   21412 portforward.go:303] error copying from remote stream to local connection: readfrom tcp6 [::1]:8081->[::1]:58190: write tcp6 [::1]:8081->[::1]:58190: write: broken pipe
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
Handling connection for 8081
E1205 14:35:20.650357   21412 portforward.go:303] error copying from remote stream to local connection: readfrom tcp6 [::1]:8081->[::1]:58513: write tcp6 [::1]:8081->[::1]:58513: write: broken pipe
E1205 14:36:42.490481   21412 portforward.go:303] error copying from remote stream to local connection: readfrom tcp6 [::1]:8081->[::1]:58273: write tcp6 [::1]:8081->[::1]:58273: write: broken pipe

[kuznero]

I also observe this behavior. But to be fare this happened to other services too.

[bakayolo]

I am getting this timeout much more than any other services actually.
Mostly when updating/syncing/creating an application.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mnaser]

I am running into this quite often... and it's only specifically with Argo's UI

[ecurtin2]

This commonly happens to me at work behind corporate firewalls, etc but I haven't noticed it if I use argo ui on local machine (minikube / docker-for-mac k8s)

[philippeboyd]

Happens to me too when running with Minikube

Forwarding from 127.0.0.1:9999 -> 8080
Forwarding from [::1]:9999 -> 8080
Handling connection for 9999
Handling connection for 9999
Handling connection for 9999
Handling connection for 9999
Handling connection for 9999
Handling connection for 9999
E0220 10:44:49.462605    3899 portforward.go:372] error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:9999->127.0.0.1:54136: write tcp4 127.0.0.1:9999->127.0.0.1:54136: write: broken pipe

[melalj]

Same here. The port forwarding keeps timing out.

[L-U-C-K-Y]

Having the same issue with the newest argocd version:

Handling connection for 8080
E0903 15:36:48.756701   11386 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
Handling connection for 8080
E0903 15:37:18.759391   11386 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
Handling connection for 8080
E0903 15:37:48.766846   11386 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
E0903 15:40:42.781019   11386 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
E0903 15:40:46.358790   11386 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
E0903 15:40:46.359085   11386 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
Handling connection for 8080
E0903 15:41:16.362030   11386 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080

[cfarrend]

Experiencing the same, thought it was an issue with resources but after increasing them I still get time outs. Usually happens after the pod has been up for a few hours, the fix has just been to kill the pod.

[kheraud]

Same experience here

Handling connection for 8080
Handling connection for 8080
E0204 10:11:09.966242   15947 portforward.go:372] error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:8080->127.0.0.1:51138: write tcp4 127.0.0.1:8080->127.0.0.1:51138: write: broken pipe
Handling connection for 8080
E0204 10:11:10.662920   15947 portforward.go:372] error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:8080->127.0.0.1:51140: write tcp4 127.0.0.1:8080->127.0.0.1:51140: write: broken pipe

I have this issue when I port forward to argocd-server. No problem when port-forwarding to other pods

[MiteshSharma]

Even I am facing the same issue on the Kubernetes server. Although after a few retries i am able to open UI.

kubectl port-forward svc/argocd-server -n argocd 8080:443
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
E0206 14:26:07.317868   30094 portforward.go:362] error creating forwarding stream for port 8080 -> 8080: Timeout occured
Handling connection for 8080
E0206 14:27:02.968909   30094 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occured
Handling connection for 8080

[edersonbadeca]

I have this issue too running in minikube when I try to sync some app.

Handling connection for 9001
Handling connection for 9001
Handling connection for 9001
E0420 16:34:27.261994   40277 portforward.go:372] error copying from remote stream to local connection: readfrom tcp6 [::1]:9001->[::1]:60542: write tcp6 [::1]:9001->[::1]:60542: write: broken pipe
E0420 16:34:27.264820   40277 portforward.go:372] error copying from remote stream to local connection: readfrom tcp6 [::1]:9001->[::1]:60540: write tcp6 [::1]:9001->[::1]:60540: write: broken pipe
E0420 16:34:28.407587   40277 portforward.go:372] error copying from remote stream to local connection: readfrom tcp6 [::1]:9001->[::1]:60520: write tcp6 [::1]:9001->[::1]:60520: write: broken pipe
E0420 16:34:28.408755   40277 portforward.go:372] error copying from remote stream to local connection: readfrom tcp6 [::1]:9001->[::1]:60532: write tcp6 [::1]:9001->[::1]:60532: write: broken pipe

[melalj]

Same here event on the latest version v2.0.0+f5119c0 - It's becoming more frequent than the 1.8.
There is plenty of CPU and RAM on the server, and I couldn't see what is the bottleneck!

[touchthesun]

getting the same issue now, only with argocd, and it was working fine yesterday. any idea what's going on here?

kubectl port-forward svc/argocd-server -n argocd 8080:443 &
andling connection for 8080
E0625 12:03:58.359264 35218 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occurred
E0625 12:04:01.072451 35218 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occurred
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
WARNING: server is not configured with TLS. Proceed (y/n)? y
Handling connection for 8080
E0625 12:04:25.140869 35218 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occurred
E0625 12:04:31.075431 35218 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occurred
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
E0625 12:04:45.143408 35218 portforward.go:340] error creating error stream for port 8080 -> 8080: Timeout occurred

[ahmetbcakici]

I was working on locally with minikube and i faced this problem too.

Edit: Probably no matter it is a cluster on a server / cloud or local, the problem related with argocd?

[ozbillwang]

Still get the same issue with latest argocd, any fixes?

[CzapBran]

I guess it's a silly answer, but has anyone tried setting the service type to NodePort?

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'

Changing the service type and then port forwarding exposed the ArgoCD UI. I am testing this locally on a Minikube cluster

[St0rmRage]

I guess it's a silly answer, but has anyone tried setting the service type to NodePort?

kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'

Changing the service type and then port forwarding exposed the ArgoCD UI. I am testing this locally on a Minikube cluster

tried this too, still same thing... why is the issue closed?

[laviniamnedelea]

I am experiencing the same behavior with minikube. This issue shouldn't be closed.

[object88]

Same here as well. The UI becomes unresponsive.

[diranged]

While we don't use this feature for our production clusters - for our pure dev/test clusters we rely on this and we run into the exact same problem.

[diranged]

I've opened a new fresh bug for this.. #14414

[gravufo]

We are hitting this issue all the time, it's extremely annoying. This issue should not be closed.
Any chance someone a maintainer can comment on this?

[harshita3199]

Did we get anything to resolve this issue. I am also getting the same issue. I am running it in my local system and trying to create an application in argo connecting private github repo. Not able to create the resources and also getting this in my local

E0310 22:16:35.509070   14518 portforward.go:379] error copying from remote stream to local connection: readfrom tcp6 [::1]:8080->[::1]:55278: write tcp6 [::1]:8080->[::1]:55278: write: broken pipe
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080

[MonicaMagoniCom]

I'm experiencing the same issue, only with ArgoCD

[born2ngopi]

i got same issue on my local machine

i use wsl2 and docker dekstop for windows

step:

1. follow this doc without setup ingress, i use port-forwading

$ kubectl port-forward svc/argocd-server -n argocd 8080:443

2. create an user with role

# user
data:
  accounts.ch: apiKey, login
# rbac
data:
  policy.csv: |
    g, ch, role:admin

3. generate token
4. then run curl

$ curl -v -H "Authorization: Bearer $TOKEN" http://localhost:8080/api/v1/applications
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET /api/v1/applications HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.81.0
> Accept: */*
> Authorization: Bearer $TOKEN
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 307 Temporary Redirect
< Content-Type: text/html; charset=utf-8
< Location: https://localhost:8080/api/v1/applications
< Date: Fri, 26 Jul 2024 03:07:46 GMT
< Content-Length: 78
<
<a href="https://localhost:8080/api/v1/applications">Temporary Redirect</a>.

* Connection #0 to host localhost left intact

output from kubectl

Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
E0726 09:49:02.785880   89886 portforward.go:381] error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:8080->127.0.0.1:36644: write tcp4 127.0.0.1:8080->127.0.0.1:36644: write: broken pipe
Handling connection for 8080
Handling connection for 8080
E0726 09:59:30.066055   89886 portforward.go:381] error copying from remote stream to local connection: readfrom tcp4 127.0.0.1:8080->127.0.0.1:56782: write tcp4 127.0.0.1:8080->127.0.0.1:56782: write: broken pipe

[Hanyman8]

Hello, we have a similar issue

E1107 13:10:30.011081   35972 portforward.go:385] error copying from remote stream to local connection: readfrom tcp6 [::1]:8080->[::1]:36988: write tcp6 [::1]:8080->[::1]:36988: write: broken pipe
E1107 13:10:41.016435   35972 portforward.go:385] error copying from remote stream to local connection: readfrom tcp6 [::1]:8080->[::1]:37026: write tcp6 [::1]:8080->[::1]:37026: write: broken pipe
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
Handling connection for 8080
E1107 13:11:11.030401   35972 portforward.go:351] error creating error stream for port 8080 -> 8080: Timeout occurred
Handling connection for 8080
E1107 13:11:11.060905   35972 portforward.go:374] error creating forwarding stream for port 8080 -> 8080: Timeout occurred
E1107 13:11:11.139602   35972 portforward.go:374] error creating forwarding stream for port 8080 -> 8080: Timeout occurred
Handling connection for 8080
Handling connection for 8080
E1107 13:11:13.208261   35972 portforward.go:374] error creating forwarding stream for port 8080 -> 8080: Timeout occurred
Handling connection for 8080
E1107 13:11:41.044253   35972 portforward.go:351] error creating error stream for port 8080 -> 8080: Timeout occurred
E1107 13:11:42.149718   35972 portforward.go:374] error creating forwarding stream for port 8080 -> 8080: Timeout occurred
E1107 13:11:48.198999   35972 portforward.go:351] error creating error stream for port 8080 -> 8080: Timeout occurred

[q-markglozier]

We have the same problem

[dpunkturban]

We have also the same problem :/

[Archisman-Mridha]

I'm also getting the same problem :(.

[L3monJuic3]

I am also getting the timeout problem