-
Notifications
You must be signed in to change notification settings - Fork 7.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pilot: skip privileged ports when building listeners for non-root gateways #28268
Conversation
@@ -37,6 +37,9 @@ spec: | |||
service.istio.io/canonical-revision: {{ .Values.revision }} | |||
{{- else}} | |||
service.istio.io/canonical-revision: latest | |||
{{- end }} | |||
{{- if not $gateway.runAsRoot }} | |||
istio.io/unprivileged-pod: "true" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any reason to use a label rather than the more common ISTIO_META_UNPRIVILEGED_POD
to pass via node metadata?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm. that might make more sense, and it doesn't require an API change. will do that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks!
/test integ-pilot-k8s-tests_istio |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM put a hold for others to review that were involved in these discussions. If no one is interested we can merge this in |
/retest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm.
/test integ-telemetry-k8s-tests_istio |
Signed-off-by: gargnupur <[email protected]> fix Signed-off-by: gargnupur <[email protected]> add yaml Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> Delete reference to old ISTIO_META_PROXY_XDS_VIA_AGENT (istio#28203) * update expose istiod * add https sample * fix tab * update host + domain * fix lint * fix lint * tweak host * fix lint * use tls port * name port correctly * change default to tls * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 7feb468. * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 98209a0. * use istiod-remote since pilot is still enabled on remote cluster * loose up on host name * adding notes * clean up this in preview profile Co-authored-by: Iris <[email protected]> Avoid telemetry cluster metadata override (istio#28171) * fix cluster metadata override * test * fix * fix * fix again * clean add telemetry test for customize metrics (istio#27844) * add test for customize metrics * address comments * add remove tag check * fix test Delete istiod pods on cleanup (istio#28205) Otherwise they stay around and can cause other tests to fail. In a concrete example, deployment "istiod-canary" stays live and interferes in pilot's TestMultiRevision test, which also deploys a "istiod-canary", but, since a deployment with that name already exists, operator doesn't redeploy it, because it's already there. Fix HTTPs on HTTP port passthrough (istio#28166) * Fix HTTPs on HTTP port passthrough * Add note remove 1.7 telemetry filters from charts (istio#28195) use correct env var name (istio#28217) Align Ingress resource status updates with Ingresses targeted in controller (istio#28225) make istiod-remote depend on base when installation (istio#28219) Add remoteIpBlocks functionality to AuthorizationPolicy (istio#27906) * create remoteIpBlocks and update ipBlocks for AuthorizationPolicy By adding remoteIpBlocks and notRemoteIpBlocks in Source, an AuthorizationPolicy can trigger actions based on the original client IP address gleaned from the X-Forwarded-For header or the proxy protocol. The ipBlocks and notIpBlocks fields have also been updated to use direct_remote_ip in Envoy instead of source_ip * use correct attribute for RemoteIpBlocks * fix unit tests and add integration tests for remote.ip attribute * fix notRemoteIp integration test * initialize headers if it is nil * Combine remoteIp tests into IngressGateway test and add release note * add titles to links * remove unneeded tests * fix quotes in releasenote, run make gen * make upgradeNotes a list Remove deprecated istio-coredns plugin (istio#28179) make stackdriver test platform agnostic (istio#28237) * make stackdriver test platform agnostic * fix * clean up Add Wasm Extension Dashboard (istio#28209) * Add WASM Extension Dashboard * update dashboard * update dashboard and add cpu/mem * address review comment * add excluded * remove extension dashboard from test allowlist.txt * update readme Clean up metadata exchange keys (istio#28249) * clean up * cleanup exchange key Remove unnecessary warning log from ingress status watcher (istio#28254) vm health checking (istio#28142) * impl with pilot * Remove redundant import * Remove redundant return * address some concerns * address more concerns * Add tests * fix ci? * fix ci? Automator: update proxy@master in istio/istio@master (istio#27786) pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list (istio#28260) * pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list Signed-off-by: Yaroslav Skopets <[email protected]> * docs: add release notes Signed-off-by: Yaroslav Skopets <[email protected]> Adjust Wasm VMs charts order and Add release note (istio#28251) * Adjust Wasm VMs charts order * add release note * replace wasm extension dashboard with real ID Issue istio#27606: Minor Bug fixes, mostly renaming (istio#28156) Cleanup ADS tests (istio#28275) * Cleanup ADS tests * fix lint * fix lint Temporarily skip ratelimit tests (istio#28286) To help master/1.8 get to a merge-able state Add warning for legacy FQDN gateway reference (istio#27948) * Add warning for legacy FQDN gateway reference * fix lint * Add more warnings Fixes for trust domain configuration (istio#28127) * Fixes for trust domain configuration We want to ensure we take values.global.trustDomain as default, fallback to meshConfig.trustDomain, and ensure this is passed through to all parts of the code. This fixes the breakage in istio/istio.io#8301 * fix lint Status improvements (istio#28136) * Consolidate ledger and status implementation * Add ownerReference for garbage collection * write observedGeneration for status * cleanup rebase errors * remove garbage from pr * fix test failures * Fix receiver linting * fix broken unit tests * fix init for route test * Fix test failures * add missing ledger to test * Add release notes * Reorganize status controller start * fix race * separate init and start funcs * add newline * remove test sprawl * reset retention Add size to ADS push log (istio#28262) Add README.md for vendor optimized profiles (istio#28155) * Add README.profiles for vendor optimized profiles * Another attempt at the table Fix operator revision handling (istio#28044) * Fix operator revision handling * Add revision to installation CR * Add revision to each resource label * Update label handling * Add deployment spec template labels, clean up logging * Fix test * Update integration test * Make gen * Fix test * Testing * Fix tests Futureproof telemetry envoyfilters a bit (istio#28176) remove the install comment (istio#28243) * remove the install comment * Revert "remove the install comment" This reverts commit 60bc649. * Update gen-eastwest-gateway.sh pilot: skip privileged ports when building listeners for non-root gateways (istio#28268) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label Automator: update proxy@master in istio/istio@master (istio#28281) istioctl bug-report: do not override system namespaces if --exclude flag is provided (istio#27989) Add ingress status integration test (istio#28263) clean up: extension configs (istio#28277) * clean up extension configs Signed-off-by: Kuat Yessenov <[email protected]> * make gen Signed-off-by: Kuat Yessenov <[email protected]> Show empty routes in pc routes (istio#28170) ``` NOTE: This output only contains routes loaded via RDS. NAME DOMAINS MATCH VIRTUAL SERVICE https.443.https-443-ingress-service1-default-0.service1-istio-autogenerated-k8s-ingress.istio-system * /* 404 https.443.https-443-ingress-service2-default-0.service2-istio-autogenerated-k8s-ingress.istio-system * /* 404 http.80 service1.demo.........io /* service1-demo-......-io-service1-istio-autogenerated-k8s-ingress.default http.80 service2.demo.........io /* service2-demo-.....i-io-service2-istio-autogenerated-k8s-ingress.default * /stats/prometheus* * /healthz/ready* ``` The first 2 lines would not show up without this PR Add warnings for unknown fields in EnvoyFilter (istio#28227) Fixes istio#26390 Update rather than patch webhook configuration (istio#28228) * Update rather than patch webhook configuration This is a far more flexible pattern, allowing us to have multiple webhooks and patch them successful. This pattern follows what the cert-manager does in their webhook patcher (see pkg/controller/cainjector), which I consider to be top quality code. * update rbac Improve error when users use removed addon (istio#28241) * Improve error when users use removed addon After and before: ``` $ grun ./istioctl/cmd/istioctl manifest generate --set addonComponents.foo.enabled=true -d manifests Error: component "foo" does not exist $ ik manifest generate --set addonComponents.foo.enabled=true -d manifests Error: stat manifests/charts/addons/foo: no such file or directory ``` * Fix test When installing istio-cni remove existing istio-cni plugin before inserting a new one (istio#28258) * Remove istio-cni plugin before inserting a new one * docs: add release notes Automator: update common-files@master in istio/istio@master (istio#28278) Make ingress gateway selector in status watcher match the one used to generate gateway (istio#28279) * Check for empty ingress service value when converting ingress to gateway * Pull ingress gateway selector logic into own func * Use same ingress gateway selector logic for status watcher as when generating gateways * Fix status watcher test Remove time.Sleep hacks for fast tests/non-flaky (istio#27741) * Remove time.Sleep hacks for fast tests * fix flake Add grafana templating query for DS_PROMETHEUS and add missing datasource (istio#28320) * Add grafana templating query for DS_PROMETHEUS and add missing datasource * make extension dashboard viewonly skip ingress test in multicluster (istio#28321) E2E test for trust domain alias client side secure naming. (istio#28206) * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * fix host to address * update script * refactor based on comments * updated comments * add build constraints * lint fix * fixes based on comments Samples: use more common images and delete useless samples (istio#28215) Signed-off-by: Xiang Dai <[email protected]> Wait until ACK before sending additional pushes (istio#28261) * Wait until ACK before sending additional pushes Fixes: istio#25685 At large scale, Envoy suffers from overload of XDS pushes, and there is no backpressure in the system. Other control planes, such as any based on go-control-plane, outperform Istio in config update propogations under load as a result. This changes adds a backpressure mechanism to ensure we do not push more configs than Envoy can handle. By slowing down the pushes, the propogation time of new configurations actually increases. We do this by keeping note, but not sending, any push requests where that TypeUrl has an un-ACKed request in flight. When we get an ACK, if there is a pending push request we will immediately trigger it. This effectively means that in a high churn environment, each proxy will always have exactly 1 outstanding push per type, and when the ACK is recieved we will immediately send a new update. This PR is co-authored by Steve, who did a huge amount of work in developing this into the state it is today, as wel as finding and testing the problem. See istio#27563 for much of this work. Co-Authored-By: Steven Dake [email protected] * Refactor and cleanup tests * Add test istioctl: fix failure when passing flags to `go test` (istio#28332) add xds proxy metrics (istio#28267) * add xds proxy metrics Signed-off-by: Rama Chavali <[email protected]> * lint Signed-off-by: Rama Chavali <[email protected]> * fix description Signed-off-by: Rama Chavali <[email protected]> remove-from-mesh: skip system namespace to remove sidecar (istio#28187) * remove-from-mesh: skip system namespace to remove sidecar * check for -i Remove accidentally merged debug logs (istio#28331) update warning message for upgrading istio version (istio#28303) * update warning message for upgrading istio version * add use before istioctl analyze Update README.md (istio#28272) * Update README.md * Update README.md Xds proxy improve (istio#28307) * Prevent goroutine leak * Accelerate by splitting upstream request and response handling * fix lint fix manifestpath for verify install. (istio#28345) Fix ADSC race (istio#28342) * Fix ADSC race * fix * fix ut * Update pkg/istio-agent/local_xds_generator.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pkg/adsc/adsc_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pilot/pkg/xds/lds_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Apply shamsher's suggestions from code review Co-authored-by: Shamsher Ansari <[email protected]> Co-authored-by: Shamsher Ansari <[email protected]> List Istio injectors (istio#27849) * Refactor and rename command * Code nits * Fix typo * Print message if no namespaces have injection * Code nits * Case where an injected namespace does not yet have pods * Code cleanup kube-inject: hide namespace flag in favour of istioNamespace (istio#28067) Automator: update proxy@master in istio/istio@master (istio#28355) Fix test race in FilterGatewayClusterConf (istio#28330) Example failure https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/4087 In generally, we need a better way to mutate feature flags in tests. Maybe a conditionally compiled mutex. Will open an issue to track Networking: Add scaffold of tunnel EDS builder (istio#28244) * add endpoint tunnel supportablity field Signed-off-by: Yuchen Dai <[email protected]> * edsbtswip Signed-off-by: Yuchen Dai <[email protected]> * fix import Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * EndpointsByNetworkFilter refactor Signed-off-by: Yuchen Dai <[email protected]> * fix pkg/pilot/ tests Signed-off-by: Yuchen Dai <[email protected]> * ep builder decide build out tunnel type Signed-off-by: Yuchen Dai <[email protected]> * add basic tunnel eds test Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix proxy metadata access Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix endpoint COW, h2support bitfield bug Signed-off-by: Yuchen Dai <[email protected]> * make gen without fmt Signed-off-by: Yuchen Dai <[email protected]> * add errgo Signed-off-by: Yuchen Dai <[email protected]> * address comment Signed-off-by: Yuchen Dai <[email protected]> * fmt Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> cni:fix order dependent test failures (istio#28349) The `interceptRuleMgrType` decalred in main.go as "iptables", it would be changed to "mock" in func resetGlobalTestVariables(). When run single test, it would be "iptables" and make test failed. Signed-off-by: Xiang Dai <[email protected]> fix uninstall test (istio#28335) * fix uninstall test * revert prow change * address comment * add logic to deduplicate Adding Route Specific RateLimiting Test Signed-off-by: gargnupur <[email protected]> remove debug info Signed-off-by: gargnupur <[email protected]> fix failure Signed-off-by: gargnupur <[email protected]>
Signed-off-by: gargnupur <[email protected]> fix Signed-off-by: gargnupur <[email protected]> add yaml Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> Delete reference to old ISTIO_META_PROXY_XDS_VIA_AGENT (#28203) * update expose istiod * add https sample * fix tab * update host + domain * fix lint * fix lint * tweak host * fix lint * use tls port * name port correctly * change default to tls * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 7feb468. * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 98209a0. * use istiod-remote since pilot is still enabled on remote cluster * loose up on host name * adding notes * clean up this in preview profile Co-authored-by: Iris <[email protected]> Avoid telemetry cluster metadata override (#28171) * fix cluster metadata override * test * fix * fix * fix again * clean add telemetry test for customize metrics (#27844) * add test for customize metrics * address comments * add remove tag check * fix test Delete istiod pods on cleanup (#28205) Otherwise they stay around and can cause other tests to fail. In a concrete example, deployment "istiod-canary" stays live and interferes in pilot's TestMultiRevision test, which also deploys a "istiod-canary", but, since a deployment with that name already exists, operator doesn't redeploy it, because it's already there. Fix HTTPs on HTTP port passthrough (#28166) * Fix HTTPs on HTTP port passthrough * Add note remove 1.7 telemetry filters from charts (#28195) use correct env var name (#28217) Align Ingress resource status updates with Ingresses targeted in controller (#28225) make istiod-remote depend on base when installation (#28219) Add remoteIpBlocks functionality to AuthorizationPolicy (#27906) * create remoteIpBlocks and update ipBlocks for AuthorizationPolicy By adding remoteIpBlocks and notRemoteIpBlocks in Source, an AuthorizationPolicy can trigger actions based on the original client IP address gleaned from the X-Forwarded-For header or the proxy protocol. The ipBlocks and notIpBlocks fields have also been updated to use direct_remote_ip in Envoy instead of source_ip * use correct attribute for RemoteIpBlocks * fix unit tests and add integration tests for remote.ip attribute * fix notRemoteIp integration test * initialize headers if it is nil * Combine remoteIp tests into IngressGateway test and add release note * add titles to links * remove unneeded tests * fix quotes in releasenote, run make gen * make upgradeNotes a list Remove deprecated istio-coredns plugin (#28179) make stackdriver test platform agnostic (#28237) * make stackdriver test platform agnostic * fix * clean up Add Wasm Extension Dashboard (#28209) * Add WASM Extension Dashboard * update dashboard * update dashboard and add cpu/mem * address review comment * add excluded * remove extension dashboard from test allowlist.txt * update readme Clean up metadata exchange keys (#28249) * clean up * cleanup exchange key Remove unnecessary warning log from ingress status watcher (#28254) vm health checking (#28142) * impl with pilot * Remove redundant import * Remove redundant return * address some concerns * address more concerns * Add tests * fix ci? * fix ci? Automator: update proxy@master in istio/istio@master (#27786) pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list (#28260) * pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list Signed-off-by: Yaroslav Skopets <[email protected]> * docs: add release notes Signed-off-by: Yaroslav Skopets <[email protected]> Adjust Wasm VMs charts order and Add release note (#28251) * Adjust Wasm VMs charts order * add release note * replace wasm extension dashboard with real ID Issue #27606: Minor Bug fixes, mostly renaming (#28156) Cleanup ADS tests (#28275) * Cleanup ADS tests * fix lint * fix lint Temporarily skip ratelimit tests (#28286) To help master/1.8 get to a merge-able state Add warning for legacy FQDN gateway reference (#27948) * Add warning for legacy FQDN gateway reference * fix lint * Add more warnings Fixes for trust domain configuration (#28127) * Fixes for trust domain configuration We want to ensure we take values.global.trustDomain as default, fallback to meshConfig.trustDomain, and ensure this is passed through to all parts of the code. This fixes the breakage in istio/istio.io#8301 * fix lint Status improvements (#28136) * Consolidate ledger and status implementation * Add ownerReference for garbage collection * write observedGeneration for status * cleanup rebase errors * remove garbage from pr * fix test failures * Fix receiver linting * fix broken unit tests * fix init for route test * Fix test failures * add missing ledger to test * Add release notes * Reorganize status controller start * fix race * separate init and start funcs * add newline * remove test sprawl * reset retention Add size to ADS push log (#28262) Add README.md for vendor optimized profiles (#28155) * Add README.profiles for vendor optimized profiles * Another attempt at the table Fix operator revision handling (#28044) * Fix operator revision handling * Add revision to installation CR * Add revision to each resource label * Update label handling * Add deployment spec template labels, clean up logging * Fix test * Update integration test * Make gen * Fix test * Testing * Fix tests Futureproof telemetry envoyfilters a bit (#28176) remove the install comment (#28243) * remove the install comment * Revert "remove the install comment" This reverts commit 60bc649. * Update gen-eastwest-gateway.sh pilot: skip privileged ports when building listeners for non-root gateways (#28268) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label Automator: update proxy@master in istio/istio@master (#28281) istioctl bug-report: do not override system namespaces if --exclude flag is provided (#27989) Add ingress status integration test (#28263) clean up: extension configs (#28277) * clean up extension configs Signed-off-by: Kuat Yessenov <[email protected]> * make gen Signed-off-by: Kuat Yessenov <[email protected]> Show empty routes in pc routes (#28170) ``` NOTE: This output only contains routes loaded via RDS. NAME DOMAINS MATCH VIRTUAL SERVICE https.443.https-443-ingress-service1-default-0.service1-istio-autogenerated-k8s-ingress.istio-system * /* 404 https.443.https-443-ingress-service2-default-0.service2-istio-autogenerated-k8s-ingress.istio-system * /* 404 http.80 service1.demo.........io /* service1-demo-......-io-service1-istio-autogenerated-k8s-ingress.default http.80 service2.demo.........io /* service2-demo-.....i-io-service2-istio-autogenerated-k8s-ingress.default * /stats/prometheus* * /healthz/ready* ``` The first 2 lines would not show up without this PR Add warnings for unknown fields in EnvoyFilter (#28227) Fixes #26390 Update rather than patch webhook configuration (#28228) * Update rather than patch webhook configuration This is a far more flexible pattern, allowing us to have multiple webhooks and patch them successful. This pattern follows what the cert-manager does in their webhook patcher (see pkg/controller/cainjector), which I consider to be top quality code. * update rbac Improve error when users use removed addon (#28241) * Improve error when users use removed addon After and before: ``` $ grun ./istioctl/cmd/istioctl manifest generate --set addonComponents.foo.enabled=true -d manifests Error: component "foo" does not exist $ ik manifest generate --set addonComponents.foo.enabled=true -d manifests Error: stat manifests/charts/addons/foo: no such file or directory ``` * Fix test When installing istio-cni remove existing istio-cni plugin before inserting a new one (#28258) * Remove istio-cni plugin before inserting a new one * docs: add release notes Automator: update common-files@master in istio/istio@master (#28278) Make ingress gateway selector in status watcher match the one used to generate gateway (#28279) * Check for empty ingress service value when converting ingress to gateway * Pull ingress gateway selector logic into own func * Use same ingress gateway selector logic for status watcher as when generating gateways * Fix status watcher test Remove time.Sleep hacks for fast tests/non-flaky (#27741) * Remove time.Sleep hacks for fast tests * fix flake Add grafana templating query for DS_PROMETHEUS and add missing datasource (#28320) * Add grafana templating query for DS_PROMETHEUS and add missing datasource * make extension dashboard viewonly skip ingress test in multicluster (#28321) E2E test for trust domain alias client side secure naming. (#28206) * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * fix host to address * update script * refactor based on comments * updated comments * add build constraints * lint fix * fixes based on comments Samples: use more common images and delete useless samples (#28215) Signed-off-by: Xiang Dai <[email protected]> Wait until ACK before sending additional pushes (#28261) * Wait until ACK before sending additional pushes Fixes: #25685 At large scale, Envoy suffers from overload of XDS pushes, and there is no backpressure in the system. Other control planes, such as any based on go-control-plane, outperform Istio in config update propogations under load as a result. This changes adds a backpressure mechanism to ensure we do not push more configs than Envoy can handle. By slowing down the pushes, the propogation time of new configurations actually increases. We do this by keeping note, but not sending, any push requests where that TypeUrl has an un-ACKed request in flight. When we get an ACK, if there is a pending push request we will immediately trigger it. This effectively means that in a high churn environment, each proxy will always have exactly 1 outstanding push per type, and when the ACK is recieved we will immediately send a new update. This PR is co-authored by Steve, who did a huge amount of work in developing this into the state it is today, as wel as finding and testing the problem. See #27563 for much of this work. Co-Authored-By: Steven Dake [email protected] * Refactor and cleanup tests * Add test istioctl: fix failure when passing flags to `go test` (#28332) add xds proxy metrics (#28267) * add xds proxy metrics Signed-off-by: Rama Chavali <[email protected]> * lint Signed-off-by: Rama Chavali <[email protected]> * fix description Signed-off-by: Rama Chavali <[email protected]> remove-from-mesh: skip system namespace to remove sidecar (#28187) * remove-from-mesh: skip system namespace to remove sidecar * check for -i Remove accidentally merged debug logs (#28331) update warning message for upgrading istio version (#28303) * update warning message for upgrading istio version * add use before istioctl analyze Update README.md (#28272) * Update README.md * Update README.md Xds proxy improve (#28307) * Prevent goroutine leak * Accelerate by splitting upstream request and response handling * fix lint fix manifestpath for verify install. (#28345) Fix ADSC race (#28342) * Fix ADSC race * fix * fix ut * Update pkg/istio-agent/local_xds_generator.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pkg/adsc/adsc_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pilot/pkg/xds/lds_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Apply shamsher's suggestions from code review Co-authored-by: Shamsher Ansari <[email protected]> Co-authored-by: Shamsher Ansari <[email protected]> List Istio injectors (#27849) * Refactor and rename command * Code nits * Fix typo * Print message if no namespaces have injection * Code nits * Case where an injected namespace does not yet have pods * Code cleanup kube-inject: hide namespace flag in favour of istioNamespace (#28067) Automator: update proxy@master in istio/istio@master (#28355) Fix test race in FilterGatewayClusterConf (#28330) Example failure https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/4087 In generally, we need a better way to mutate feature flags in tests. Maybe a conditionally compiled mutex. Will open an issue to track Networking: Add scaffold of tunnel EDS builder (#28244) * add endpoint tunnel supportablity field Signed-off-by: Yuchen Dai <[email protected]> * edsbtswip Signed-off-by: Yuchen Dai <[email protected]> * fix import Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * EndpointsByNetworkFilter refactor Signed-off-by: Yuchen Dai <[email protected]> * fix pkg/pilot/ tests Signed-off-by: Yuchen Dai <[email protected]> * ep builder decide build out tunnel type Signed-off-by: Yuchen Dai <[email protected]> * add basic tunnel eds test Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix proxy metadata access Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix endpoint COW, h2support bitfield bug Signed-off-by: Yuchen Dai <[email protected]> * make gen without fmt Signed-off-by: Yuchen Dai <[email protected]> * add errgo Signed-off-by: Yuchen Dai <[email protected]> * address comment Signed-off-by: Yuchen Dai <[email protected]> * fmt Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> cni:fix order dependent test failures (#28349) The `interceptRuleMgrType` decalred in main.go as "iptables", it would be changed to "mock" in func resetGlobalTestVariables(). When run single test, it would be "iptables" and make test failed. Signed-off-by: Xiang Dai <[email protected]> fix uninstall test (#28335) * fix uninstall test * revert prow change * address comment * add logic to deduplicate Adding Route Specific RateLimiting Test Signed-off-by: gargnupur <[email protected]> remove debug info Signed-off-by: gargnupur <[email protected]> fix failure Signed-off-by: gargnupur <[email protected]>
Signed-off-by: gargnupur <[email protected]> fix Signed-off-by: gargnupur <[email protected]> add yaml Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> Delete reference to old ISTIO_META_PROXY_XDS_VIA_AGENT (istio#28203) * update expose istiod * add https sample * fix tab * update host + domain * fix lint * fix lint * tweak host * fix lint * use tls port * name port correctly * change default to tls * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 7feb468. * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 98209a0. * use istiod-remote since pilot is still enabled on remote cluster * loose up on host name * adding notes * clean up this in preview profile Co-authored-by: Iris <[email protected]> Avoid telemetry cluster metadata override (istio#28171) * fix cluster metadata override * test * fix * fix * fix again * clean add telemetry test for customize metrics (istio#27844) * add test for customize metrics * address comments * add remove tag check * fix test Delete istiod pods on cleanup (istio#28205) Otherwise they stay around and can cause other tests to fail. In a concrete example, deployment "istiod-canary" stays live and interferes in pilot's TestMultiRevision test, which also deploys a "istiod-canary", but, since a deployment with that name already exists, operator doesn't redeploy it, because it's already there. Fix HTTPs on HTTP port passthrough (istio#28166) * Fix HTTPs on HTTP port passthrough * Add note remove 1.7 telemetry filters from charts (istio#28195) use correct env var name (istio#28217) Align Ingress resource status updates with Ingresses targeted in controller (istio#28225) make istiod-remote depend on base when installation (istio#28219) Add remoteIpBlocks functionality to AuthorizationPolicy (istio#27906) * create remoteIpBlocks and update ipBlocks for AuthorizationPolicy By adding remoteIpBlocks and notRemoteIpBlocks in Source, an AuthorizationPolicy can trigger actions based on the original client IP address gleaned from the X-Forwarded-For header or the proxy protocol. The ipBlocks and notIpBlocks fields have also been updated to use direct_remote_ip in Envoy instead of source_ip * use correct attribute for RemoteIpBlocks * fix unit tests and add integration tests for remote.ip attribute * fix notRemoteIp integration test * initialize headers if it is nil * Combine remoteIp tests into IngressGateway test and add release note * add titles to links * remove unneeded tests * fix quotes in releasenote, run make gen * make upgradeNotes a list Remove deprecated istio-coredns plugin (istio#28179) make stackdriver test platform agnostic (istio#28237) * make stackdriver test platform agnostic * fix * clean up Add Wasm Extension Dashboard (istio#28209) * Add WASM Extension Dashboard * update dashboard * update dashboard and add cpu/mem * address review comment * add excluded * remove extension dashboard from test allowlist.txt * update readme Clean up metadata exchange keys (istio#28249) * clean up * cleanup exchange key Remove unnecessary warning log from ingress status watcher (istio#28254) vm health checking (istio#28142) * impl with pilot * Remove redundant import * Remove redundant return * address some concerns * address more concerns * Add tests * fix ci? * fix ci? Automator: update proxy@master in istio/istio@master (istio#27786) pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list (istio#28260) * pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list Signed-off-by: Yaroslav Skopets <[email protected]> * docs: add release notes Signed-off-by: Yaroslav Skopets <[email protected]> Adjust Wasm VMs charts order and Add release note (istio#28251) * Adjust Wasm VMs charts order * add release note * replace wasm extension dashboard with real ID Issue istio#27606: Minor Bug fixes, mostly renaming (istio#28156) Cleanup ADS tests (istio#28275) * Cleanup ADS tests * fix lint * fix lint Temporarily skip ratelimit tests (istio#28286) To help master/1.8 get to a merge-able state Add warning for legacy FQDN gateway reference (istio#27948) * Add warning for legacy FQDN gateway reference * fix lint * Add more warnings Fixes for trust domain configuration (istio#28127) * Fixes for trust domain configuration We want to ensure we take values.global.trustDomain as default, fallback to meshConfig.trustDomain, and ensure this is passed through to all parts of the code. This fixes the breakage in istio/istio.io#8301 * fix lint Status improvements (istio#28136) * Consolidate ledger and status implementation * Add ownerReference for garbage collection * write observedGeneration for status * cleanup rebase errors * remove garbage from pr * fix test failures * Fix receiver linting * fix broken unit tests * fix init for route test * Fix test failures * add missing ledger to test * Add release notes * Reorganize status controller start * fix race * separate init and start funcs * add newline * remove test sprawl * reset retention Add size to ADS push log (istio#28262) Add README.md for vendor optimized profiles (istio#28155) * Add README.profiles for vendor optimized profiles * Another attempt at the table Fix operator revision handling (istio#28044) * Fix operator revision handling * Add revision to installation CR * Add revision to each resource label * Update label handling * Add deployment spec template labels, clean up logging * Fix test * Update integration test * Make gen * Fix test * Testing * Fix tests Futureproof telemetry envoyfilters a bit (istio#28176) remove the install comment (istio#28243) * remove the install comment * Revert "remove the install comment" This reverts commit 60bc649. * Update gen-eastwest-gateway.sh pilot: skip privileged ports when building listeners for non-root gateways (istio#28268) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label Automator: update proxy@master in istio/istio@master (istio#28281) istioctl bug-report: do not override system namespaces if --exclude flag is provided (istio#27989) Add ingress status integration test (istio#28263) clean up: extension configs (istio#28277) * clean up extension configs Signed-off-by: Kuat Yessenov <[email protected]> * make gen Signed-off-by: Kuat Yessenov <[email protected]> Show empty routes in pc routes (istio#28170) ``` NOTE: This output only contains routes loaded via RDS. NAME DOMAINS MATCH VIRTUAL SERVICE https.443.https-443-ingress-service1-default-0.service1-istio-autogenerated-k8s-ingress.istio-system * /* 404 https.443.https-443-ingress-service2-default-0.service2-istio-autogenerated-k8s-ingress.istio-system * /* 404 http.80 service1.demo.........io /* service1-demo-......-io-service1-istio-autogenerated-k8s-ingress.default http.80 service2.demo.........io /* service2-demo-.....i-io-service2-istio-autogenerated-k8s-ingress.default * /stats/prometheus* * /healthz/ready* ``` The first 2 lines would not show up without this PR Add warnings for unknown fields in EnvoyFilter (istio#28227) Fixes istio#26390 Update rather than patch webhook configuration (istio#28228) * Update rather than patch webhook configuration This is a far more flexible pattern, allowing us to have multiple webhooks and patch them successful. This pattern follows what the cert-manager does in their webhook patcher (see pkg/controller/cainjector), which I consider to be top quality code. * update rbac Improve error when users use removed addon (istio#28241) * Improve error when users use removed addon After and before: ``` $ grun ./istioctl/cmd/istioctl manifest generate --set addonComponents.foo.enabled=true -d manifests Error: component "foo" does not exist $ ik manifest generate --set addonComponents.foo.enabled=true -d manifests Error: stat manifests/charts/addons/foo: no such file or directory ``` * Fix test When installing istio-cni remove existing istio-cni plugin before inserting a new one (istio#28258) * Remove istio-cni plugin before inserting a new one * docs: add release notes Automator: update common-files@master in istio/istio@master (istio#28278) Make ingress gateway selector in status watcher match the one used to generate gateway (istio#28279) * Check for empty ingress service value when converting ingress to gateway * Pull ingress gateway selector logic into own func * Use same ingress gateway selector logic for status watcher as when generating gateways * Fix status watcher test Remove time.Sleep hacks for fast tests/non-flaky (istio#27741) * Remove time.Sleep hacks for fast tests * fix flake Add grafana templating query for DS_PROMETHEUS and add missing datasource (istio#28320) * Add grafana templating query for DS_PROMETHEUS and add missing datasource * make extension dashboard viewonly skip ingress test in multicluster (istio#28321) E2E test for trust domain alias client side secure naming. (istio#28206) * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * fix host to address * update script * refactor based on comments * updated comments * add build constraints * lint fix * fixes based on comments Samples: use more common images and delete useless samples (istio#28215) Signed-off-by: Xiang Dai <[email protected]> Wait until ACK before sending additional pushes (istio#28261) * Wait until ACK before sending additional pushes Fixes: istio#25685 At large scale, Envoy suffers from overload of XDS pushes, and there is no backpressure in the system. Other control planes, such as any based on go-control-plane, outperform Istio in config update propogations under load as a result. This changes adds a backpressure mechanism to ensure we do not push more configs than Envoy can handle. By slowing down the pushes, the propogation time of new configurations actually increases. We do this by keeping note, but not sending, any push requests where that TypeUrl has an un-ACKed request in flight. When we get an ACK, if there is a pending push request we will immediately trigger it. This effectively means that in a high churn environment, each proxy will always have exactly 1 outstanding push per type, and when the ACK is recieved we will immediately send a new update. This PR is co-authored by Steve, who did a huge amount of work in developing this into the state it is today, as wel as finding and testing the problem. See istio#27563 for much of this work. Co-Authored-By: Steven Dake [email protected] * Refactor and cleanup tests * Add test istioctl: fix failure when passing flags to `go test` (istio#28332) add xds proxy metrics (istio#28267) * add xds proxy metrics Signed-off-by: Rama Chavali <[email protected]> * lint Signed-off-by: Rama Chavali <[email protected]> * fix description Signed-off-by: Rama Chavali <[email protected]> remove-from-mesh: skip system namespace to remove sidecar (istio#28187) * remove-from-mesh: skip system namespace to remove sidecar * check for -i Remove accidentally merged debug logs (istio#28331) update warning message for upgrading istio version (istio#28303) * update warning message for upgrading istio version * add use before istioctl analyze Update README.md (istio#28272) * Update README.md * Update README.md Xds proxy improve (istio#28307) * Prevent goroutine leak * Accelerate by splitting upstream request and response handling * fix lint fix manifestpath for verify install. (istio#28345) Fix ADSC race (istio#28342) * Fix ADSC race * fix * fix ut * Update pkg/istio-agent/local_xds_generator.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pkg/adsc/adsc_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pilot/pkg/xds/lds_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Apply shamsher's suggestions from code review Co-authored-by: Shamsher Ansari <[email protected]> Co-authored-by: Shamsher Ansari <[email protected]> List Istio injectors (istio#27849) * Refactor and rename command * Code nits * Fix typo * Print message if no namespaces have injection * Code nits * Case where an injected namespace does not yet have pods * Code cleanup kube-inject: hide namespace flag in favour of istioNamespace (istio#28067) Automator: update proxy@master in istio/istio@master (istio#28355) Fix test race in FilterGatewayClusterConf (istio#28330) Example failure https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/4087 In generally, we need a better way to mutate feature flags in tests. Maybe a conditionally compiled mutex. Will open an issue to track Networking: Add scaffold of tunnel EDS builder (istio#28244) * add endpoint tunnel supportablity field Signed-off-by: Yuchen Dai <[email protected]> * edsbtswip Signed-off-by: Yuchen Dai <[email protected]> * fix import Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * EndpointsByNetworkFilter refactor Signed-off-by: Yuchen Dai <[email protected]> * fix pkg/pilot/ tests Signed-off-by: Yuchen Dai <[email protected]> * ep builder decide build out tunnel type Signed-off-by: Yuchen Dai <[email protected]> * add basic tunnel eds test Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix proxy metadata access Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix endpoint COW, h2support bitfield bug Signed-off-by: Yuchen Dai <[email protected]> * make gen without fmt Signed-off-by: Yuchen Dai <[email protected]> * add errgo Signed-off-by: Yuchen Dai <[email protected]> * address comment Signed-off-by: Yuchen Dai <[email protected]> * fmt Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> cni:fix order dependent test failures (istio#28349) The `interceptRuleMgrType` decalred in main.go as "iptables", it would be changed to "mock" in func resetGlobalTestVariables(). When run single test, it would be "iptables" and make test failed. Signed-off-by: Xiang Dai <[email protected]> fix uninstall test (istio#28335) * fix uninstall test * revert prow change * address comment * add logic to deduplicate Adding Route Specific RateLimiting Test Signed-off-by: gargnupur <[email protected]> remove debug info Signed-off-by: gargnupur <[email protected]> fix failure Signed-off-by: gargnupur <[email protected]>
This is also needed in 1.8 to avoid issues like #27566 |
/cherrypick release-1.8 |
@elfinhe: #28268 failed to apply on top of branch "release-1.8":
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@elfinhe: new issue created for failed cherrypick: #30517 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
…eways (istio#28268) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label
…eways (#28268) (#30528) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label Co-authored-by: Daniel Grimm <[email protected]>
To fix #27566 -- even though the crash is gone, the gateway will still send a NACK and will essentially be blocked from receiving further updates.
I tried to make it as non-intrusive as possible, only marking the pod if you're deploying a non-root gateway, so that root gateway deployments do not require any changes (as they're also not affected by the problem).