-
Notifications
You must be signed in to change notification settings - Fork 7.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add size to ADS push log #28262
Merged
Merged
Add size to ADS push log #28262
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
howardjohn
added
the
release-notes-none
Indicates a PR that does not require release notes.
label
Oct 25, 2020
istio-testing
added
the
do-not-merge/work-in-progress
Block merging of a PR because it isn't ready yet.
label
Oct 25, 2020
google-cla
bot
added
the
cla: yes
Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA.
label
Oct 25, 2020
Skipping CI for Draft Pull Request. |
istio-testing
added
the
size/M
Denotes a PR that changes 30-99 lines, ignoring generated files.
label
Oct 25, 2020
istio-testing
removed
the
do-not-merge/work-in-progress
Block merging of a PR because it isn't ready yet.
label
Oct 26, 2020
/retest |
hzxuzhonghu
approved these changes
Oct 27, 2020
howardjohn
force-pushed
the
pilot/size-logs
branch
from
October 27, 2020 01:49
df432a4
to
e6b7e98
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
gargnupur
added a commit
to gargnupur/istio
that referenced
this pull request
Nov 13, 2020
Signed-off-by: gargnupur <[email protected]> fix Signed-off-by: gargnupur <[email protected]> add yaml Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> Delete reference to old ISTIO_META_PROXY_XDS_VIA_AGENT (istio#28203) * update expose istiod * add https sample * fix tab * update host + domain * fix lint * fix lint * tweak host * fix lint * use tls port * name port correctly * change default to tls * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 7feb468. * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 98209a0. * use istiod-remote since pilot is still enabled on remote cluster * loose up on host name * adding notes * clean up this in preview profile Co-authored-by: Iris <[email protected]> Avoid telemetry cluster metadata override (istio#28171) * fix cluster metadata override * test * fix * fix * fix again * clean add telemetry test for customize metrics (istio#27844) * add test for customize metrics * address comments * add remove tag check * fix test Delete istiod pods on cleanup (istio#28205) Otherwise they stay around and can cause other tests to fail. In a concrete example, deployment "istiod-canary" stays live and interferes in pilot's TestMultiRevision test, which also deploys a "istiod-canary", but, since a deployment with that name already exists, operator doesn't redeploy it, because it's already there. Fix HTTPs on HTTP port passthrough (istio#28166) * Fix HTTPs on HTTP port passthrough * Add note remove 1.7 telemetry filters from charts (istio#28195) use correct env var name (istio#28217) Align Ingress resource status updates with Ingresses targeted in controller (istio#28225) make istiod-remote depend on base when installation (istio#28219) Add remoteIpBlocks functionality to AuthorizationPolicy (istio#27906) * create remoteIpBlocks and update ipBlocks for AuthorizationPolicy By adding remoteIpBlocks and notRemoteIpBlocks in Source, an AuthorizationPolicy can trigger actions based on the original client IP address gleaned from the X-Forwarded-For header or the proxy protocol. The ipBlocks and notIpBlocks fields have also been updated to use direct_remote_ip in Envoy instead of source_ip * use correct attribute for RemoteIpBlocks * fix unit tests and add integration tests for remote.ip attribute * fix notRemoteIp integration test * initialize headers if it is nil * Combine remoteIp tests into IngressGateway test and add release note * add titles to links * remove unneeded tests * fix quotes in releasenote, run make gen * make upgradeNotes a list Remove deprecated istio-coredns plugin (istio#28179) make stackdriver test platform agnostic (istio#28237) * make stackdriver test platform agnostic * fix * clean up Add Wasm Extension Dashboard (istio#28209) * Add WASM Extension Dashboard * update dashboard * update dashboard and add cpu/mem * address review comment * add excluded * remove extension dashboard from test allowlist.txt * update readme Clean up metadata exchange keys (istio#28249) * clean up * cleanup exchange key Remove unnecessary warning log from ingress status watcher (istio#28254) vm health checking (istio#28142) * impl with pilot * Remove redundant import * Remove redundant return * address some concerns * address more concerns * Add tests * fix ci? * fix ci? Automator: update proxy@master in istio/istio@master (istio#27786) pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list (istio#28260) * pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list Signed-off-by: Yaroslav Skopets <[email protected]> * docs: add release notes Signed-off-by: Yaroslav Skopets <[email protected]> Adjust Wasm VMs charts order and Add release note (istio#28251) * Adjust Wasm VMs charts order * add release note * replace wasm extension dashboard with real ID Issue istio#27606: Minor Bug fixes, mostly renaming (istio#28156) Cleanup ADS tests (istio#28275) * Cleanup ADS tests * fix lint * fix lint Temporarily skip ratelimit tests (istio#28286) To help master/1.8 get to a merge-able state Add warning for legacy FQDN gateway reference (istio#27948) * Add warning for legacy FQDN gateway reference * fix lint * Add more warnings Fixes for trust domain configuration (istio#28127) * Fixes for trust domain configuration We want to ensure we take values.global.trustDomain as default, fallback to meshConfig.trustDomain, and ensure this is passed through to all parts of the code. This fixes the breakage in istio/istio.io#8301 * fix lint Status improvements (istio#28136) * Consolidate ledger and status implementation * Add ownerReference for garbage collection * write observedGeneration for status * cleanup rebase errors * remove garbage from pr * fix test failures * Fix receiver linting * fix broken unit tests * fix init for route test * Fix test failures * add missing ledger to test * Add release notes * Reorganize status controller start * fix race * separate init and start funcs * add newline * remove test sprawl * reset retention Add size to ADS push log (istio#28262) Add README.md for vendor optimized profiles (istio#28155) * Add README.profiles for vendor optimized profiles * Another attempt at the table Fix operator revision handling (istio#28044) * Fix operator revision handling * Add revision to installation CR * Add revision to each resource label * Update label handling * Add deployment spec template labels, clean up logging * Fix test * Update integration test * Make gen * Fix test * Testing * Fix tests Futureproof telemetry envoyfilters a bit (istio#28176) remove the install comment (istio#28243) * remove the install comment * Revert "remove the install comment" This reverts commit 60bc649. * Update gen-eastwest-gateway.sh pilot: skip privileged ports when building listeners for non-root gateways (istio#28268) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label Automator: update proxy@master in istio/istio@master (istio#28281) istioctl bug-report: do not override system namespaces if --exclude flag is provided (istio#27989) Add ingress status integration test (istio#28263) clean up: extension configs (istio#28277) * clean up extension configs Signed-off-by: Kuat Yessenov <[email protected]> * make gen Signed-off-by: Kuat Yessenov <[email protected]> Show empty routes in pc routes (istio#28170) ``` NOTE: This output only contains routes loaded via RDS. NAME DOMAINS MATCH VIRTUAL SERVICE https.443.https-443-ingress-service1-default-0.service1-istio-autogenerated-k8s-ingress.istio-system * /* 404 https.443.https-443-ingress-service2-default-0.service2-istio-autogenerated-k8s-ingress.istio-system * /* 404 http.80 service1.demo.........io /* service1-demo-......-io-service1-istio-autogenerated-k8s-ingress.default http.80 service2.demo.........io /* service2-demo-.....i-io-service2-istio-autogenerated-k8s-ingress.default * /stats/prometheus* * /healthz/ready* ``` The first 2 lines would not show up without this PR Add warnings for unknown fields in EnvoyFilter (istio#28227) Fixes istio#26390 Update rather than patch webhook configuration (istio#28228) * Update rather than patch webhook configuration This is a far more flexible pattern, allowing us to have multiple webhooks and patch them successful. This pattern follows what the cert-manager does in their webhook patcher (see pkg/controller/cainjector), which I consider to be top quality code. * update rbac Improve error when users use removed addon (istio#28241) * Improve error when users use removed addon After and before: ``` $ grun ./istioctl/cmd/istioctl manifest generate --set addonComponents.foo.enabled=true -d manifests Error: component "foo" does not exist $ ik manifest generate --set addonComponents.foo.enabled=true -d manifests Error: stat manifests/charts/addons/foo: no such file or directory ``` * Fix test When installing istio-cni remove existing istio-cni plugin before inserting a new one (istio#28258) * Remove istio-cni plugin before inserting a new one * docs: add release notes Automator: update common-files@master in istio/istio@master (istio#28278) Make ingress gateway selector in status watcher match the one used to generate gateway (istio#28279) * Check for empty ingress service value when converting ingress to gateway * Pull ingress gateway selector logic into own func * Use same ingress gateway selector logic for status watcher as when generating gateways * Fix status watcher test Remove time.Sleep hacks for fast tests/non-flaky (istio#27741) * Remove time.Sleep hacks for fast tests * fix flake Add grafana templating query for DS_PROMETHEUS and add missing datasource (istio#28320) * Add grafana templating query for DS_PROMETHEUS and add missing datasource * make extension dashboard viewonly skip ingress test in multicluster (istio#28321) E2E test for trust domain alias client side secure naming. (istio#28206) * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * fix host to address * update script * refactor based on comments * updated comments * add build constraints * lint fix * fixes based on comments Samples: use more common images and delete useless samples (istio#28215) Signed-off-by: Xiang Dai <[email protected]> Wait until ACK before sending additional pushes (istio#28261) * Wait until ACK before sending additional pushes Fixes: istio#25685 At large scale, Envoy suffers from overload of XDS pushes, and there is no backpressure in the system. Other control planes, such as any based on go-control-plane, outperform Istio in config update propogations under load as a result. This changes adds a backpressure mechanism to ensure we do not push more configs than Envoy can handle. By slowing down the pushes, the propogation time of new configurations actually increases. We do this by keeping note, but not sending, any push requests where that TypeUrl has an un-ACKed request in flight. When we get an ACK, if there is a pending push request we will immediately trigger it. This effectively means that in a high churn environment, each proxy will always have exactly 1 outstanding push per type, and when the ACK is recieved we will immediately send a new update. This PR is co-authored by Steve, who did a huge amount of work in developing this into the state it is today, as wel as finding and testing the problem. See istio#27563 for much of this work. Co-Authored-By: Steven Dake [email protected] * Refactor and cleanup tests * Add test istioctl: fix failure when passing flags to `go test` (istio#28332) add xds proxy metrics (istio#28267) * add xds proxy metrics Signed-off-by: Rama Chavali <[email protected]> * lint Signed-off-by: Rama Chavali <[email protected]> * fix description Signed-off-by: Rama Chavali <[email protected]> remove-from-mesh: skip system namespace to remove sidecar (istio#28187) * remove-from-mesh: skip system namespace to remove sidecar * check for -i Remove accidentally merged debug logs (istio#28331) update warning message for upgrading istio version (istio#28303) * update warning message for upgrading istio version * add use before istioctl analyze Update README.md (istio#28272) * Update README.md * Update README.md Xds proxy improve (istio#28307) * Prevent goroutine leak * Accelerate by splitting upstream request and response handling * fix lint fix manifestpath for verify install. (istio#28345) Fix ADSC race (istio#28342) * Fix ADSC race * fix * fix ut * Update pkg/istio-agent/local_xds_generator.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pkg/adsc/adsc_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pilot/pkg/xds/lds_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Apply shamsher's suggestions from code review Co-authored-by: Shamsher Ansari <[email protected]> Co-authored-by: Shamsher Ansari <[email protected]> List Istio injectors (istio#27849) * Refactor and rename command * Code nits * Fix typo * Print message if no namespaces have injection * Code nits * Case where an injected namespace does not yet have pods * Code cleanup kube-inject: hide namespace flag in favour of istioNamespace (istio#28067) Automator: update proxy@master in istio/istio@master (istio#28355) Fix test race in FilterGatewayClusterConf (istio#28330) Example failure https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/4087 In generally, we need a better way to mutate feature flags in tests. Maybe a conditionally compiled mutex. Will open an issue to track Networking: Add scaffold of tunnel EDS builder (istio#28244) * add endpoint tunnel supportablity field Signed-off-by: Yuchen Dai <[email protected]> * edsbtswip Signed-off-by: Yuchen Dai <[email protected]> * fix import Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * EndpointsByNetworkFilter refactor Signed-off-by: Yuchen Dai <[email protected]> * fix pkg/pilot/ tests Signed-off-by: Yuchen Dai <[email protected]> * ep builder decide build out tunnel type Signed-off-by: Yuchen Dai <[email protected]> * add basic tunnel eds test Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix proxy metadata access Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix endpoint COW, h2support bitfield bug Signed-off-by: Yuchen Dai <[email protected]> * make gen without fmt Signed-off-by: Yuchen Dai <[email protected]> * add errgo Signed-off-by: Yuchen Dai <[email protected]> * address comment Signed-off-by: Yuchen Dai <[email protected]> * fmt Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> cni:fix order dependent test failures (istio#28349) The `interceptRuleMgrType` decalred in main.go as "iptables", it would be changed to "mock" in func resetGlobalTestVariables(). When run single test, it would be "iptables" and make test failed. Signed-off-by: Xiang Dai <[email protected]> fix uninstall test (istio#28335) * fix uninstall test * revert prow change * address comment * add logic to deduplicate Adding Route Specific RateLimiting Test Signed-off-by: gargnupur <[email protected]> remove debug info Signed-off-by: gargnupur <[email protected]> fix failure Signed-off-by: gargnupur <[email protected]>
istio-testing
pushed a commit
that referenced
this pull request
Nov 16, 2020
Signed-off-by: gargnupur <[email protected]> fix Signed-off-by: gargnupur <[email protected]> add yaml Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> Delete reference to old ISTIO_META_PROXY_XDS_VIA_AGENT (#28203) * update expose istiod * add https sample * fix tab * update host + domain * fix lint * fix lint * tweak host * fix lint * use tls port * name port correctly * change default to tls * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 7feb468. * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 98209a0. * use istiod-remote since pilot is still enabled on remote cluster * loose up on host name * adding notes * clean up this in preview profile Co-authored-by: Iris <[email protected]> Avoid telemetry cluster metadata override (#28171) * fix cluster metadata override * test * fix * fix * fix again * clean add telemetry test for customize metrics (#27844) * add test for customize metrics * address comments * add remove tag check * fix test Delete istiod pods on cleanup (#28205) Otherwise they stay around and can cause other tests to fail. In a concrete example, deployment "istiod-canary" stays live and interferes in pilot's TestMultiRevision test, which also deploys a "istiod-canary", but, since a deployment with that name already exists, operator doesn't redeploy it, because it's already there. Fix HTTPs on HTTP port passthrough (#28166) * Fix HTTPs on HTTP port passthrough * Add note remove 1.7 telemetry filters from charts (#28195) use correct env var name (#28217) Align Ingress resource status updates with Ingresses targeted in controller (#28225) make istiod-remote depend on base when installation (#28219) Add remoteIpBlocks functionality to AuthorizationPolicy (#27906) * create remoteIpBlocks and update ipBlocks for AuthorizationPolicy By adding remoteIpBlocks and notRemoteIpBlocks in Source, an AuthorizationPolicy can trigger actions based on the original client IP address gleaned from the X-Forwarded-For header or the proxy protocol. The ipBlocks and notIpBlocks fields have also been updated to use direct_remote_ip in Envoy instead of source_ip * use correct attribute for RemoteIpBlocks * fix unit tests and add integration tests for remote.ip attribute * fix notRemoteIp integration test * initialize headers if it is nil * Combine remoteIp tests into IngressGateway test and add release note * add titles to links * remove unneeded tests * fix quotes in releasenote, run make gen * make upgradeNotes a list Remove deprecated istio-coredns plugin (#28179) make stackdriver test platform agnostic (#28237) * make stackdriver test platform agnostic * fix * clean up Add Wasm Extension Dashboard (#28209) * Add WASM Extension Dashboard * update dashboard * update dashboard and add cpu/mem * address review comment * add excluded * remove extension dashboard from test allowlist.txt * update readme Clean up metadata exchange keys (#28249) * clean up * cleanup exchange key Remove unnecessary warning log from ingress status watcher (#28254) vm health checking (#28142) * impl with pilot * Remove redundant import * Remove redundant return * address some concerns * address more concerns * Add tests * fix ci? * fix ci? Automator: update proxy@master in istio/istio@master (#27786) pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list (#28260) * pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list Signed-off-by: Yaroslav Skopets <[email protected]> * docs: add release notes Signed-off-by: Yaroslav Skopets <[email protected]> Adjust Wasm VMs charts order and Add release note (#28251) * Adjust Wasm VMs charts order * add release note * replace wasm extension dashboard with real ID Issue #27606: Minor Bug fixes, mostly renaming (#28156) Cleanup ADS tests (#28275) * Cleanup ADS tests * fix lint * fix lint Temporarily skip ratelimit tests (#28286) To help master/1.8 get to a merge-able state Add warning for legacy FQDN gateway reference (#27948) * Add warning for legacy FQDN gateway reference * fix lint * Add more warnings Fixes for trust domain configuration (#28127) * Fixes for trust domain configuration We want to ensure we take values.global.trustDomain as default, fallback to meshConfig.trustDomain, and ensure this is passed through to all parts of the code. This fixes the breakage in istio/istio.io#8301 * fix lint Status improvements (#28136) * Consolidate ledger and status implementation * Add ownerReference for garbage collection * write observedGeneration for status * cleanup rebase errors * remove garbage from pr * fix test failures * Fix receiver linting * fix broken unit tests * fix init for route test * Fix test failures * add missing ledger to test * Add release notes * Reorganize status controller start * fix race * separate init and start funcs * add newline * remove test sprawl * reset retention Add size to ADS push log (#28262) Add README.md for vendor optimized profiles (#28155) * Add README.profiles for vendor optimized profiles * Another attempt at the table Fix operator revision handling (#28044) * Fix operator revision handling * Add revision to installation CR * Add revision to each resource label * Update label handling * Add deployment spec template labels, clean up logging * Fix test * Update integration test * Make gen * Fix test * Testing * Fix tests Futureproof telemetry envoyfilters a bit (#28176) remove the install comment (#28243) * remove the install comment * Revert "remove the install comment" This reverts commit 60bc649. * Update gen-eastwest-gateway.sh pilot: skip privileged ports when building listeners for non-root gateways (#28268) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label Automator: update proxy@master in istio/istio@master (#28281) istioctl bug-report: do not override system namespaces if --exclude flag is provided (#27989) Add ingress status integration test (#28263) clean up: extension configs (#28277) * clean up extension configs Signed-off-by: Kuat Yessenov <[email protected]> * make gen Signed-off-by: Kuat Yessenov <[email protected]> Show empty routes in pc routes (#28170) ``` NOTE: This output only contains routes loaded via RDS. NAME DOMAINS MATCH VIRTUAL SERVICE https.443.https-443-ingress-service1-default-0.service1-istio-autogenerated-k8s-ingress.istio-system * /* 404 https.443.https-443-ingress-service2-default-0.service2-istio-autogenerated-k8s-ingress.istio-system * /* 404 http.80 service1.demo.........io /* service1-demo-......-io-service1-istio-autogenerated-k8s-ingress.default http.80 service2.demo.........io /* service2-demo-.....i-io-service2-istio-autogenerated-k8s-ingress.default * /stats/prometheus* * /healthz/ready* ``` The first 2 lines would not show up without this PR Add warnings for unknown fields in EnvoyFilter (#28227) Fixes #26390 Update rather than patch webhook configuration (#28228) * Update rather than patch webhook configuration This is a far more flexible pattern, allowing us to have multiple webhooks and patch them successful. This pattern follows what the cert-manager does in their webhook patcher (see pkg/controller/cainjector), which I consider to be top quality code. * update rbac Improve error when users use removed addon (#28241) * Improve error when users use removed addon After and before: ``` $ grun ./istioctl/cmd/istioctl manifest generate --set addonComponents.foo.enabled=true -d manifests Error: component "foo" does not exist $ ik manifest generate --set addonComponents.foo.enabled=true -d manifests Error: stat manifests/charts/addons/foo: no such file or directory ``` * Fix test When installing istio-cni remove existing istio-cni plugin before inserting a new one (#28258) * Remove istio-cni plugin before inserting a new one * docs: add release notes Automator: update common-files@master in istio/istio@master (#28278) Make ingress gateway selector in status watcher match the one used to generate gateway (#28279) * Check for empty ingress service value when converting ingress to gateway * Pull ingress gateway selector logic into own func * Use same ingress gateway selector logic for status watcher as when generating gateways * Fix status watcher test Remove time.Sleep hacks for fast tests/non-flaky (#27741) * Remove time.Sleep hacks for fast tests * fix flake Add grafana templating query for DS_PROMETHEUS and add missing datasource (#28320) * Add grafana templating query for DS_PROMETHEUS and add missing datasource * make extension dashboard viewonly skip ingress test in multicluster (#28321) E2E test for trust domain alias client side secure naming. (#28206) * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * fix host to address * update script * refactor based on comments * updated comments * add build constraints * lint fix * fixes based on comments Samples: use more common images and delete useless samples (#28215) Signed-off-by: Xiang Dai <[email protected]> Wait until ACK before sending additional pushes (#28261) * Wait until ACK before sending additional pushes Fixes: #25685 At large scale, Envoy suffers from overload of XDS pushes, and there is no backpressure in the system. Other control planes, such as any based on go-control-plane, outperform Istio in config update propogations under load as a result. This changes adds a backpressure mechanism to ensure we do not push more configs than Envoy can handle. By slowing down the pushes, the propogation time of new configurations actually increases. We do this by keeping note, but not sending, any push requests where that TypeUrl has an un-ACKed request in flight. When we get an ACK, if there is a pending push request we will immediately trigger it. This effectively means that in a high churn environment, each proxy will always have exactly 1 outstanding push per type, and when the ACK is recieved we will immediately send a new update. This PR is co-authored by Steve, who did a huge amount of work in developing this into the state it is today, as wel as finding and testing the problem. See #27563 for much of this work. Co-Authored-By: Steven Dake [email protected] * Refactor and cleanup tests * Add test istioctl: fix failure when passing flags to `go test` (#28332) add xds proxy metrics (#28267) * add xds proxy metrics Signed-off-by: Rama Chavali <[email protected]> * lint Signed-off-by: Rama Chavali <[email protected]> * fix description Signed-off-by: Rama Chavali <[email protected]> remove-from-mesh: skip system namespace to remove sidecar (#28187) * remove-from-mesh: skip system namespace to remove sidecar * check for -i Remove accidentally merged debug logs (#28331) update warning message for upgrading istio version (#28303) * update warning message for upgrading istio version * add use before istioctl analyze Update README.md (#28272) * Update README.md * Update README.md Xds proxy improve (#28307) * Prevent goroutine leak * Accelerate by splitting upstream request and response handling * fix lint fix manifestpath for verify install. (#28345) Fix ADSC race (#28342) * Fix ADSC race * fix * fix ut * Update pkg/istio-agent/local_xds_generator.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pkg/adsc/adsc_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pilot/pkg/xds/lds_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Apply shamsher's suggestions from code review Co-authored-by: Shamsher Ansari <[email protected]> Co-authored-by: Shamsher Ansari <[email protected]> List Istio injectors (#27849) * Refactor and rename command * Code nits * Fix typo * Print message if no namespaces have injection * Code nits * Case where an injected namespace does not yet have pods * Code cleanup kube-inject: hide namespace flag in favour of istioNamespace (#28067) Automator: update proxy@master in istio/istio@master (#28355) Fix test race in FilterGatewayClusterConf (#28330) Example failure https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/4087 In generally, we need a better way to mutate feature flags in tests. Maybe a conditionally compiled mutex. Will open an issue to track Networking: Add scaffold of tunnel EDS builder (#28244) * add endpoint tunnel supportablity field Signed-off-by: Yuchen Dai <[email protected]> * edsbtswip Signed-off-by: Yuchen Dai <[email protected]> * fix import Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * EndpointsByNetworkFilter refactor Signed-off-by: Yuchen Dai <[email protected]> * fix pkg/pilot/ tests Signed-off-by: Yuchen Dai <[email protected]> * ep builder decide build out tunnel type Signed-off-by: Yuchen Dai <[email protected]> * add basic tunnel eds test Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix proxy metadata access Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix endpoint COW, h2support bitfield bug Signed-off-by: Yuchen Dai <[email protected]> * make gen without fmt Signed-off-by: Yuchen Dai <[email protected]> * add errgo Signed-off-by: Yuchen Dai <[email protected]> * address comment Signed-off-by: Yuchen Dai <[email protected]> * fmt Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> cni:fix order dependent test failures (#28349) The `interceptRuleMgrType` decalred in main.go as "iptables", it would be changed to "mock" in func resetGlobalTestVariables(). When run single test, it would be "iptables" and make test failed. Signed-off-by: Xiang Dai <[email protected]> fix uninstall test (#28335) * fix uninstall test * revert prow change * address comment * add logic to deduplicate Adding Route Specific RateLimiting Test Signed-off-by: gargnupur <[email protected]> remove debug info Signed-off-by: gargnupur <[email protected]> fix failure Signed-off-by: gargnupur <[email protected]>
daixiang0
pushed a commit
to daixiang0/istio
that referenced
this pull request
Nov 19, 2020
Signed-off-by: gargnupur <[email protected]> fix Signed-off-by: gargnupur <[email protected]> add yaml Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix comments Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> fix build Signed-off-by: gargnupur <[email protected]> Delete reference to old ISTIO_META_PROXY_XDS_VIA_AGENT (istio#28203) * update expose istiod * add https sample * fix tab * update host + domain * fix lint * fix lint * tweak host * fix lint * use tls port * name port correctly * change default to tls * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Update samples/multicluster/expose-istiod.yaml Co-authored-by: Iris <[email protected]> * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 7feb468. * Revert "Update samples/multicluster/expose-istiod.yaml" This reverts commit 98209a0. * use istiod-remote since pilot is still enabled on remote cluster * loose up on host name * adding notes * clean up this in preview profile Co-authored-by: Iris <[email protected]> Avoid telemetry cluster metadata override (istio#28171) * fix cluster metadata override * test * fix * fix * fix again * clean add telemetry test for customize metrics (istio#27844) * add test for customize metrics * address comments * add remove tag check * fix test Delete istiod pods on cleanup (istio#28205) Otherwise they stay around and can cause other tests to fail. In a concrete example, deployment "istiod-canary" stays live and interferes in pilot's TestMultiRevision test, which also deploys a "istiod-canary", but, since a deployment with that name already exists, operator doesn't redeploy it, because it's already there. Fix HTTPs on HTTP port passthrough (istio#28166) * Fix HTTPs on HTTP port passthrough * Add note remove 1.7 telemetry filters from charts (istio#28195) use correct env var name (istio#28217) Align Ingress resource status updates with Ingresses targeted in controller (istio#28225) make istiod-remote depend on base when installation (istio#28219) Add remoteIpBlocks functionality to AuthorizationPolicy (istio#27906) * create remoteIpBlocks and update ipBlocks for AuthorizationPolicy By adding remoteIpBlocks and notRemoteIpBlocks in Source, an AuthorizationPolicy can trigger actions based on the original client IP address gleaned from the X-Forwarded-For header or the proxy protocol. The ipBlocks and notIpBlocks fields have also been updated to use direct_remote_ip in Envoy instead of source_ip * use correct attribute for RemoteIpBlocks * fix unit tests and add integration tests for remote.ip attribute * fix notRemoteIp integration test * initialize headers if it is nil * Combine remoteIp tests into IngressGateway test and add release note * add titles to links * remove unneeded tests * fix quotes in releasenote, run make gen * make upgradeNotes a list Remove deprecated istio-coredns plugin (istio#28179) make stackdriver test platform agnostic (istio#28237) * make stackdriver test platform agnostic * fix * clean up Add Wasm Extension Dashboard (istio#28209) * Add WASM Extension Dashboard * update dashboard * update dashboard and add cpu/mem * address review comment * add excluded * remove extension dashboard from test allowlist.txt * update readme Clean up metadata exchange keys (istio#28249) * clean up * cleanup exchange key Remove unnecessary warning log from ingress status watcher (istio#28254) vm health checking (istio#28142) * impl with pilot * Remove redundant import * Remove redundant return * address some concerns * address more concerns * Add tests * fix ci? * fix ci? Automator: update proxy@master in istio/istio@master (istio#27786) pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list (istio#28260) * pilot: GlobalUnicastIP of a model.Proxy should be set to the 1st applicable IP address in the list Signed-off-by: Yaroslav Skopets <[email protected]> * docs: add release notes Signed-off-by: Yaroslav Skopets <[email protected]> Adjust Wasm VMs charts order and Add release note (istio#28251) * Adjust Wasm VMs charts order * add release note * replace wasm extension dashboard with real ID Issue istio#27606: Minor Bug fixes, mostly renaming (istio#28156) Cleanup ADS tests (istio#28275) * Cleanup ADS tests * fix lint * fix lint Temporarily skip ratelimit tests (istio#28286) To help master/1.8 get to a merge-able state Add warning for legacy FQDN gateway reference (istio#27948) * Add warning for legacy FQDN gateway reference * fix lint * Add more warnings Fixes for trust domain configuration (istio#28127) * Fixes for trust domain configuration We want to ensure we take values.global.trustDomain as default, fallback to meshConfig.trustDomain, and ensure this is passed through to all parts of the code. This fixes the breakage in istio/istio.io#8301 * fix lint Status improvements (istio#28136) * Consolidate ledger and status implementation * Add ownerReference for garbage collection * write observedGeneration for status * cleanup rebase errors * remove garbage from pr * fix test failures * Fix receiver linting * fix broken unit tests * fix init for route test * Fix test failures * add missing ledger to test * Add release notes * Reorganize status controller start * fix race * separate init and start funcs * add newline * remove test sprawl * reset retention Add size to ADS push log (istio#28262) Add README.md for vendor optimized profiles (istio#28155) * Add README.profiles for vendor optimized profiles * Another attempt at the table Fix operator revision handling (istio#28044) * Fix operator revision handling * Add revision to installation CR * Add revision to each resource label * Update label handling * Add deployment spec template labels, clean up logging * Fix test * Update integration test * Make gen * Fix test * Testing * Fix tests Futureproof telemetry envoyfilters a bit (istio#28176) remove the install comment (istio#28243) * remove the install comment * Revert "remove the install comment" This reverts commit 60bc649. * Update gen-eastwest-gateway.sh pilot: skip privileged ports when building listeners for non-root gateways (istio#28268) * pilot: skip privileged ports when building listeners for non-root gateways * Add release note * Use ISTIO_META_UNPRIVILEGED_POD env var instead of a Pod label Automator: update proxy@master in istio/istio@master (istio#28281) istioctl bug-report: do not override system namespaces if --exclude flag is provided (istio#27989) Add ingress status integration test (istio#28263) clean up: extension configs (istio#28277) * clean up extension configs Signed-off-by: Kuat Yessenov <[email protected]> * make gen Signed-off-by: Kuat Yessenov <[email protected]> Show empty routes in pc routes (istio#28170) ``` NOTE: This output only contains routes loaded via RDS. NAME DOMAINS MATCH VIRTUAL SERVICE https.443.https-443-ingress-service1-default-0.service1-istio-autogenerated-k8s-ingress.istio-system * /* 404 https.443.https-443-ingress-service2-default-0.service2-istio-autogenerated-k8s-ingress.istio-system * /* 404 http.80 service1.demo.........io /* service1-demo-......-io-service1-istio-autogenerated-k8s-ingress.default http.80 service2.demo.........io /* service2-demo-.....i-io-service2-istio-autogenerated-k8s-ingress.default * /stats/prometheus* * /healthz/ready* ``` The first 2 lines would not show up without this PR Add warnings for unknown fields in EnvoyFilter (istio#28227) Fixes istio#26390 Update rather than patch webhook configuration (istio#28228) * Update rather than patch webhook configuration This is a far more flexible pattern, allowing us to have multiple webhooks and patch them successful. This pattern follows what the cert-manager does in their webhook patcher (see pkg/controller/cainjector), which I consider to be top quality code. * update rbac Improve error when users use removed addon (istio#28241) * Improve error when users use removed addon After and before: ``` $ grun ./istioctl/cmd/istioctl manifest generate --set addonComponents.foo.enabled=true -d manifests Error: component "foo" does not exist $ ik manifest generate --set addonComponents.foo.enabled=true -d manifests Error: stat manifests/charts/addons/foo: no such file or directory ``` * Fix test When installing istio-cni remove existing istio-cni plugin before inserting a new one (istio#28258) * Remove istio-cni plugin before inserting a new one * docs: add release notes Automator: update common-files@master in istio/istio@master (istio#28278) Make ingress gateway selector in status watcher match the one used to generate gateway (istio#28279) * Check for empty ingress service value when converting ingress to gateway * Pull ingress gateway selector logic into own func * Use same ingress gateway selector logic for status watcher as when generating gateways * Fix status watcher test Remove time.Sleep hacks for fast tests/non-flaky (istio#27741) * Remove time.Sleep hacks for fast tests * fix flake Add grafana templating query for DS_PROMETHEUS and add missing datasource (istio#28320) * Add grafana templating query for DS_PROMETHEUS and add missing datasource * make extension dashboard viewonly skip ingress test in multicluster (istio#28321) E2E test for trust domain alias client side secure naming. (istio#28206) * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * trust domain alias secure naming e2e test * add dynamic certs and test options * move under ca_custom_root test folder * fix host to address * update script * refactor based on comments * updated comments * add build constraints * lint fix * fixes based on comments Samples: use more common images and delete useless samples (istio#28215) Signed-off-by: Xiang Dai <[email protected]> Wait until ACK before sending additional pushes (istio#28261) * Wait until ACK before sending additional pushes Fixes: istio#25685 At large scale, Envoy suffers from overload of XDS pushes, and there is no backpressure in the system. Other control planes, such as any based on go-control-plane, outperform Istio in config update propogations under load as a result. This changes adds a backpressure mechanism to ensure we do not push more configs than Envoy can handle. By slowing down the pushes, the propogation time of new configurations actually increases. We do this by keeping note, but not sending, any push requests where that TypeUrl has an un-ACKed request in flight. When we get an ACK, if there is a pending push request we will immediately trigger it. This effectively means that in a high churn environment, each proxy will always have exactly 1 outstanding push per type, and when the ACK is recieved we will immediately send a new update. This PR is co-authored by Steve, who did a huge amount of work in developing this into the state it is today, as wel as finding and testing the problem. See istio#27563 for much of this work. Co-Authored-By: Steven Dake [email protected] * Refactor and cleanup tests * Add test istioctl: fix failure when passing flags to `go test` (istio#28332) add xds proxy metrics (istio#28267) * add xds proxy metrics Signed-off-by: Rama Chavali <[email protected]> * lint Signed-off-by: Rama Chavali <[email protected]> * fix description Signed-off-by: Rama Chavali <[email protected]> remove-from-mesh: skip system namespace to remove sidecar (istio#28187) * remove-from-mesh: skip system namespace to remove sidecar * check for -i Remove accidentally merged debug logs (istio#28331) update warning message for upgrading istio version (istio#28303) * update warning message for upgrading istio version * add use before istioctl analyze Update README.md (istio#28272) * Update README.md * Update README.md Xds proxy improve (istio#28307) * Prevent goroutine leak * Accelerate by splitting upstream request and response handling * fix lint fix manifestpath for verify install. (istio#28345) Fix ADSC race (istio#28342) * Fix ADSC race * fix * fix ut * Update pkg/istio-agent/local_xds_generator.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pkg/adsc/adsc_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Update pilot/pkg/xds/lds_test.go Co-authored-by: Shamsher Ansari <[email protected]> * Apply shamsher's suggestions from code review Co-authored-by: Shamsher Ansari <[email protected]> Co-authored-by: Shamsher Ansari <[email protected]> List Istio injectors (istio#27849) * Refactor and rename command * Code nits * Fix typo * Print message if no namespaces have injection * Code nits * Case where an injected namespace does not yet have pods * Code cleanup kube-inject: hide namespace flag in favour of istioNamespace (istio#28067) Automator: update proxy@master in istio/istio@master (istio#28355) Fix test race in FilterGatewayClusterConf (istio#28330) Example failure https://prow.istio.io/view/gs/istio-prow/logs/unit-tests_istio_postsubmit/4087 In generally, we need a better way to mutate feature flags in tests. Maybe a conditionally compiled mutex. Will open an issue to track Networking: Add scaffold of tunnel EDS builder (istio#28244) * add endpoint tunnel supportablity field Signed-off-by: Yuchen Dai <[email protected]> * edsbtswip Signed-off-by: Yuchen Dai <[email protected]> * fix import Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * EndpointsByNetworkFilter refactor Signed-off-by: Yuchen Dai <[email protected]> * fix pkg/pilot/ tests Signed-off-by: Yuchen Dai <[email protected]> * ep builder decide build out tunnel type Signed-off-by: Yuchen Dai <[email protected]> * add basic tunnel eds test Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix proxy metadata access Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> * fix endpoint COW, h2support bitfield bug Signed-off-by: Yuchen Dai <[email protected]> * make gen without fmt Signed-off-by: Yuchen Dai <[email protected]> * add errgo Signed-off-by: Yuchen Dai <[email protected]> * address comment Signed-off-by: Yuchen Dai <[email protected]> * fmt Signed-off-by: Yuchen Dai <[email protected]> * make gen Signed-off-by: Yuchen Dai <[email protected]> cni:fix order dependent test failures (istio#28349) The `interceptRuleMgrType` decalred in main.go as "iptables", it would be changed to "mock" in func resetGlobalTestVariables(). When run single test, it would be "iptables" and make test failed. Signed-off-by: Xiang Dai <[email protected]> fix uninstall test (istio#28335) * fix uninstall test * revert prow change * address comment * add logic to deduplicate Adding Route Specific RateLimiting Test Signed-off-by: gargnupur <[email protected]> remove debug info Signed-off-by: gargnupur <[email protected]> fix failure Signed-off-by: gargnupur <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
cla: yes
Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA.
release-notes-none
Indicates a PR that does not require release notes.
size/M
Denotes a PR that changes 30-99 lines, ignoring generated files.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.