Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

testing: TestPullImageInactivityTimeout failing intermittently on windows #1864

Closed
adnxn opened this issue Feb 18, 2019 · 2 comments
Closed

testing: TestPullImageInactivityTimeout failing intermittently on windows #1864

adnxn opened this issue Feb 18, 2019 · 2 comments
Labels
kind/testing os/windows

Comments

@adnxn
Copy link
Contributor

adnxn commented Feb 18, 2019 •
edited
Loading

TestPullImageInactivityTimeout appears to be failing intermittently on windows. see below for logs, though they appear to be truncated.

2019-02-18T21:14:12Z [WARN] DockerGoClient: failed to pull image image: Could not transition to pullBegin; timed out after waiting 5m0s
2019-02-18T21:14:12Z [WARN] DockerGoClient: failed to pull image image: Could not transition to pullBegin; timed out after waiting 5m0s
2019-02-18T21:14:12Z [WARN] DockerGoClient: failed to pull image image: Could not transition to pullBegin; timed out after waiting 5m0s
2019-02-18T21:14:12Z [WARN] DockerGoClient: failed to pull image image: Could not transition to pullBegin; timed out after waiting 5m0s
2019-02-18T21:14:12Z [WARN] DockerGoClient: failed to pull image image: Could not transition to pullBegin; timed out after waiting 5m0s
2019-02-18T21:14:13Z [WARN] DockerGoClient: inactivity time exceeded timeout while pulling image image
2019-02-18T21:14:13Z [WARN] DockerGoClient: failed to pull image image: inactivity time exceeded timeout while pulling image
2019-02-18T21:14:13Z [WARN] DockerGoClient: inactivity time exceeded timeout while pulling image image
2019-02-18T21:14:13Z [WARN] DockerGoClient: failed to pull image image: inactivity time exceeded timeout while pulling image
2019-02-18T21:14:14Z [WARN] DockerGoClient: inactivity time exceeded timeout while pulling image image
2019-02-18T21:14:14Z [WARN] DockerGoClient: failed to pull image image: inactivity time exceeded timeout while pulling image
--- FAIL: TestPullImageInactivityTimeout (1.72s)
	docker_client_test.go:188: 
			Error Trace:	docker_client_test.go:188
			Error:      	An error is expected but got nil.
			Test:       	TestPullImageInactivityTimeout
			Messages:   	Expected error for pull inactivity timeout
	docker_client_test.go:119: missing call(s) to *mock_sdkclient.MockClient.ImagePull(is anything, is equal to image:latest, is anything) C:/Users/ADMINI~1/AppData/Local/Temp/amazon-ecs-agent/go/src/github.com/aws/amazon-ecs-agent/agent/dockerclient/dockerapi/docker_client_test.go:175
	docker_client_test.go:119: aborting test due to missing call(s)
2019-02-18T21:14:14Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-agent:latest
2019-02-18T21:14:14Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-pause:0.1.0
2019-02-18T21:14:14Z [WARN] Unable to fetch user data: blackholed
2019-02-18T21:14:14Z [CRITICAL] Unable to communicate with EC2 Metadata service to infer region: blackholed
2019-02-18T21:14:14Z [CRITICAL] Configuration key not set, key: AWSRegion
2019-02-18T21:14:14Z [WARN] DockerGoClient: failed to pull image registry.endpoint/myimage:tag: test error
2019-02-18T21:14:15Z [WARN] DockerGoClient: Error while pulling image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:15Z [WARN] DockerGoClient: failed to pull image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:15Z [WARN] DockerGoClient: Error while pulling image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:15Z [WARN] DockerGoClient: failed to pull image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:15Z [WARN] DockerGoClient: Error while pulling image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:15Z [WARN] DockerGoClient: failed to pull image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:15Z [WARN] DockerGoClient: Error while pulling image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:15Z [WARN] DockerGoClient: failed to pull image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:16Z [WARN] DockerGoClient: Error while pulling image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:16Z [WARN] DockerGoClient: failed to pull image image: toomanyrequests: Rate exceeded
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-agent:latest
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-pause:0.1.0
2019-02-18T21:14:16Z [WARN] Unable to fetch user data: blackholed
2019-02-18T21:14:16Z [CRITICAL] Unable to communicate with EC2 Metadata service to infer region: blackholed
2019-02-18T21:14:16Z [CRITICAL] Configuration key not set, key: AWSRegion
2019-02-18T21:14:16Z [INFO] DockerGoClient: error stopping container id: test error
2019-02-18T21:14:16Z [INFO] DockerGoClient: process within container 123 died due to OOM
2019-02-18T21:14:16Z [INFO] DockerGoClient: All events have been received
2019-02-18T21:14:16Z [ERROR] DockerGoClient: error while listening to Docker Events : some error happened
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-agent:latest
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-pause:0.1.0
2019-02-18T21:14:16Z [WARN] Unable to fetch user data: blackholed
2019-02-18T21:14:16Z [CRITICAL] Unable to communicate with EC2 Metadata service to infer region: blackholed
2019-02-18T21:14:16Z [CRITICAL] Configuration key not set, key: AWSRegion
2019-02-18T21:14:16Z [ERROR] DockerGoClient: Docker SDK client unable to ping Docker daemon. Ensure Docker is running: test error
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-agent:latest
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-pause:0.1.0
2019-02-18T21:14:16Z [WARN] Unable to fetch user data: blackholed
2019-02-18T21:14:16Z [CRITICAL] Unable to communicate with EC2 Metadata service to infer region: blackholed
2019-02-18T21:14:16Z [CRITICAL] Configuration key not set, key: AWSRegion
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-agent:latest
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-pause:0.1.0
2019-02-18T21:14:16Z [WARN] Unable to fetch user data: blackholed
2019-02-18T21:14:16Z [CRITICAL] Unable to communicate with EC2 Metadata service to infer region: blackholed
2019-02-18T21:14:16Z [CRITICAL] Configuration key not set, key: AWSRegion
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-agent:latest
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-pause:0.1.0
2019-02-18T21:14:16Z [WARN] Unable to fetch user data: blackholed
2019-02-18T21:14:16Z [CRITICAL] Unable to communicate with EC2 Metadata service to infer region: blackholed
2019-02-18T21:14:16Z [CRITICAL] Configuration key not set, key: AWSRegion
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-agent:latest
2019-02-18T21:14:16Z [INFO] Image excluded from cleanup: amazon/amazon-ecs-pause:0.1.0
2019-02-18T21:14:16Z [WARN] Unable to fetch user data: blackholed
2019-02-18T21:14:16Z [CRITICAL] Unable to communicate with EC2 Metadata service to infer region: blackholed
2019-02-18T21:14:16Z [CRITICAL] Configuration key not set, key: AWSRegion
2019-02-18T21:14:16Z [WARN] DockerGoClient: Unable to retrieve stats for container foo: test error
2019-02-18T21:14:16Z [WARN] DockerGoClient: Unable to decode stats for container foo: invalid character 's' looking for beginning of value
2019-02-18T21:14:18Z [WARN] DockerGoClient: inactivity time exceeded timeout while retrieving stats for container foo
2019-02-18T21:14:18Z [WARN] DockerGoClient: no container healthcheck data returned by Docker
FAIL
FAIL	github.com/aws/amazon-ecs-agent/agent/dockerclient/dockerapi	5.673s
@adnxn
Copy link
Contributor Author

adnxn commented Feb 18, 2019

observing failing test in #1849 with unrelated changes.

@adnxn adnxn mentioned this issue Feb 18, 2019
Merged
@fenxiong fenxiong mentioned this issue Feb 26, 2019
Merged
8 tasks
@adnxn adnxn removed the kind/bug label Feb 26, 2019
@suneyz suneyz mentioned this issue Feb 28, 2019
Merged
@adnxn
Copy link
Contributor Author

adnxn commented Mar 6, 2019

well - turns out this test never fails on travis. recently we updated the testing infrastructure to use gp2 volumes and this test is no longer failing intermittently. closing issue for now.

@adnxn adnxn closed this as completed Mar 6, 2019
@fenxiong fenxiong mentioned this issue Mar 6, 2019
Merged
@fenxiong fenxiong mentioned this issue Apr 4, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/testing os/windows
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@adnxn