test: deflake the test start_containers_in_parallel
#748
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The test often fails, especially in GitHub actions. The test attempts to start four containers on the same image and sets a deadline to prove that they start in parallel rather than in serial. The failure messages indicate tha this deadline is exceeded. This appears to happen when the image of the container being started has to be pulled. Sometimes the pull operation takes so much time that the deadline is exceeded just due to that. This adds some lines to the test which start and then stop a container on the same image before attempting to start the containers in parallel. Doing so ensures that the image has already been pulled when the real test starts running. Then the containers should start in close to the intended two seconds, so the test should pass more reliably. Manual experiments appear to support this thesis: Before this change, running ```sh $ docker image rm hello-world $ cargo test --test async_runner --no-default-features --features blocking start_containers_in_parallel ``` would consistently fail due to an exceeded deadline. With this change, the test consistently passes. Note: This and other tests still depend on access to the Docker Hub and fail when they cannot pull the images they need. This can also lead to flakiness.
✅ Deploy Preview for testcontainers-rust ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
DDtKey
reviewed
Oct 20, 2024
|
Thank you for the contribution! 👍 I've left just one comment to pull the image in more appropriate way. |
DDtKey
changed the title
start_containers_in_parallel
DDtKey
approved these changes
Oct 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The test often fails, especially in GitHub actions. The test attempts to start four containers on the same image and sets a deadline to prove that they start in parallel rather than in serial. The failure messages indicate tha this deadline is exceeded.
This appears to happen when the image of the container being started has to be pulled. Sometimes the pull operation takes so much time that the deadline is exceeded just due to that.
This adds some lines to the test which start and then stop a container on the same image before attempting to start the containers in parallel. Doing so ensures that the image has already been pulled when the real test starts running. Then the containers should start in close to the intended two seconds, so the test should pass more reliably.
Manual experiments appear to support this thesis: Before this change, running
$ docker image rm hello-world $ cargo test --test async_runner --no-default-features --features blocking start_containers_in_parallelwould consistently fail due to an exceeded deadline. With this change, the test consistently passes.
Note: This and other tests still depend on access to the Docker Hub and fail when they cannot pull the images they need. This can also lead to flakiness.