Fix randomization seed #1329
Merged
Fix randomization seed #1329
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ghost
assigned 
deivid-rodriguez
force-pushed
the
fix_randomization_seed
branch
from
39dfe92 to
7df75c5
Compare
If you don't explicitly pass a seed, cucumber would inform it has run test with an incorrect seed, so it would be impossible to actually reproduce a failing run.
deivid-rodriguez
force-pushed
the
fix_randomization_seed
branch
from
7df75c5 to
73da89e
Compare
|
I rebased this PR that should be fully focused now and I added a changelog entry for it. |
luke-hill
approved these changes
Dec 7, 2018
|
It's the information about the running time, something like "0m1.234s", which obviously might differ for both runs. |
|
Ok, going in! Thanks for your review @luke-hill! |
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
While debugging some apparently order dependent failures on my cucumber scenarios, I noticed that it's currently impossible to reproduce order dependent failures, because the seed cucumber says it has used is not the seed it has really used.
Details
My changes make sure that the seed that cucumber uses to randomize the scenarios is the same that cucumber informs in the UI, so the same scenario order can actually be reproduced in subsequent runs.
In order to test this, I needed to use some stuff from newer versions of
aruba, so I upgraded the scenarios to use that first. Happy to extract that to a separate PR, of course.Motivation and Context
This change makes it possible to reproduce order dependent failures when running scenarios in random order. Previously, it was not working.
How Has This Been Tested?
I added a test that runs cucumber in random order and with summary format, then runs cucumber again with the seed the previous command informed, and then checks that both commands give the same output (excluding runtime).
Screenshots (if appropriate):
Types of changes
Checklist: