Background not executed for Scenario Outline defined after Scenario in the same Feature file.

[mgrebenets]

I have a feature file with both Scenario and Scenario outline.
For example

Feature: My Feature
  As a user I want...

Background:
  Given the app has launched
  I wait for landing page to appear

Scenario: My Scenario
  Given I am on the landing page
  I do some thing

Scenario Outline: My Scenario Outline
  Given I am on the landing page
  I do <some thing>

Examples:
  | some thing |
  | the thing |

So when I run all scenarios from this feature file, I have noticed that the Background is executed for "My Scenario", but not executed for "My Scenario Outline".
The "Given I am on the landing page" step fails for the outline.

If I swap scenario and scenario outline (put scenario outline first in the file), then this problem does not occur.

Is it a known limitation?
Should I always put outlines before scenarios?
Or is it a bug?

[mgrebenets]

Is there a command line switch of way to specify format in cucumber.yml to see the complete execution log?

[mattwynne]

There are some known issues with this which will be fixed with the 2.0 release. Thanks for letting us know and please bear with us while we work on the problem.

[mgrebenets]

Thanks, I'll take that into account and follow the rule of thumb so far, which is "outlines before scenarios".

[siannopollo]

I also experienced this issue recently. From what I observed, this only happened to the first Scenario Outline to occur after a Scenario if the first thing run was a Scenario. If a Scenario Outline was the first scenario in the file then this didn't happen to subsequent Scenario Outlines preceded by a Scenario.

[MikeAndSal]

We've also just stumbled across this issue. Using a very simple test feature file, it appears that the background steps for the first example of the first scenario outline in the feature file is called before anything else. The attached screenshot will hopefully show the issue. For the scenario outline "Outline Test", the first example "one" does not display the background debug text (while all the other outline scenarios and examples do). It appears that this background step is executed long before it gets anywhere near the scenario outline.

[mikefarah]

Also came across this issue, I've downgraded to the 1.2.5 release until this is fixed :(

[carthoris]

or just make everything an outline. That works too.

[mattwynne]

This should be fixed on master now. If someone could test that I'd be very grateful.

[9ikhan]

I tested this. If scenario outline follows a scenario in the feature file then, background is not run for the first example.

[tooky]

@9ikhan do you have time to create a pull request against master with a failing scenario that demonstrates this for us please?

[mattwynne]

@9ikhan you reproduced this with the master branch (not the current release) of Cucumber?

[9ikhan]

@mattwynne I tested this against latest cucumber release (1.3.10). I will try to clone the master to local and test this scenario.

[subakva]

I just ran into this issue with ruby 2.0 and cucumber 1.3.10. Moving Scenario Outlines to the top of the feature files solved the problem. Oddly, these features all worked flawlessly on our development machines (OS X), and failed consistently on our Jenkins machine (RedHat Linux).

[nickdengler]

It also looks like if you have multiple Scenario Outlines in one feature, the background will only run before the first one.

[mattwynne]

Please do not leave further feedback about reproducing this bug on 1.3.x

We believe it's fixed (along with a load of other stuff) on the master branch which will be released as 2.0. I'd really appreciate it if someone could test this manually on the master branch, or send us a PR with a test.

[seaman1]

I've reproduced this issue on 1.3.14 (ruby 2.1).
In my case it influences first scenario in feature (which is not Scenario Outline). Scenario Outline is located further (on 3-rd or 4-th position).
Issue solved by putting Scenario Outline as first scenario in feature. This also solves the problem if you have multiple Scenario Outline's in feature - just put them to the top.

[mattwynne]

@seaman1 please could you try with the master branch of cucumber? I think it’s gone but I’d love confirmation from someone who’s reproduced this.

[mattwynne]

Pleast re-test with cucumber 2.0.0.beta.1 which was released on Friday. If the bug occurs, repoen this ticket.

[jorlebarmend]

I am requesting to re-open this topic because I have recently had the same thing happen - though I cannot discover the root cause.

Scenario outlines are failing when placed after a scenario in our tests. This is happening on a development laptop (Mac OSX) and also on a staging Jenkins machine (running Linux).

I came upon this thread after trying several different things to solve this issue. It happened to be that only up until last week this was not an issue, and then tests started to fail consistently in the exact same place.

I tried the suggestion above of moving the Scenario outline to be the first test after the background and this has solved the issue, however I would like to know if there's a way to discover what the toot cause is.

Is there any information I could provide that might help diagnose this situation.

Also I would lie to point out that this is not happening on another development machine, nor on our production/live Jenkins, even when running the same code.

Please note we are using cucumber 1.3.15 as the reporting gem we use is dependant on this and cannot utilised a higher version - but we did not have any issues with this until last week -

[mattwynne]

@jorlebarmend thanks for reporting this.

There are some significant internal changes in version 2.0 of Cucumber that have made lots of bugs like this go away. I'm afraid we won't be making any effort to fix them in the 1.3.x codebase.

Instead, I'd prefer to help get you upgraded. Which reporting gem are you using? Do you know why it's not compatible with 2.0?

[jorlebarmend]

@mattwynne the information I have at the moment is that we are using the Json expanded formatter, which we need for the Jenkins cucumber plugin. As far as I know this has not been updated to work with a newer version of cucumber.

[brasmusson]

@jorlebarmend Are you talking about the producing json files for the use of MasterThought Cucumber-Reporting tools using the --expand option? Which was implemented in v1.3.17 by #700? That functionality is also available in v2.0.0.rc.X.

[jorlebarmend]

@brasmusson, unfortunately not bras - we are using the expanded formatter ruby file from this gist

https://gist.github.com/blt04/9866357

[brasmusson]

@jorlebarmend Well, since both that formatter and the #700 changes to Cucumbers own json formatter aims at solving the same issue, to produce json files that can be used by the MasterThought Cucumber-Reporting tools - also when using Scenario Outlines, you definitely have the options to upgrade to Cucumber v2.0.0.rc.X (or v1.3.17-19) and use Cucumbers own json formatter to produce json files for you. The limitation you get is that you need to use the --expand option when producing those json files. In return you get all the improvements made in the v2.0.0 track.

[sgran]

I can confirm that we had the issue with both 1.3.5 and 1.3.10 on linux. An upgrade to 2.4.0 on linux seems to have resolved it for us.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.