cucumber --fail-fast switch

[danascheider]

Any interest in adding a --fail-fast switch like RSpec has, that would cause Cucumber to exit after the current scenario if the scenario fails? I've thought on a number of occasions that this functionality would be useful, so unless somebody has deliberately decided not to include it, I'd be glad to make a PR.

[mattwynne]

Hi @danascheider - thanks for the offer that sounds like a great idea. Please make sure you add an acceptance test in features/docs that explains how it works as well as unit tests for the underlying code.

If you need any help with the implementation just give us a shout.

[danascheider]

Will do, thanks @mattwynne.

[danascheider]

@mattwynne - Am I correct in concluding that the CLI doesn't have access to the report object while the tests are running? I've been trying to have the runner check the report for failures while the suite is running, without much success. At first blush, it looks like the cucumber-ruby-core would have to be modified to allow that to happen, but intuitively that doesn't seem like a good idea, and I feel like the core and the CLI are better separated than that. Off the top of your head, is there something I'm missing, or would changing core be a valid implementation?

[mattwynne]

Good question! The report is constructed here and we use the Fanout class to proxy the method calls to the report and forward them on to a collection of other reports. I think you could use that mechanism to add in a FailFastReport that would watch for any failing test steps and maybe throw an exception that we can trap in the CLI here.

Does that make any sense? This code is still in a bit of a transitionary state as we tidy up post 2.0, so sorry it's not easier to navigate around.

[danascheider]

Thanks! I've actually found the code pretty easy to work with for the most part - it was just that one thing I was unclear on.

[danascheider]

@mattwynne - Would you suggest giving the aforementioned FailFastReport its own output stream? I was trying to avoid that because I wanted to make sure it played nice with the other formatters, but currently the way I'm handling the error it raises is forcing Cucumber to exit before they have a chance to print their output. I'll give it its own output stream if you think that would be a good idea, but I just wanted a sanity check. Sorry to draw this out, I've been really busy!

[mattwynne]

Don't even mention it @danascheider - we're all busy and any contribution is very much appreciated.

It occurs to me that you could try setting the global Cucumber.wants_to_quit flag, which might be a way to trigger the graceful shutdown you'd like.

If that doesn't work, show me what you mean with the output streams - I'm not quite following you at the moment.

[danascheider]

I did try that, and the result was that it exited immediately without further output. My basic question was, when I run the tests normally using bundle exec cucumber, the output (after the dots) looks like this:

(::) failed steps (::)

Exit status was 1 but expected it to be 0. Output:
  | feature          | "Funcionalidade", "Característica", "Caracteristica"                                         |
  | background       | "Contexto", "Cenário de Fundo", "Cenario de Fundo", "Fundo"                                  |
  | scenario         | "Cenário", "Cenario"                                                                         |
  | scenario_outline | "Esquema do Cenário", "Esquema do Cenario", "Delineação do Cenário", "Delineacao do Cenario" |
  | examples         | "Exemplos", "Cenários", "Cenarios"                                                           |
  | given            | "* ", "Dado ", "Dada ", "Dados ", "Dadas "                                                   |
  | when             | "* ", "Quando "                                                                              |
  | then             | "* ", "Então ", "Entao "                                                                     |
  | and              | "* ", "E "                                                                                   |
  | but              | "* ", "Mas "                                                                                 |
  | given (code)     | "Dado", "Dada", "Dados", "Dadas"                                                             |
  | when (code)      | "Quando"                                                                                     |
  | then (code)      | "Então", "Entao"                                                                             |
  | and (code)       | "E"                                                                                          |
  | but (code)       | "Mas"                                                                                        |

(RSpec::Expectations::ExpectationNotMetError)

features/docs/gherkin/language_help.feature:15:in `Then it should pass with:'
Failing Scenarios:

cucumber features/docs/gherkin/language_help.feature:13 # Scenario: Get help for Portuguese language
206 scenarios (1 failed, 205 passed)

1211 steps (1 failed, 1210 passed)

0m50.034s

On the other hand, when I run bundle exec cucumber --fail-fast, the final output looks like this (including the last of the dots):

.......................................Fdana@x220:~/Development/cucumber-ruby$

I'm trying to make the suite exit after the failing scenario, but still include the output at the end - kind of like what happens when you CTRL+C out of it. I suppose I could just send the INT signal from the rescue block - the only reason I hadn't been doing that was because it seemed like exit status 1 would be more appropriate than 2. That would probably be the easiest way to implement this, so if you don't think the exit code is a big deal I'll just do that. Otherwise I'll see what else I can come up with.

[mattwynne]

Hi @danascheider - sorry for the lag.

Can you put what you have in a PR and I can have a look at it?

[danascheider]

My turn to apologize for the lag, @mattwynne. I was about to make the PR when I noticed I had screwed something up in a major way, so I'll make the PR as soon as I fix it. Should be pretty quick.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.