Fixed lengthy stacktrace when running cucumber -f stepdefs when steps aren't defined

[xtrasimplicity]

Summary

Currently, when running cucumber -f stepdefs against features which don't include step definitions, a lengthy stacktrace is shown to the user. This is thoroughly described in issue #1285.

Details

I've simply added a condition to prevent a NoMethodError from being raised, if a step hasn't been defined for a test case. When an error isn't raised, example snippets are printed to stdout, as shown in the following test case:

Scenario: Run with --format stepdefs when some steps are undefined
  Given a file named "features/calculator.feature" with:
  """
  Feature: Calculator
    Scenario: Adding numbers
      When I add 4 and 5
      Then I should get 9
  """
  When I run `cucumber -f stepdefs features/calculator.feature`
  Then it should pass with:
  """
  You can implement step definitions for undefined steps with these snippets:

  When("I add {int} and {int}") do |int, int2|
    pending # Write code here that turns the phrase above into concrete actions
  end

  Then("I should get {int}") do |int|
    pending # Write code here that turns the phrase above into concrete actions
  end
  """

Motivation and Context

As described in issue #1285, it can be difficult for new users to understand what went wrong, when they mis-type a step, or fail to define one, and are shown a 40 line stacktrace.

How Has This Been Tested?

I've added an additional feature to features/docs/formatters/usage_formatter.feature.

Types of changes

• Refactor (code change that does not change external functionality)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• I've added tests for my code
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

---

This change is 

[olleolleolle]

This is a helpful step to make people more welcome to the project.

[xtrasimplicity]

Thanks for reviewing these PRs, @olleolleolle! :)

[mbigras]

Nice @xtrasimplicity!

Question:

Do you think the exit status should be non-zero?

I think there shouldn't be a long stack trace but the exit status should be non-zero because all the steps weren't successfully run. Also, there are probably tools make decisions about what to do next based on cucumber's exit status. Exiting 0 for an undefined step might not be what folks expect.

[xtrasimplicity]

@mbigras, The fix I implemented changed the on_step_finished event, which I suspect may be called from other areas of the code. My changes won't have made any impact on the exit codes that cucumber returns.

Whether or not a test suite with undefined steps should yield a non-zero exit code is something that I would consider to be a breaking change.

I'm also not sure whether returning a successful exit code when undefined steps are present was the desired functionality, or whether it just turned out to be that way. @olleolleolle / @mvz, would you be able to provide some insight into this? Thanks!

[mvz]

I haven't been around long enough to know the original intent, but to me it makes sense that output format and exit code are orthogonal concerns, so the exit code with -f stepdeps should be the same as, say -f pretty or -f progress.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.