JSON formatter shows incorrect line number for steps in a Scenario Outline

[sjieg]

Configuration

• Cucumber: 2.* (Confirmed issue on 2.4.0 & 2.0.0)
• Gherkin: 4.1.1

Issue not present on Cucumber 1.3.20, that's why I assume it came with version 2.
Upon further inspection, it seems the introduction of Cucumber Core caused this issue

Summary

The line numbers given to a step in the json formatter is always the line number of the Example row. Because of this, it's not possible to distinguish the steps from each other, as the line number is their unique value.

Expected Behavior

In Cucumber 1.3.20 the correct JSON output is given, I've cut out the important pieces:

Feature: This is line 1

  Scenario Outline: This is line 3
    Given this step is line 4
    When when this step uses "<outline-item>" this should be line 5
    Then this is line 6
    Examples:
      | outline-item   |
      | example-line-9 |

cucumber --f json >> output.json

[
  {
    "keyword": "Feature",
    "name": "This is line 1",
    "line": 1,
    "elements": [
      {
        "keyword": "Scenario Outline",
        "name": "This is line 3",
        "line": 9,
        "steps": [
          {
            "keyword": "Given ",
            "name": "this step is line 4",
            "line": 4 # <---
          },
          {
            "keyword": "When ",
            "name": "when this step uses \"example-line-9\" this should be line 5",
            "line": 5 # <---
          },
          {
            "keyword": "Then ",
            "name": "this is line 6",
            "line": 6 # <---
          }
        ]
      }
    ]
  }
]

Note how the step line numbers are the same as the actual line number of the step.

Current Behavior

In cucumber 2.4.0 the following JSON output is given:

[
  {
    "uri": "features/outline_test.feature",
    "id": "this-is-line-1",
    "keyword": "Feature",
    "name": "This is line 1",
    "description": "",
    "line": 1,
    "elements": [
      {
        "id": "this-is-line-1;this-is-line-3;;2",
        "keyword": "Scenario Outline",
        "name": "This is line 3",
        "description": "",
        "line": 9, # <--- This is correct, it refers to the line number of the example
        "type": "scenario",
        "steps": [
          {
            "keyword": "Given ",
            "name": "this step is line 4",
            "line": 9 # <--- Should be line 4
          },
          {
            "keyword": "When ",
            "name": "when this step uses \"example-line-9\" this should be line 5",
            "line": 9 # <--- Should be line 5
          },
          {
            "keyword": "Then ",
            "name": "this is line 6",
            "line": 9 # <--- Should be line 6
          }
        ],
      }
    ]
  }
]

Context & Motivation

This issue has affected us at spriteCloud in the following way:

• We run Cucumber and screenshots are saved on the server it ran on.
• Then the screenshots are exported to the results database
• Before exporting, the file is linked to the step where the screenshot was taken
  This is going fine with regular Scenario's, but when it is an Outline we're not able to link the screenshot to the step and thus we can't visualize the given error.

Your Environment

• Version used: Cucumber 2.4.0
• Operating System and version: Windows 10, Ubuntu 16 and other linux systems.
• Link to your project: http://calliope.pro

[sjieg]

I've looked into this issue a bit further and I think I have pinpointed the location of the issue.
In Cucumber Core /core/compiler.rb:133

        def steps(row)
          outline_steps.map { |s| s.to_step(row) }
        end

Before this step outlined_steps contains the line number for every step.
After this step, all the line numbers are overwritten by the to_step(row) function.

I'll continue looking for a possible fix tomorrow. Until then, any feedback is welcome!

[sjieg]

A fix has been committed to the cucumber-ruby-core project: cucumber/cucumber-ruby-core#129 and the issue will be handled there: cucumber/cucumber-ruby-core#128

[brasmusson]

I reopen this issue because issues with the Json Formatter, which indeed is part of this project, should be raised here and not in the cucumber-ruby-core project.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs.

[danascheider]

@brasmusson Do I understand correctly that this issue does not apply to Cucumber 3.x?

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.