Use Gherkin3 instead of Gherkin2

[brasmusson]

This PR make Cucumber-Ruby-Core use (the parser of) Gherkin3 instead of Gherkin2. Some notes:

• The one file apart from the parser that was used from Gherkin2 (tag_expression.rb), is copied into Cucumber::Core:Gherkin to remove all dependencies to Gherkin2.
• To be able to use the Gherkin3 parser, a hook method is needed in the parser to be able to subclass the Gherkin3::AstBuilder (the integrate-ruby-core branch)
• The Cucumber-Ruby-Core AST classes are not created directly from the parser, instead builder are still created by Cucumber::Core::Gherkin:AstBuilder which delays the creation of the AST classes until the whole feature file has been parsed. The main reason to do this is to be able to set the language for the Step and ExamplesTableRow AST classes on creation (so that the AST classes can be immutable). The Step AST class needs the language to be able to calculate the actual keyword for the snippets ("And", "But" and "*" are converted to the appropriate of "Given", "When" or "Then"). The ExamplesTableRow AST class needs the language to be able to supply the "Scenario" keyword in the --expand mode.
• The Gherkin3 parser associates all comments (wherever they occur in the file) with the Feature AST node. To lessen the impact in Cucumber (particular the Pretty and HTML formatter), the comments are distributed down the AST tree at creation, so that the comment appear in the same Cucumber-Ruby-Core AST class as when using Gherkin2 (to be able to do this is another reason to use builders to delay the creation of the AST classes).

[mattwynne]

Woo! Do you see any reason this can't go into production?

[brasmusson]

I do not really see any reason that this can't go into production. The three things to do to get it there are:

• Release a Gherkin3 gem (with the change in the branch integrate-ruby-core), now Gherkin3 use the name 'gherkin3' that could be v0.1.0 or something.
• Release a new version of Cucumber-Ruby-Core
• Release a new version of Cucumber-Ruby (with Use Gherkin3 instead of Gherkin2 cucumber-ruby#884)

The defensive approach would be to first release a Cucumber-Ruby v2.0.1 with what already is on the master of Cucumber-Ruby and Cucumber-Ruby-Core, and then (at the same time almost) release a Cucumber-Ruby v2.1.0 using Gherkin3. Then no unforeseen issue with the Gherkin3 integration would block anyone to use the changes already made after Cucumber-Ruby v2.0.0.

[tooky]

@mattwynne @brasmusson I guess we should probably benchmark it with a large set of features before we release it. We are moving from a C parser to ruby parser...

History: https://groups.google.com/forum/#!topic/cukes/usj_xgK0aKg

[mattwynne]

Do you have release karma for cucumber-ruby and cucumber-ruby-core yet @brasmusson? Want ‘it?

[brasmusson]

@mattwynne No, I do not have release karma for cucumber-ruby and cucumber-ruby-core. Sure, why not? You can give it to me.

[mattwynne]

On 7 Jul 2015, at 16:34, Björn Rasmusson [email protected] wrote:
@mattwynne https://github.com/mattwynne No, I do not have release karma for cucumber-ruby and cucumber-ruby-core. Sure, why not? You can give it to me.

Great! Please see https://github.com/cucumber/cucumber-ruby/blob/master/CONTRIBUTING.md#gaining-release-karma https://github.com/cucumber/cucumber-ruby/blob/master/CONTRIBUTING.md#gaining-release-karma

[brasmusson]

I used the parser_spec.rb to create time measurements for the parsing Gherkin files to Cucumber-Ruby-Core AST:s. Simply put one feature file with (for details see 238d43b):

• 1 four step Backgound
• n four step Scenarios
• n three step Scenario Outlines each with
  • 2 Examples tables with two body rows each

(and also with some comments, descriptions, and tags).

With Ruby 2.1 on my (not so modern/fast) Linux box the results were:

n	Gherkin3 (seconds)	Gherkin2 (seconds)
100	1.1	0.4
200	3.2	0.9
400	10.7	1.9

With Ruby 2.1 and Gherkin3 on Travis the results were:

n	time (seconds)
100	0.7
200	2.1
400	5.9

[aslakhellesoy]

So it's 3-4 times slower than Gherkin2. Not too shabby considering this is pure ruby competing against C. Not worth optimising IMO - this should be fast enough.

[mattwynne]

It doesn't make for a great headline, but I agree that parse time is rarely the bottleneck for someone's feature suite.

Presumably, if this does start to look like a problem for our users, we can use Berp to generate a C-based parser for Ruby to use?

[aslakhellesoy]

The bottleneck is most likely in string scanning in the lexer, so it would have to be hand-written in C. Berp only generates the parser, not the lexer. Doable for sure if we need it.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.