Proposal: Step definition inventory in message protocol

[aslakhellesoy]

Large projects have lots of step definitions. Knowing what step definitions are available can become a bit of. challenge for users. Tools like IDEA help, but it relies on sources to be available. Some teams package up reusable step definitions in modules, and these aren’t easily available.

If the message protocol supported a way to represent all available step definition expressions, this metadata could be embedded in modules (for example in cucumber-metadata.protobuf or cucumber-metadata.json). Other authoring and documentation tools could also take advantage of this metadata to provide a list of step definitions, autocomplete and validation.

See this discussion for more context.

[amp4045]

Some additional bits, a Use-case I was particularly interested in was the ability to very quickly search in my step inventory for all’s steps of a “type”. I think of a pivot table such that if I had say 100 step definitions and all I cared about were to see all my step definitions that handled ingesting data - the pivot table (or something similar) could easily handle filter and aggregation of filters.

If we crated this information in such a format tools like confluence (atlassian) could actually take our data and make the pivot table for us. This, I would think, would be worth some investigation. I sort of envisioned a JSON file created from a test run, which CI tools such as Jenkins could take the JSON from that build (such as a nightly build) and send it off to confluence. That way, users have the most up-to-date version of their step inventory.

[danascheider]

@aslakhellesoy I really, really like this idea as I've had this problem personally on my larger projects and haven't been sure how to solve it other than creating my own index of step defs and then trying to keep it updated every single time a step def is changed or added. @downright-development your idea is interesting as well and would likely be useful to many users, especially those in larger organisations and those where nontechnical users may want easy-reference information about the suite.

[aslakhellesoy]

None of the Cucumber implementations emit the message protocol yet, that’s slated for #415. It will take some time, but that’s the plan.

[christianhujer]

My 2 cents on JSON vs Protocol Buffers: I think it should be JSON, not Protocol Buffers, at least for the start. While Protocol Buffers are more efficient than JSON, they are binary, whereas JSON is human-readable, and especially in the beginning, that's a huge benefit of JSON over Protocol Buffers. And I think that in this context, the performance benefit of Protocol Buffers over JSON is probably too small to outweigh the usability benefits of JSON over Protocol Buffers.

(I'm not against Protocol Buffers in general, actually in my latest project, the REST endpoints support both via HTTP Content Negotiation.)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs.

[stale]

This issue has been automatically closed because of inactivity. You can support the Cucumber core team on opencollective.

[amp4045]

@aslakhellesoy Loved that you reopened this! What's the current plan on the protobuff issue? I've been away for some time.

[aslakhellesoy]

@downright-development a bunch of contributors are meeting in London on Wednesday to decide on the way forward with protobuf.

[davidjgoss]

@aslakhellesoy I’m interested in looking at this.

I had exactly the requirements you describe and ended up implementing a step library tool internally at work. It works by parsing the TypeScript code which is obviously not ideal eg we can’t use it with cucumber-jvm.

Would a good first step be a draft PR with a slimmed down port of the react component to cucumber-react, to then start discussion of what the messages would need?

[aslakhellesoy]

We have defined a StepDefinition message now, and fake-cucumber (which we use for testing) emits it. You can see them in the compatibility kit ndjson files, for example:

https://github.com/cucumber/cucumber/blob/63e4a0cc3763928557743fcf493ddafb0d578c0a/compatibility-kit/javascript/features/examples-tables/examples-tables.ndjson#L9

I’d love to build step definition search/autocomplete into @cucumber/react. I’d prefer that over a fork. Would you consider helping with that?

/cc @vincent-psarga

[aslakhellesoy]

Can we close this issue now? It’s about adding the message to the protocol, which is now done.

[davidjgoss]

@aslakhellesoy yes definitely, that’s what I meant, to add it there not fork. Sorry didn’t realise StepDefinition was already in messages - I will start having a look

[aslakhellesoy]

@tooky you might be interested in this too

[mpkorstanje]

Other authoring and documentation tools could also take advantage of this metadata to provide a list of step definitions, autocomplete and validation.

This probebly excludes IDE's. As that requires the glue code to be executed. So they're kinda tied to static analysis. But it would be nice if Cucumber had a component that could be given cucumber expressions and parameter types discovered by static analysis and would then tell you which steps would match.

(This was intended to add some nuance, but I'm starting to see some possibilities here).

Edit: Part of that is already done by Cucumber expressions but currently lacks the type information that an expression would produces.

[aslakhellesoy]

@mpkorstanje It would be possible for an IDE to cache the messages from the last execution (on disk) and use that for autocomplete. And then invalidate the cache if a step definition's expression is modified.

[mpkorstanje]

That'd be quite slow still. And because code is executed (world must be created) may have unintended side effects. The side effects makes it unusable for IDEs.

[mpkorstanje]

Resolved by #946