[Visual Studio] Unused Step Definitions

[lilwa29]

Hi !

We are using ReqNRoll.VisualStudio extension for our projects. We have thousands of feature files that need to be maintained, and sometimes it is hard to visualize which steps are not used anymore.

Could you please consider to add a way to view unused step definitions, which need to be cleanup?

Maybe a kind of Intellisense "highlight" like an analyzer would do (Example: https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1801).

Thanks!

[ajeckmans]

As a workaround. I usually run code coverage and then you'll get a nice overview as well. Any steps with 0% coverage are unused.

[gasparnagy]

@lilwa29 Yes, I also have this on my wish list for long time.

If anyone would like to try implementing this, I think the "Find step definition usages" command is a good start. That also goes through all steps of all feature files in order to find matches with the current step definition. This new command could do something similar, but just collect all matches and finally exclude that from the full set of step definitions. I'm happy to help if you get stuck.

[clrudolphi]

I will make an attempt at this.

My understanding is that this feature will need to find all Binding classes and from that all methods marked with a Step attribute (Given, When, Then, etc) and for each run the same algorithm as the existing 'Find step definition usages' command. Those methods that have no usages are those we want right?
The core of this new feature would be placed in to a new class in the Services sub-folder of the Editor folder of the Reqnroll.VisualStudio project.
We need a new Command in the Commands sub-folder.

I will try to get that far and then report back. May need some help later getting this wired up to the UI (menu items, dialog boxes, etc).

@gasparnagy - I see in the Reqnroll.VisualStudio solution that there appears to be functionality that duplicates or is very similar to code already in Reqnroll (eg, code to parse feature files, code to match Steps with StepDefinitions, etc.) Why was this approach taken? (versus having the VS extension call in to the Reqnroll functionality directly). I'm curious about the rationale/reasoning that went in to this design.

[clrudolphi]

Proof of Concept PR submitted: here

I think this is close but not quite ready. I suspect it will report multiple "unused" for a single Step method for each Given/When/Then Attribute applied to that Method (when those attributes don't match with feature steps). These should probably be deduplicated.

[gasparnagy]

@clrudolphi Notes on the high level VSext concept:

• The VSext will need to work with multiple different Reqnroll (and SpecFlow) versions and also it was designed to allow runtime plugins to dynamically register step definitions or change the step definition behavior (e.g. the cucumber expression support was a SpecFlow plugin).
• Because of that there is no point in referencing a single Reqnroll runtime library, because we anyway need to handle multiple versions. Linking a single runtime version have caused problems with our own Reqnroll tests, so finally the decision was to NOT reference Reqnroll/SpecFlow, but try to delegate as much task to them as possible, and copy those few helper methods as code that is required.
• The delegation is done through the "connector" infrastructure. Basically there are standalone .NET console apps that are linked or specialized to a particular SpecFlow/Reqnroll version range and able to print out the discovered step definitions as JSON from a particular step assembly.
• Whenever a user builds the Reqnroll project, the VSext will invoke the appropriate connector executable as a separate process, get the step definitions as JSON and load them to a service called "binding registry". These step definitions always have a regex (even if the user originally used cucumber expression), so the matching is simple, just have to apply the regex to the step plus a few extra checks.
• Because of the special needs (e.g. finding the matching step of a scenario outline is not necessarily a single match, but also finding the matching step definition of a background step), the matching logic that is implemented in Reqnroll is anyway not directly usable here.
• For this feature (I haven't checked your PR yet), you don't need to find the step definitions, as they are already available in the binding registry and the matching logic is also already there, so I think you might not need to add anything to the services folder, but I might be wrong with that.
• Parsing the feature files is mainly done by the Gherkin library anyway, Reqnroll just adds a very thin wrapper around that. The VSext also uses the same Gherkin library, but has its own wrapper (similar to the one of Reqnroll), because again the needs are slightly different (e.g. we need to collect errors and try to continue the parsing after a parsing error). So this is why it looks to have a "copy", but at the end it is not.
• In the VSext I tried to be strict on allowing testability with BDD scenarios for all features and there is a lot of magic & VS stubs have been implemented to make that possible, but at the end it works pretty well, so that should be also used for this feature as well.

This is what came to my mind now, I will also check the PR and make more comments. If anything I wrote is unclear, please let me know.