Is it possible to throw a build error in VS for a missing step definition?

[jacon-dev]

I've tried searching this but I couldn't get anywhere with it so I'm hoping a member of the community might know.

Where a step definition is clearly missing, is it possible to have visual studio throw a build error to say that there is a missing step definition? I've increased build verbosity but the ReqnRoll output remains the same. I've looked to enable tracing in ReqnRoll but I can't seem to find it.

If its not possible thats fine, but im curious how when we have hundreds of tests what better ways we can work to avoid missing renaming steps etc.?

Worth noting I know there is configuration for behaviour at runtime for missing steps, we have this configured to Error. But it would be great if we could have something similar for compile time too to prevent runtime issues where steps are missing.

Any help appreciated.

[clrudolphi]

There is nothing in the existing build process to provide what you are looking for. Steps are bound to Step Definitions dynamically at run-time.

Last month, over on Discord, there was discussion of using the DryRunInvoker from within a run-time plugin and use of that technique might provide a basis for an approach that could be incorporated into a build script (post compile but before actual test execution).

[jacon-dev]

Thanks @clrudolphi

How does visual studio currently recognise that there is a missing step? Obviously we can navigate to a feature file and the step that has a missing implementation and it’s clear there isn’t a corresponding implementation.

Could there be an opportunity to piggy back on whatever functionality exists there?

[clrudolphi]

The Reqnroll VisualStudio Extension has logic that examines all of the steps against the known binding methods (via reflection). It uses a mismatch to color the step text in the editor to indicate an undefined step.

The Reqnroll runtime uses much of the same logic to dynamically match up the steps with bindings.
Since both rely on reflection, these can't be used during generation/compilation as the assembly with the step binding methods may not have been compiled yet.

[clrudolphi]

If you're willing to give it a try, here is a start of how the DryRunInvoker might be used. When this is compiled into an assembly (with a name that ends in Reqnroll.Plugin) and taken as a dependency of your project, it will short-circuit execution of your steps and error out when a step is not defined. The testing framework (I used MsTest in a sample project on my own) marked the test as 'Skipped' because of the missing step definition, so it's not quite a 'failed build', but this is pretty close to what you wanted.

using DryRun.ReqnrollPlugin;
using Reqnroll.Bindings;
using Reqnroll.CommonModels;
using Reqnroll.Configuration;
using Reqnroll.EnvironmentAccess;
using Reqnroll.ErrorHandling;
using Reqnroll.Infrastructure;
using Reqnroll.Plugins;
using Reqnroll.Tracing;
using Reqnroll.UnitTestProvider;
using System;
using System.Diagnostics;
using System.Threading.Tasks;

[assembly: RuntimePlugin(typeof(DryRunPlugin))]
namespace DryRun.ReqnrollPlugin
{
    public class DryRunPlugin : IRuntimePlugin
    {
        public void Initialize(RuntimePluginEvents runtimePluginEvents, RuntimePluginParameters runtimePluginParameters, UnitTestProviderConfiguration unitTestProviderConfiguration)
        {
            runtimePluginEvents.CustomizeGlobalDependencies += RuntimePluginEvents_CustomizeGlobalDependencies;
        }

        private void RuntimePluginEvents_CustomizeGlobalDependencies(object sender, CustomizeGlobalDependenciesEventArgs e)
        {
            e.ObjectContainer.RegisterTypeAs<IAsyncBindingInvoker>(typeof(WrappedBindingInvoker));
        }
    }
    internal class WrappedBindingInvoker : IAsyncBindingInvoker
    {
        private const string ENV_VAR = "Reqnroll__DryRun";
        IAsyncBindingInvoker _invoker;

        public WrappedBindingInvoker(IEnvironmentWrapper environment, ReqnrollConfiguration config, IErrorProvider errorProvider, IBindingDelegateInvoker delegateInvoker)
        {
            var setting = environment.GetEnvironmentVariable(ENV_VAR);
            var dryRunEnabled = setting is Success<string> ? Convert.ToBoolean(((Success<string>)setting).Result) : false;
            _invoker = dryRunEnabled ? new DryRunBindingInvoker() as IAsyncBindingInvoker : new BindingInvoker(config, errorProvider, delegateInvoker);

        }
        public Task<object> InvokeBindingAsync(IBinding binding, IContextManager contextManager, object[] arguments, ITestTracer testTracer, DurationHolder durationHolder)
        {
            return _invoker.InvokeBindingAsync(binding, contextManager, arguments, testTracer, durationHolder);
        }
    }
}

[jacon-dev]

Might well give it a look into, thanks very much!