Reqnroll
Long delays in VSTest startup after converting from SpecFlow #618
Answered
by
obligaron
Gustavusss
asked this question in
SpecFlow migration Q&A
-
Reqnroll Version2.4.1 Which test runner are you using?NUnit Test Runner Version Number4.3.2 .NET Implementation.NET 8.0 Test Execution MethodCommand line – PLEASE SPECIFY THE FULL COMMAND LINE Content of reqnroll.json configuration fileNone Issue DescriptionWe recently converted our BDD project from SpecFlow 3.9.74 to ReqnRoll 2.4.1. After the conversion we can run tests fine on our Windows development machines or on our Linux VMs, but when running in containers on our kubernetes cluster we're getting undesired behavior. We run the tests with "dotnet test --no-build" and are seeing very long startup times - its taking about 90 seconds from when we issue the command to it hitting our code bound to BeforeScenario. After a bunch of troubleshooting including trying different versions of ReqnRoll back to 1.0.0, we set up the ReqnRollQuickstart project to run in our container, running through the documentation to the point that the "Client has a simple basket" scenario runs. This scenario run in our container is seeing execution times of around 90 seconds (run locally as a checkout its below 1 second). Running unit tests via "dotnet test" in our unit test project that doesn't use ReqnRoll sees very fast execution times. Running with dotnet test --diag, we see a long period in both the vstest log and testhost log where the two processes are sitting in a polling state, presumably waiting for something to happen. I don't understand this area well enough to know what they are waiting for. I will be copying the logs in as comments as trying to attach them failed. (ed- Posted, they're ugly in GitHub but I tried copy-pasting them back into a text editor and they are readable there) It does appear that the issue started before the ReqnRoll fork - as part of testing we set ourselves up to use SpecFlow 3.10.2-beta, and it has the same long delay startup that we're experiencing with ReqnRoll and were not seeing with SpecFlow 3.9.74. Its possible we're encountering an issue the community has seen before but whose paper trail was lost when the SpecFlow repos+issues were deleted. Steps to ReproduceUnfortunately don't know what part of our environment is not meshing well with ReqnRoll, for us to reproduce its just to run "dotnet test" on our BDD project running in a kubernetes container Link to Repro ProjectNo response |
Beta Was this translation helpful? Give feedback.
Answered by
obligaron
May 21, 2025
Replies: 6 comments 3 replies
-
|
vstest log: |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
testhost log: |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Could you please share all the used packages and versions? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Certainly! I'll provide the full listing from the ReqnrollQuickstart project below. For our main app, its a long package list, but notable differences would be that it is using NUnit 4.3.2 and ReqnRoll 2.4.1. Since both are seeing the issue I'm giving the listing for the simple quickstart project: Project 'ReqnrollQuickstart.Specs' has the following package references
Transitive Package Resolved
|
Beta Was this translation helpful? Give feedback.
2 replies
-
|
Nice that there is a solution. But I'm confused, you say Nunit but it uses the mstest packages? |
Beta Was this translation helpful? Give feedback.
-
|
I was running two different tests, one using our real application, which uses NUnit, and one using the https://github.com/reqnroll/Quickstart project, which uses MStest. The package list is for the Quickstart project. |
Beta Was this translation helpful? Give feedback.
-
|
Could you try turning telemetry off? |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
304NotModified
-
|
!!! That fixed it! Thank you so much. Is there any way to turn off telemetry from within the project w/o relying on an EV (configuration file or anything)? The EV would cover everything but a few edge cases, so we're back up and running either way, just curious |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Currently, this is only possible by setting the environment variable. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Could you try turning telemetry off?
This can be done by setting the environment variable
REQNROLL_TELEMETRY_ENABLEDto0.