-
Notifications
You must be signed in to change notification settings - Fork 33
Description
The way that a test run links to a test plan makes it so that changing the test plan configuration breaks the test result.
Adding a new test case makes it seem like the test run was not fully executed.
Removing a test case breaks the bar with the results compared to the list of test cases:
Plan
Fixing the bar would be rather easy, however the underlying problem is that changing a test plan has an effect on runs from the past. This itself should be fixed somehow.
An Idea would be to create a copy of a test plan configuration everytime a test run is started. We can have multiple test runs refer to the same copy of a test plan as long as it doesn't change.
We could also introduce versioning to a test plan, automatically creating a new version if the plan has already been used in an active test run.
Both options have the same effect and from my view only differ regarding the action that triggers them.