-
Notifications
You must be signed in to change notification settings - Fork 27
Open
Description
The test framework aims to address the following issues:
- Parallelization : run integration tests in parallel
- Idempotent : can be run multiple times with the same inputs and environment, and it will always produce the same result—pass or fail—without side effects.
- Filterable : run isolated problematic test cases on demand
- Reproducible : when a test fails, can directly inspect resources in the remote environment
- Debug-able : can enter the debugger to isolate behavior
- Direct code coverage
- Data driven : Run the same tests with different inputs and expect the same behavioral result based on inputs
- Extensible : Drop-in adding of test cases to common setup
Some high level goals, that don't necessarily pertain to test-strategy
- Work-around existing code structure, without functionality & refactors to enable testability
- Allow for future refactors to improve stability, and better reproducibility of bugs