Why we don't use one of existing plugins
- Test scenario creation (calling methods and creating arguments for them)
- Genetic algorithms, machine learning, …
- Selection of argument values (fuzzing)
- Black box fuzzing - random
- White box fuzzing - code based
- Gray box fuzzing - feedback based
Goal: test and compare different combinations of tools/algorithms from the first and second stages
Task: find an Intellij plugin for convenient work with algorithms of fuzzing and test generation
- Generates JUnit tests for the selected class
- 1000+ downloads on https://plugins.jetbrains.com
- Free
- Open-source (ability to modify source code)
- Active support (ability to modify source code without long refactoring or bug-fixing)
- Generation options (ability to choose different fuzzers and scenario generation tools)
It turns out that none of the observed plugins fits our goal (they work with concrete tools). Also, it seems more reasonable to implement a plugin from scratch because all we can reuse is running action, the rest of the functionality is different.
- access to a large amount of metadata making the generated tests very dynamic
- ability to decide whether or not to generate tests for all overloads of a method and how to name them
- no active support
- no documentation
- light
- no active support
- no documentation
- needs Maven?
- Java, Scala or Groovy test code with JUnit 4/5, TestNG, Spock or Specs2 frameworks
- auto-generate Mockito mocks
- generate test params and assertion statements
- generate relevant mocked return statements
- hard to work with only java part
- doesn’t merge generated test files if tests already exist
- not being maintained anymore
- needs annotations
- not free
- not an open-source
- just used to call EvoSuite from inside IntelliJ IDEA
- hard to build only plugin sources
- configurable test naming convention
- not an open-source