0.8.0 Beta

Hi!

This beta contains several new features. A central focus of the next major release is improving the coverage tool's branch detection.

Beginning with this version the following syntax is measured as a branch:

OR conditions

When a logical expression is composed with the || operator, both sides can be considered branches. To test the entire expression

if (a == 1 || a == 2)

... a must equal 1, 2 and neither of those values. (Thanks to Gnosis engineer @rmeissner for proposing this in #175)

Ternary Conditionals

Long ago, when Solidity was 0.4, solidity-coverage treated ternary conditionals like regular if/else statements. Some language improvements v0.5 subsequently made this impossible. Now it's back...

Modifier Invocations

Solidity-coverage already covers the code within modifier definitions. However, each modifier invocation at the function level should really be considered its own branch. Some of the most critical logic in Solidity contracts is handled this way (ex: onlyOwner and nonReentrant). Testing the pass/fail cases for each occurrence of these gates protects you from accidentally removing them during a refactor or emergency patch.

Because it's possible to write a modifier which performs a preparatory task and never reverts, there's a new option (modifierWhitelist) which allows you to exclude specific modifiers from branch measurement.

And if you don't like modifier invocation coverage you can turn it off by setting the option measureModifierCoverage to false.

(Many thanks to OpenZeppelin engineer @nventuro for proposing this improvement in #286 and helping to design it.)

Test Matrix

The hardhat and truffle plugins support a new cli flag: --matrix. (Short for "test matrix".)

This flag generates a JSON object that maps which tests in your suite hit which lines of code. (An example can be seen at docs/matrix.md. More info can be found in the advanced docs, here.)

This data is useful for many advanced testing applications - Consensys security researcher @JoranHonig has written two that are worth checking out.

• vertigo: a mutation testing framework
• tarantula: a fault localization tool

Coverage is often a core component of fuzzing and generative test strategies because it helps narrow the range of inputs required to traverse every path in the code. If you're working on applications like this and have ideas for how solidity-coverage might serve your ends, please feel free to open an issue.

What's Coming!

We're hoping to get a major release out by the beginning of February (2021). It will include two additional features you can run in CI to get better feedback about what's changing in your code from PR to PR.

• A unified diff of the public API changes in your contracts
• Natspec documentation coverage