Skip to content

Add ability to always treat the test case as interesting initially #21

@dkasak

Description

@dkasak

The ability to specify that lithium should treat the test case as interesting initially (before any chunk removals are done, regardless of whether it is actually interesting or not) would be a nice addition.

It would allow lithium to act as a fuzzer, as explained in the Reducers are Fuzzers blog post.

Some use cases:

  • It often happens that I have a hunch that a certain part of the code is broken, but I'm unable to trigger the faulty behaviour with a test case. What sometimes works is writing a test case that exercises this part of the code but is uninteresting (e.g. doesn't crash) and then letting lithium crashes run while forcing it to treat the case as interesting.
  • A fix for a bug had been deployed but you have a hunch that the fix isn't a proper one and just fixes it for some particular test case. This would allow you to uncover potential related test cases that still trigger the bug.

I'm not sure how to implement this best in lithium, though. Any suggestions? For now, I've just hacked together a non-crashes interestingness test for my personal use. It behaves exactly as crashes, except it forces the test case to be interesting in the initial run. The downside of this approach is that you can't reuse this code with other interestingness tests for which you might want to turn this behaviour on.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions