Merge branch 'dev' into real_time_benchmarks

Author: domenukk

.gitmodules

@@ -1,3 +0,0 @@
-[submodule "fuzzers/qemufuzzer/qemu-fuzz"]
-	path = fuzzers/qemufuzzer/qemu-fuzz
-	url = git@github.com:AFLplusplus/qemu-fuzz.git

Cargo.toml

@@ -16,5 +16,5 @@ exclude = [
     "fuzzers/libfuzzer_stb_image",
     "fuzzers/libfuzzer_libmozjpeg",
     "fuzzers/frida_libpng",
-    "fuzzers/qemu_user",
+    "fuzzers/baby_fuzzer",
 ]

README.md

@@ -4,7 +4,13 @@ Advanced Fuzzing Library - Slot your own fuzzers together and extend their featu
 
 LibAFL is written and maintained by Andrea Fioraldi <andreafioraldi@gmail.com> and Dominik Maier <mail@dmnk.co>.
 
-It is released as Open Source Software under the [Apache v2](LICENSE-APACHE) or [MIT](LICENSE-MIT) licenses.
+## What
+
+LibAFL is a collection of reusable pieces of fuzzers, written in Rust.
+
+It offers a main crate that provide building blocks for custom fuzzers, [libafl](./libafl), a library containing common code that can be used for targets instrumentation, [libafl_targets](./libafl_targets), and a library providing facilities to wrap compilers, [libafl_cc](./libafl_cc).
+
+LibAFL is fast, multi-platform, no_std compatible, and scales over cores (and machines in the near future!).
 
 ## Getting started
 
@@ -20,29 +26,50 @@ Build the library using
 cargo build --release
 ```
 
-Build the documentation with
+Build the API documentation with
 
 ```
 cargo doc
 ```
 
-We collect example fuzzers in `./fuzzers`. They can be build using `cargo build --example [fuzzer_name] --release`.
+Browse the LibAFL book with (requires [mdbook](https://github.com/rust-lang/mdBook))
+
+```
+cd docs && mdbook serve
+```
+
+We collect example fuzzers in [`./fuzzers`](./fuzzers/).
 
-The best-tested fuzzer is `./fuzzers/libfuzzer_libpng`, a clone of libfuzzer using libafl for a libpng harness.
-See its readme [here](./fuzzers/libfuzzer_libpng/README.md).
+The best-tested fuzzer is [`./fuzzers/libfuzzer_libpng`](./fuzzers/libfuzzer_libpng), a multicore libfuzzer-like fuzzer using LibAFL for a libpng harness.
 
-## The Core Concepts
+## Resources
 
-The entire library is based on some core concepts that we think can generalize Fuzz Testing.
++ [Installation guide](./docs/src/getting_started/setup.md)
 
-We're still working on extending the documentation.
++ Our RC3 [talk](http://www.youtube.com/watch?v=3RWkT1Q5IV0 "Fuzzers Like LEGO") explaining the core concepts
 
-In the meantime, you can watch the Video from last year's RC3, here:
++ [Online API documentation](https://docs.rs/libafl/)
 
-[![Video explaining libAFL's core concepts](http://img.youtube.com/vi/3RWkT1Q5IV0/3.jpg)](http://www.youtube.com/watch?v=3RWkT1Q5IV0 "Fuzzers Like LEGO")
++ The LibAFL book (very WIP) [online](https://aflplus.plus/libafl-book) or in the [repo](./docs/src/)
 
 ## Contributing
 
 Check the [TODO.md](./TODO.md) file for features that we plan to support.
 
 For bugs, feel free to open issues or contact us directly. Thank you for your support. <3
+
+#### License
+
+<sup>
+Licensed under either of <a href="LICENSE-APACHE">Apache License, Version
+2.0</a> or <a href="LICENSE-MIT">MIT license</a> at your option.
+</sup>
+
+<br>
+
+<sub>
+Unless you explicitly state otherwise, any contribution intentionally submitted
+for inclusion in this crate by you, as defined in the Apache-2.0 license, shall
+be dual licensed as above, without any additional terms or conditions.
+</sub>
+

TODO.md

@@ -1,22 +1,22 @@
 # TODOs
 
-- [x] ~~Minset corpus scheduler~~ still doc missing
-- [ ] Win32 shared mem and crash handler to have Windows in-process executor
-- [x] Other feedbacks examples (e.g. maximize allocations to spot OOMs)
 - [ ] Other objectives examples (e.g. execution of a given program point)
 - [ ] Objective-Specific Corpuses (named per objective)
-- [x] A macro crate with derive directives (e.g. for SerdeAny impl).
 - [ ] Good documentation
-- [ ] LLMP brotli compression
+- [ ] LLMP compression
 - [ ] AFL-Style Forkserver Executor
-- [x] Restarting EventMgr could use forks on unix
-- [ ] Android Ashmem support
 - [ ] Restart Count in Fuzzing Loop
 - [ ] LAIN / structured fuzzing example
-- [ ] Errors in the Fuzzer should exit the fuzz run
 - [ ] More informative outpus, deeper introspection (stats, what mutation did x, etc.)
-- [x] Timeouts for executors
 - [ ] Timeout handling for llmp clients (no ping for n seconds -> treat as disconnected)
 - [ ] LLMP Cross Machine Link (2 brokers connected via TCP)
 - [ ] "Launcher" example that spawns broker + n clients
 - [ ] Heap for signal handling (bumpallo or llmp directly?)
+- [x] ~~Minset corpus scheduler~~ still doc missing
+- [x] Win32 shared mem and crash handler to have Windows in-process executor
+- [x] Other feedbacks examples (e.g. maximize allocations to spot OOMs)
+- [x] A macro crate with derive directives (e.g. for SerdeAny impl).
+- [x] Restarting EventMgr could use forks on unix
+- [x] Android Ashmem support
+- [x] Errors in the Fuzzer should exit the fuzz run
+- [x] Timeouts for executors

docs/README.md

@@ -2,7 +2,7 @@
 
 This project contains the out-of-source LibAFL documentation as a book.
 
-Here you can find tutorials, examples and detailed explanations.
+Here you can find tutorials, examples, and detailed explanations.
 
 For the API documentation instead, run `cargo doc` in the LibAFl root folder.
 

docs/src/SUMMARY.md

@@ -14,7 +14,6 @@
 - [Design](./design/design.md)
   - [Core Concepts](./design/core_concepts.md)
   - [Architecture](./design/architecture.md)
-  - [The State](./design/state.md)
 
 - [Understanding Metadata](./medatata/metadata.md)
   - [Definition](./medatata/definition.md)

docs/src/design/architecture.md

@@ -8,6 +8,6 @@ The LibAFL code reuse meachanism is so based on components rather than sub-class
 
 Thinking about similar fuzzers, you can observe that most of the times the data structures that are modified are the ones related to testcases and the fuzzer global state.
 
-Beside the entities described previously, we then introduce the Testcase and State entities. The Testcase is a container for an Input stored in the Corpus and its metadata (so, in the implementation, the Corpus stores Testcases) and the State contains all the metadata that are evolved while running the fuzzer, Corpus included.
+Beside the entities described previously, we introduce the Testcase and State entities. The Testcase is a container for an Input stored in the Corpus and its metadata (so, in the implementation, the Corpus stores Testcases) and the State contains all the metadata that are evolved while running the fuzzer, Corpus included.
 
 

docs/src/design/state.md

@@ -6,19 +6,20 @@ LibAFL, as most of the Rust projects, can be built using `cargo` from the root d
 $ cargo build --release
 ```
 
-Note that the `--release` flag is optional for development, but it is needed for fuzzing at a decent speed,
-otherwise you will experience a slowdown of even more than 10x.
+Note that the `--release` flag is optional for development, but you needed to add it to fuzzing at a decent speed.
+Slowdowns of 10x or more are not uncommon for Debug builds.
 
-The LibAFL repository is composed by multiple crates, and the top-level Cargo.toml is just an orchestrator that groups these crates
-in a workspace. Building from the root directory will build all the crates in the workspace.
+The LibAFL repository is composed of multiple crates.
+The top-level Cargo.toml is the workspace file grouping these crates.
+Calling `cargo build` from the root directory will compile all crates in the workspace.
 
-## Build example fuzzers
+## Build Example Fuzzers
 
-You can notice that in the repository there is a `fuzzers/` folder.
-This folder contains a set of crates that are not part of the workspace, so that are not built issuing `cargo build` from the top-level directory.
+We group example fuzzers in the `./fuzzers` directory of the LibAFL repository.
+The directory contains a set of crates that are not part of the workspace.
 
-These crates are examples of fuzzers using particular features of LibAFL combined sometimes with instrumentation backends (e.g. [SanitizerCoverage](https://clang.llvm.org/docs/SanitizerCoverage.html), [Frida](https://frida.re/), ...).
+Each of these example fuzzers uses particular features of LibAFL, sometimes combined with different instrumentation backends (e.g. [SanitizerCoverage](https://clang.llvm.org/docs/SanitizerCoverage.html), [Frida](https://frida.re/), ...).
 
-The user can use these crates as examples and as skeleton for its custom fuzzer using a similar set of features.
+You can use these crates as examples and as skeletons for custom fuzzers with similar featuresets.
 
-To build an example fuzzer you have to invoke cargo from its folder (`fuzzers/[FUZZER_NAME]).
+To build an example fuzzer you have to invoke cargo from its respective folder (`fuzzers/[FUZZER_NAME]).

docs/src/getting_started/build.md

@@ -1,14 +1,29 @@
 # Introduction
 
-Fuzzers are important assets in the pockets of security researchers and even developers nowadays.
-A wide range of cool state-of-the-art tools like [AFL++](https://github.com/AFLplusplus/AFLplusplus), [libFuzzer](https://llvm.org/docs/LibFuzzer.html) or [honggfuzz](https://github.com/google/honggfuzz) are avaiable to users and they do their job in a very effective way.
+Fuzzers are important assets in the pockets of security researchers and developers alike.
+A wide range of cool state-of-the-art tools like [AFL++](https://github.com/AFLplusplus/AFLplusplus), [libFuzzer](https://llvm.org/docs/LibFuzzer.html) or [honggfuzz](https://github.com/google/honggfuzz) are available to users. They do their job in a very effective way, finding thousands of bugs.
 
-From the power user perspective, however, these tools are limited because not designed with the extensibility as first-class citizen.
-Usually, a fuzzer developer has to choose if fork one of these existing tools with the result of having a tons of fuzzers derived from others which are in any case incompatible with each other, or creating a new fuzzer from scratch, reinventing the wheel and usually giving up on features that are complex to reimplement.
+From the power user perspective, however, these tools are limited.
+Their design does not treat extensibility as a first-class citizen.
+Usually, a fuzzer developer can choose to either fork one of these existing tools, or to create a new fuzzer from scratch.
+In any case, researchers end up with tons of fuzzers, all of which are incompatible with each other.
+Their outstanding features can not just be combined for new projects.
+Instead, we keep reinventing the wheel and may completely miss out on features that are complex to reimplement.
 
-Here comes LibAFL, a library that IS NOT a fuzzer, but a collection of reusable pieces of fuzzers written in Rust.
-LibAFL helps you writing your own custom fuzzer, tailored for a specific target or for a particular instrumentation backend, without reinventing the wheel or forking an existing fuzzer.
+Here comes LibAFL, a library that IS NOT a fuzzer, but a collection of reusable pieces of fuzzers, written in Rust.
+LibAFL helps you develop your own custom fuzzer, tailored for your specific needs.
+Be it a specific target, a particular instrumentation backend, or a custom mutator, you can leverage existing bits and pieces to craft the fastest and most efficient fuzzer you can envision.
 
-## Why you should use LibAFL
+## Why LibAFL?
 
-TODO list here killer features (no_std, multi platform, scalability, ...)
+LibAFL gives you many of the benefits of an off-the-shelf fuzzer, while being completely customizable.
+Some highlight features currently include:
+- `multi platform`: LibAFL works pretty much anywhere you can find a Rust compiler for. We already used it on *Windows*, *Android*, *MacOS*, and *Linux*, on *x86_64*, *aarch64*, ...
+- `portable`: `LibAFL` can be built in `no_std` mode. This means it does not require a specific OS-dependent runtime to function. Define an allocator and a way to map pages, you should be good to inject LibAFL in obscure targets like embedded devices, hypervisors, or maybe even WebAssembly?
+- `adaptable`: Given year of experience fine-tuning *AFLplusplus* and our academic fuzzing background, we could incorporate recent fuzzing trends into LibAFL's deign and make it future-proof.
+To give an example, as opposed to old-skool fuzzers, a `BytesInput` is just one of the potential forms of inputs:
+feel free to use and mutate an Abstract Syntax Tree instead, for structured fuzzing.
+- `scalable`: As part of LibAFL, we developed `Low Level Message Passing`, `LLMP` for short, which allows LibAFL to scale almost linearly over cores. That is, if you chose to use this feature - it is your fuzzer, after all. Scaling to multiple machines over TCP is on the near road-map.
+- `fast`: We do everything we can at compiletime so that the runtime overhead is as minimal as it can get.
+- `bring your own target`: We support binary-only modes, like Frida-Mode with ASAN and CmpLog, as well as multiple compilation passes for sourced-based instrumentation, and of course supoprt custom instrumentation.
+- `usable`: This one is on you to decide. Dig right in!