Document all the things

Author: oli-obk

ui_test/README.md

@@ -7,25 +7,37 @@ A smaller version of compiletest-rs
 
 ## Supported magic comment annotations
 
-* `// ignore-XXX` avoids running the test on targets whose triple contains `XXX`
+If your test tests for failure, you need to add a `//~` annotation where the error is happening
+to make sure that the test will always keep failing with a specific message at the annotated line.
+
+`//~ ERROR: XXX` make sure the stderr output contains `XXX` for an error in the line where this comment is written
+
+* Also supports `HELP`, `WARN` or `NOTE` for different kind of message
+    * if one of those levels is specified explicitly, *all* diagnostics of this level or higher need an annotation. If you want to avoid this, just leave out the all caps level note entirely.
+* If the all caps note is left out, a message of any level is matched. Leaving it out is not allowed for `ERROR` levels.
+* This checks the output *before* normalization, so you can check things that get normalized away, but need to
+    be careful not to accidentally have a pattern that differs between platforms.
+
+In order to change how a single test is tested, you can add various `//@` comments to the test.
+Any other comments will be ignored, and all `//@` comments must be formatted precisely as
+their command specifies, or the test will fail without even being run.
+
+* `//@ignore-XXX` avoids running the test on targets whose triple contains `XXX`
     * `XXX` can also be one of `64bit`, `32bit` or `16bit`
-* `// only-XXX` avoids running the test on targets whose triple **does not** contain `XXX`
+* `//@only-XXX` avoids running the test on targets whose triple **does not** contain `XXX`
     * `XXX` can also be one of `64bit`, `32bit` or `16bit`
-* `// stderr-per-bitwidth` produces one stderr file per bitwidth, as they may differ significantly sometimes
+* `//@stderr-per-bitwidth` produces one stderr file per bitwidth, as they may differ significantly sometimes
 * `//@error-pattern: XXX` make sure the stderr output contains `XXX`
-* `//~ ERROR: XXX` make sure the stderr output contains `XXX` for an error in the line where this comment is written
-    * Also supports `HELP`, `WARN` or `NOTE` for different kind of message
-        * if one of those levels is specified explicitly, *all* diagnostics of this level or higher need an annotation. If you want to avoid this, just leave out the all caps level note entirely.
-    * If the all caps note is left out, a message of any level is matched. Leaving it out is not allowed for `ERROR` levels.
-    * This checks the output *before* normalization, so you can check things that get normalized away, but need to
-      be careful not to accidentally have a pattern that differs between platforms.
-* `// revisions: XXX YYY` runs the test once for each space separated name in the list
+* `//@revisions: XXX YYY` runs the test once for each space separated name in the list
     * emits one stderr file per revision
     * `//~` comments can be restricted to specific revisions by adding the revision name before the `~` in square brackets: `//[XXX]~`
-* `// compile-flags: XXX` appends `XXX` to the command line arguments passed to the rustc driver
-* `// rustc-env: XXX=YYY` sets the env var `XXX` to `YYY` for the rustc driver execution.
+* `//@compile-flags: XXX` appends `XXX` to the command line arguments passed to the rustc driver
+    * you can specify this multiple times, and all the flags will accumulate
+* `//@rustc-env: XXX=YYY` sets the env var `XXX` to `YYY` for the rustc driver execution.
     * for Miri these env vars are used during compilation via rustc and during the emulation of the program
-* `// normalize-stderr-test: "REGEX" -> "REPLACEMENT"` replaces all matches of `REGEX` in the stderr with `REPLACEMENT`. The replacement may specify `$1` and similar backreferences to paste captures.
+    * you can specify this multiple times, accumulating all the env vars
+* `//@normalize-stderr-test: "REGEX" -> "REPLACEMENT"` replaces all matches of `REGEX` in the stderr with `REPLACEMENT`. The replacement may specify `$1` and similar backreferences to paste captures.
+    * you can specify multiple such commands, there is no need to create a single regex that handles multiple replacements that you want to perform.
 
 ## Significant differences to compiletest-rs
 

ui_test/src/comments.rs

@@ -75,7 +75,7 @@ impl Comments {
     /// Parse comments in `content`.
     /// `path` is only used to emit diagnostics if parsing fails.
     ///
-    /// This function will only parse `//@` and `//~` style comments
+    /// This function will only parse `//@` and `//~` style comments (and the `//[xxx]~` variant)
     /// and ignore all others
     fn parse_checked(path: &Path, content: &str) -> Result<Self> {
         let mut this = Self::default();