-
Notifications
You must be signed in to change notification settings - Fork 1.6k
RFC: Add descriptive names to doctests #3311
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
7641de1
Initial draft of `doctest-name` RFC
casey 78f4c1d
Typo: teest -> test
casey 1ccd36e
Address feedback
casey 9472e1a
Require that test names be valid rust identifiers
casey cc3c07a
Use spaces instead of commas
casey 424f3cd
Use commas to separate attributes
casey 7f90cc4
Note that raw identifiers are permitted
casey 8d13ad0
Warn if IDENTs are non-unique.
casey File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,142 @@ | ||
- Feature Name: doctest-name | ||
- Start Date: 2010-10-07 | ||
- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) | ||
- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) | ||
|
||
# Summary | ||
[summary]: #summary | ||
|
||
Allow giving documentation tests names for identification in test runner output | ||
by including `name=IDENT` in doctest code block info strings. | ||
|
||
# Motivation | ||
[motivation]: #motivation | ||
|
||
When doctests run, the test runner prints the name of the module that contains | ||
them and the line number that they start on. If a module contains many | ||
doctests, it is hard to identify which doctest is which in the test runner | ||
output by line number alone. | ||
|
||
By giving users the option to give names to doctests, identifying individual | ||
doctests in test runner output will be easier. Additionally, users could re-run | ||
specific tests using `cargo test --doc -- $TEST_NAME`. | ||
|
||
# Guide-level explanation | ||
[guide-level-explanation]: #guide-level-explanation | ||
|
||
Doctests are written as fenced markdown code blocks that start and end with | ||
lines containing three backticks. | ||
|
||
The first set of backticks may be followed by an info string. For example, to | ||
tell the Rust compiler to ignore the doctest: | ||
|
||
````rust | ||
|
||
// This doctest won't run. | ||
```ignore | ||
assert_eq!(2 + 2, 4); | ||
``` | ||
|
||
```` | ||
|
||
Normally, doctests do not have names, and when run, the test runner will | ||
identify them by their test binary, module name, and line number: | ||
|
||
``` | ||
Doc-tests foo | ||
|
||
running 1 test | ||
test src/lib.rs - bar (line 37) ... ok | ||
|
||
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out | ||
``` | ||
|
||
If you'd like to have the test runner print a more informative name, perhaps | ||
because there are many doctests in a single file, you can put `name=IDENT`, | ||
where `IDENT` is the name the test should have, which must be an identifier, in | ||
the doctest's info string. | ||
|
||
|
||
````rust | ||
|
||
```name=arithmetic | ||
assert_eq!(2 + 2, 4); | ||
``` | ||
|
||
```` | ||
|
||
``` | ||
Doc-tests foo | ||
|
||
running 1 test | ||
test src/lib.rs - bar::arithmetic (line 37) ... ok | ||
|
||
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out | ||
``` | ||
|
||
# Reference-level explanation | ||
[reference-level-explanation]: #reference-level-explanation | ||
|
||
The set of valid words that may appear in code block info strings is extended | ||
to include patterns of the form `name=IDENT`. | ||
|
||
This `IDENT` will be the name of the documentation test generated by the code | ||
block, and must be a valid Rust identifier, which includes raw `r#`-prefixed | ||
identifiers. | ||
|
||
When the test runner runs the documentation test, the test will be identified | ||
by this name, in addition to the test binary, module, and line number. | ||
|
||
Multiple `name=IDENT` words may not appear in a single code block's info | ||
string. | ||
|
||
Non-unique `IDENT`s and `IDENTS` that conflict with other items within the | ||
current namespace will produce a warn-by-default error that may become a hard | ||
error in the future. | ||
|
||
`name=IDENT` may be combined with existing annotations like `rust` or | ||
`should_panic` by separating the annotations with commas: | ||
|
||
```` | ||
|
||
```rust,name=foo | ||
assert!(true) | ||
``` | ||
|
||
```` | ||
|
||
The ability to include multiple comma-separated annotations is an existing | ||
feature. | ||
|
||
# Drawbacks | ||
[drawbacks]: #drawbacks | ||
|
||
- Adds complexity to code block parsing. | ||
|
||
- Requires parsing multi-word info strings, which are standard markdown, but | ||
which does increase the complexity of the markdown parser. | ||
|
||
# Rationale and alternatives | ||
[rationale-and-alternatives]: #rationale-and-alternatives | ||
|
||
This design seems like the simplest design possible, although viable | ||
alternatives may have been overlooked. | ||
|
||
The impact of adopting this RFC is minimal. Doctests test output will remain a | ||
little hard to make sense of. | ||
|
||
# Prior art | ||
[prior-art]: #prior-art | ||
|
||
Unit and integration tests have names. This change would bring doctests closer | ||
to parity, by allowing features like test filtering to work with doctests. | ||
|
||
# Unresolved questions | ||
[unresolved-questions]: #unresolved-questions | ||
|
||
None. | ||
casey marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# Future possibilities | ||
[future-possibilities]: #future-possibilities | ||
|
||
None. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.