Merge pull request #19 from adolgert/docs/explain-up-front

Docs/explain up front

Author: Drew Dolgert

docs/make.jl

@@ -17,8 +17,10 @@ makedocs(;
         "Home" => "index.md",
         "Manual" => [
             "Guide" => "man/guide.md",
+            "Examples" => "man/examples.md",
+            "Testing Methods" => "man/methods.md",
             "Engines" => "man/engines.md",
-            "Usage" => "man/usage.md"
+            "IPOG" => "man/ipog.md"
             ],
         "Reference" => "reference.md",
         "Contributing" => "contributing.md"

docs/src/contributing.md

@@ -12,9 +12,9 @@ and my email is listed.
 
 ## Branch process
 
-- The `main` branch is for development.
-- Releases go to `release`.
+The trunk is called `main`. I use PRs from branches or other repos to main.
+If this project needs more than that, let's discuss it.
 
 ## Conduct
 
-Let's follow the contributor covenant.
+Let's follow the [Contributor Covenant Code of Conduct](@ref).

docs/src/index.md

@@ -1,10 +1,12 @@
 
 # UnitTestDesign
 
-A [Julia](http://julialang.org) package to generate test cases for unit tests.
-This provides **all-pairs** and higher-order algorithms.
+A [Julia](http://julialang.org) package to generate parameters for unit tests
+and test data for unit tests. You tell it possible values for each parameter of a function,
+and it selects combinations of parameters that are most likely to find
+problems in that function.
 
-* All-pairs, all-triples, and higher-order coverage of test cases.
+* [All-pairs](http://pairwise.org/), all-triples, and higher-order coverage of test cases.
 * Combinatorial excursions from a base test case.
 * Convenient ways to avoid uninteresting parameter combinations,
   add necessary test cases, and increase coverage for subsets of parameters.
@@ -16,24 +18,7 @@ pkg> add https://github.com/adolgert/UnitTestDesign.jl
 ```
 
 ## Description
-
-If we have a function-under-test that takes four arguments, each of which
-can have three possible values, then there are 81 possible combinations of
-inputs. The [`all_pairs`](@ref) function selects 10 inputs that contain every pair
-parameter values at least once.
-
-```@repl
-using UnitTestDesign
-test_cases = all_pairs(
-    [1, 2, 3], ["low", "mid" ,"high"], [1.0, 3.7, 4.9], [:greedy, :relax, :optim])
-```
-
-Each item in this array is a set of parameters for unit testing the function.
-This test set is an example of all-pairs because we can pick any two parameters (second and fourth),
-pick any of the values for those parameters ("mid" and :relax), and find
-them in one of the test cases (the 2nd one).
-
-Use a test set for unit testing:
+Write a unit test using the [`all_pairs`](@ref) function.
 
 ```julia
 test_set = all_pairs(
@@ -44,6 +29,16 @@ for test_case in test_set
     @test test_result == known_result(test_case)
 end
 ```
-
 This package doesn't help write the code that knows what the
 [correct test result](https://en.wikipedia.org/wiki/Test_oracle) should be.
+
+In order to test every possible input to the function above, you would
+need 81 tests, but this generates 10 tests that are more likely to find
+most faults because they include every combination of each pair of parameter
+values.
+
+```@repl
+using UnitTestDesign
+test_cases = all_pairs(
+    [1, 2, 3], ["low", "mid" ,"high"], [1.0, 3.7, 4.9], [:greedy, :relax, :optim])
+```

docs/src/man/code_of_conduct.md

@@ -0,0 +1,134 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[INSERT CONTACT METHOD].
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
+
+Community Impact Guidelines were inspired by 
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available 
+at [https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations
+

docs/src/man/examples.md

@@ -0,0 +1,73 @@
+# Examples
+
+## Test parameter generation
+
+For a small test set, generate tests is quick.
+
+```julia
+test_set = all_pairs(
+    [(-3, -2),  (2, 3), (-2, 3), (4.999, 5), (0, 1e7)],
+    [0.1, 0.01, 0.001],
+    [:RungeKutta, :Midpoint]
+    )
+for test_case in test_set
+    a, b = test_case[1]
+    result = custom_integrate(a, b, test_case[2:end]...)
+    @test result == compare_with_symbolic_integration(a, b)
+end
+```
+
+
+## Save larger test suites instead of regenerating
+
+For a larger test set, it can be worthwhile to make test suites
+and save them.
+```julia
+wide_test = all_triples(fill(1:4, 30)...)
+JLD.save("test_artifact.jld", "wide_test", wide_test)
+```
+These could then be made available for testing through
+[Pkg.Artifacts](https://julialang.github.io/Pkg.jl/v1.5/artifacts/).
+
+
+## Test data generation
+
+We use combinatoric testing because it finds all code paths. Code
+paths are lines of code chosen by the same if-then decision.
+Some code has few code paths but many branches. For instance,
+`data_frame = all_data_frame[data_frame[:, :time] > 10]` has two
+branches, one for times less than ten and one for times greater than ten.
+
+This library will create test data as easily as it creates test cases.
+
+```@example
+using UnitTestDesign  # hide
+using DataFrames
+names = ["time", "event", "who", "location"]
+at = all_triples([0.1, 5.1, 10.9], [:infect, :recover], [10, 11], ["forest", "home"])
+DataFrame(Dict(names[i] => [row[i] for row in at] for i in 1:length(names))...)
+```
+
+## Excursions
+
+If you want to start with a single test that you know is a common
+way to call a function, then an excursion can generate variations
+around that common way. It picks one parameter and walks through
+its values. Then another. If you choose a pair-wise excursion, it
+walks all pairs away from the initial value.
+
+```@example
+using UnitTestDesign  # hide
+values_excursion([1, 2], [true, false], ["c", "b", "a"])
+```
+
+## Filtering a factorial
+
+The full-factorial test generates every possible test. If some combinations
+of parameters aren't interesting or allowed for a function, you can
+exclude them by using an extra argument.
+```@example
+using UnitTestDesign  # hide
+disallow = (a, b, c) -> b == 7 && c == false
+full_factorial([1, 2, 3], [7, 8], [true, false]; disallow = disallow)
+```

docs/src/man/guide.md

@@ -22,7 +22,7 @@
     - [`full_factorial`](@ref): Generates all combinations of parameters, filtering
       those that aren't permitted.
 
-## Same interface for all of test generators
+## Same interface for all test generators
 
 ### Increase coverage for subsets of parameters
 
@@ -34,7 +34,8 @@ to request that the subset of parameters have greater coverage.
 For instance, this requrests that the first, third, and fourth parameters
 have 3-way coverage, meaning full-factorial, while the second parameter has
 only 2-way coverage. This is more meaningful when there are lots of parameters.
-```julia
+```@example
+using UnitTestDesign  # hide
 test_set = all_pairs(
     [1, 2, 3], ["low", "mid" ,"high"], [1.0, 3.7, 4.9], [:greedy, :relax, :optim];
     wayness = Dict(3 => [[1, 3, 4]])
@@ -48,6 +49,7 @@ together, should be covered at the given level. This means that, were you to hav
 parameters, you could request that parameters `[3:6]` and parameters `[25:30]` be
 covered with triples.
 ```julia
+array_of_forty_parameters = fill(1:4, 40)
 test_set = all_pairs(
     array_of_forty_parameters...;
     wayness = Dict(3 => [[3:6], [25:30]])
@@ -61,11 +63,12 @@ Keep the test engine from making tests that aren't allowed for
 your function. Pass it a filter function, one that returns `true`
 whenever a parameter combinations is forbidden.
 
-```julia
+```@example
+using UnitTestDesign  # hide
 disallow(n, level, value, kind) = level == "high" && kind == :optim
 test_set = all_pairs(
     [1, 2, 3], ["low", "mid" ,"high"], [1.0, 3.7, 4.9], [:greedy, :relax, :optim];
-    filter = disallow
+    disallow = disallow
     )
 ```
 
@@ -76,7 +79,8 @@ If there are particular tests that must be run, these already include
 some of the tuples that should be covered. You can pass the must-run
 test cases, and they will be included among the test cases.
 
-```julia
+```@example
+using UnitTestDesign  # hide
 must_test = [[1, "mid", 3.7, :relax], [1, "mid", 4.9, :relax]]
 test_cases = all_pairs(
     [1, 2, 3], ["low", "mid" ,"high"], [1.0, 3.7, 4.9], [:greedy, :relax, :optim];