Recipe

Write a property-based test

Use when: a function takes inputs from a large range, and you can state a rule its output must follow for every one of them

Goal: a proptest! test that checks the rule on a few hundred generated inputs, and a small failing input if the rule breaks

Taught in Parameterized and property-based tests

1. 1

   Write the rule as one sentence that doesn't name any single input.

   Example

   For split_even(total, people): "no two shares differ by more than one cent", and "the shares add up to total".

   • the function has an opposite, such as format and parse — the round trip: undo it and get the input back.
   • the output is made of parts — the parts add up to the whole, or there are the right number of them.
   • you can only say "this input gives that answer" — it's an example, so put it in a table-driven test instead.

   Why: proptest invents the inputs, so the check can't depend on knowing each answer in advance.

2. 2

   Put proptest under [dev-dependencies] in Cargo.toml. In the test module, write use proptest::prelude::*;.

   Example

   proptest = "1.11.0", on its own line under [dev-dependencies].

   Why: only the tests use it, so the library's users never download it.

3. 3

   Give each input a range that covers every valid value and nothing else.

   Example

   total in 0u64..100_000, people in 1u32..20. The range for people starts at 1, because splitting a bill among 0 people isn't a valid call.

   Why: an input the function was never meant to take fails for a reason that isn't a bug.

4. 4

   Wrap a #[test] function in proptest! { ... }, with the strategies as its parameters. Check the rule with prop_assert! or prop_assert_eq!, and give it a message that prints the output.

   Example

   prop_assert!(most - least <= 1, "shares were {:?}", shares);

   Why: prop_assert! reports the failure to proptest, which then shrinks the input.

5. 5

   Run cargo test. If it fails, find minimal failing input in the report, and work that one case out on paper.

   Example

   The report says total = 56, people = 3 and shares were [18, 18, 20]. By hand, 56 = 3 × 18 + 2. The two spare cents should go to two different people: [19, 19, 18].

   Why: the shrunk input is small enough to check by hand, and the gap between your answer and the output points at the bug.

6. 6

   Commit the proptest-regressions folder. Fix the code, and run the tests until the property holds.

   Example

   proptest saved the case in proptest-regressions/lib.txt. Every later run tries it first.

   Why: the saved input stays a test, even on a run whose random inputs would have missed it.