Property-based testing deserves its moment

Example-based tests check the cases you thought of. Property-based tests check the cases you did not.

The technique has been around since QuickCheck in 1999 and has stayed niche. It fits the current moment unusually well, and it is worth another look.

the idea#

Instead of "for this input, expect this output," you state a property that should hold for all inputs, and the framework generates hundreds of inputs trying to break it.

from hypothesis import given, strategies as st

@given(st.lists(st.integers()))
def test_sort_is_idempotent(xs):
    assert sorted(sorted(xs)) == sorted(xs)

@given(st.lists(st.integers()))
def test_sort_preserves_elements(xs):
    assert sorted(xs).count == xs.count   # same multiset

@given(st.text())
def test_roundtrip(s):
    assert decode(encode(s)) == s

The framework generates empty lists, single elements, lists with duplicates, huge values, negative numbers, and — critically — when it finds a failure, it shrinks the input to the minimal case that still fails.

That shrinking is what makes the technique practical. A failure on a 400-element list is not actionable. The same failure shrunk to [0, 0] tells you exactly what is wrong.

the properties that are actually useful#

The hard part is not the tooling. It is identifying properties, and there is a small catalogue that covers most cases.

Round trip. decode(encode(x)) == x. Serialization, compression, parsing, encryption. This one property catches an enormous number of real bugs and applies almost everywhere.

Invariants. Something that is always true regardless of operations. A balanced tree stays balanced. A sorted list stays sorted. Account balances sum to the same total across a transfer. A cache never returns stale data past its TTL.

Oracle. Compare against a simpler, slower, obviously-correct implementation. You have an optimized version; write the naive one, and assert they agree. This is extremely powerful and underused — it is how you test the fast path against the version you can reason about.

Idempotence. f(f(x)) == f(x). Normalization, deduplication, and — importantly — any API operation that claims to be idempotent. If you have idempotency keys, this is how you actually verify them.

Commutativity and associativity. Order should not matter. Merge operations, set operations, CRDT merges.

Metamorphic relations. When you cannot state the correct output, state how the output should change. Adding an item to a cart should increase the total by that item's price. Searching for a more specific query should return a subset. Sorting descending should be the reverse of sorting ascending.

This last category is the one that unlocks property testing for business logic, where there is no obvious oracle.

where it fits best#

Parsers and serializers. Round trip, always.

Data structures. Invariants after every operation sequence.

State machines. Generate random valid operation sequences, assert the invariants hold throughout. This finds ordering bugs that no hand-written test would.

Anything with an obvious naive implementation. Oracle testing.

Financial and unit arithmetic. Rounding, currency, conversions. The edge cases are numerous and boring, which is exactly what a generator is for.

Concurrent code, with a framework that generates interleavings. This is the hardest category to test any other way.

where it does not fit#

UI. The properties are aesthetic.

Anything where the correct output requires human judgment.

Integration tests against real systems. Generation implies many runs, and many runs against a real database is slow.

Code where you cannot state a property. If you genuinely cannot, that may itself be information about the design — code with no statable invariants is code with no contract.

the practical advice#

Do not replace example tests. Keep them. They document intent and they are the fastest way to communicate what a function is for. Add properties alongside.

Start with round-trip properties. Easiest to state, highest hit rate for real bugs.

Save the failing seed. When a property fails, the framework gives you the counterexample. Add it as a regression example test so it is checked deterministically forever.

Bound the input space. Unbounded generation produces absurd inputs and slow tests. Constrain to realistic ranges: strings up to a reasonable length, numbers in a plausible domain.

Run more cases in CI than locally. A hundred examples locally for fast feedback, a thousand in CI, ten thousand in a nightly run.

why now#

Two reasons this fits the current moment.

Verification is the bottleneck. Property testing is verification you write once and that checks a large input space forever. That is exactly the leverage the moment calls for.

Stating a property is a task worth doing carefully by hand, and generating the implementation is not. Writing "the total must always equal the sum of line items" requires understanding the domain; nothing else in the test does.

That is a good division of labor, and it points at where human effort should concentrate: on specifying what must be true, and letting everything else be derived.