Property-Based Testing

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

This page is written against Scala 3.10 (3.10.0, released 2026-10-12) and every example was compiled and run with Scala 3.10.0 through Scala CLI on JDK 26 (checked on 2026-10-16). Property-based testing needs nothing that is specific to Scala 3.10: the libraries used here are built for Scala 3, so the examples also work on the 3.9 LTS and 3.3 LTS lines (changing the //> using scala directive is enough; they were run on 3.10.0 only).

ScalaCheck is not part of the Scala Toolkit and is not an official Scala project. It is a third-party library (maintained under the Typelevel organization) that is on Maven Central. This page uses it through munit-scalacheck, MUnit’s own integration module, so that properties are ordinary tests in the same suites you already run with scala test. The two libraries and their versions at the time of writing are:

Library Version Role

org.scalacheck::scalacheck

1.20.0

Generators (Gen), Arbitrary, properties (Prop), shrinking, the property runner.

org.scalameta::munit-scalacheck

1.3.1

ScalaCheckSuite: lets property(...) appear next to test(...) in an MUnit suite. Brings ScalaCheck and MUnit 1.3.6 with it.

The suites in this page use the Toolkit test setup from Testing with MUnit (//> using test.toolkit 0.9.2) and add one line for the integration. Asynchronous tests, fixtures and ScalaTest are in MUnit Async, Fixtures & Other Frameworks. For the Java and Groovy counterparts see Java Testing and Testing with JUnit and GroovyTest.

Properties versus examples

An example-based test fixes one input and one expected output:

test("reversing one list"):
  assertEquals(List(1, 2, 3).reverse, List(3, 2, 1))

Source: Scala Toolkit: How to write tests?

A property states something that must hold for every input of some type, and the framework searches for an input that breaks it. "Reversing a list twice gives the original list" is one sentence and covers empty lists, one element, thousands of elements and duplicates, none of which you had to think of. The two styles complement each other: examples document a few important cases and pin down a specific expected value, properties cover the space around them and find the cases nobody imagined.

A property test has four moving parts, each described below.

  1. A generator (Gen[A]) produces random values of a type.

  2. An Arbitrary[A] is the given that tells ScalaCheck which generator to use by default for the type A.

  3. A property (Prop) is a function from generated values to a boolean (or to an assertion) that must hold for all of them.

  4. On a failure, shrinking searches for a smaller input that still fails, so the report shows a minimal counterexample.

Setting up with Scala CLI

//> using scala 3.10.0
//> using test.toolkit 0.9.2
//> using test.dep org.scalameta::munit-scalacheck::1.3.1

Source: MUnit: ScalaCheck integration

The sbt equivalent adds one line next to the test dependencies (see sbt & Build Tools):

libraryDependencies += "org.scalameta" %% "munit-scalacheck" % "1.3.1" % Test

Source: MUnit: ScalaCheck integration

Tests live in files ending in .test.scala (or in a test directory), and scala test --server=false . runs them. A suite extends munit.ScalaCheckSuite instead of munit.FunSuite; ScalaCheckSuite is a FunSuite, so test and assertEquals keep working.

A first suite

The first suite checks four properties of List that need no custom types. Each property body returns a Prop; forAll builds one from a function. The property can return a Boolean or an MUnit assertion.

//> using scala 3.10.0
//> using test.dep org.scalameta::munit::1.2.1
//> using test.dep org.scalameta::munit-scalacheck::1.3.1

package trailhead

import munit.ScalaCheckSuite
import org.scalacheck.Prop.forAll

class ListProps extends ScalaCheckSuite:

  test("an example: reversing one list"):
    assertEquals(List(1, 2, 3).reverse, List(3, 2, 1))

  property("reversing twice gives the original list"):
    forAll { (xs: List[Int]) =>
      xs.reverse.reverse == xs
    }

  property("concatenation is associative"):
    forAll { (a: List[Int], b: List[Int], c: List[Int]) =>
      (a ++ b) ++ c == a ++ (b ++ c)
    }

  property("the empty list is the identity of concatenation"):
    forAll { (xs: List[Int]) =>
      (xs ++ Nil == xs) && (Nil ++ xs == xs)
    }

  property("sorting is idempotent"):
    forAll { (xs: List[Int]) =>
      xs.sorted.sorted == xs.sorted
    }

  property("reverse preserves the size (invariant)"):
    forAll { (xs: List[String]) =>
      xs.reverse.size == xs.size
    }
Test run trailhead.ListProps started
trailhead.ListProps: finished 0.066s
Test run trailhead.ListProps finished: 0 failed, 0 ignored, 6 total 0.068s

The suite ran six tests: one example and five properties, each executed 100 times with different inputs by default. forAll finds the generator through Arbitrary; the library provides givens for Int, String, Boolean, Option, List, tuples, and many other standard types, so forAll { (xs: List[Int]) => ... } needs nothing else. The (xs: List[Int]) => parameter types are what select the generators, so they must be written.

Source: MUnit: ScalaCheck integration

The property taxonomy

Most useful properties fall into a handful of recurring shapes. Thinking in these shapes is what makes property tests quick to write, because you do not start from "what should I assert?" but from "which law does this function obey?".

Shape Statement Example

Round trip (inverse)

decode(encode(x)) == x

parse(render(trail)) == Some(trail)

Idempotence

f(f(x)) == f(x)

xs.sorted.sorted == xs.sorted, normalizing a path twice.

Invariant

Something about x is unchanged by f

xs.reverse.size == xs.size; a sort keeps the same elements.

Commutativity

f(a, b) == f(b, a)

a + b == b + a, set union.

Associativity

f(f(a, b), c) == f(a, f(b, c))

List concatenation; needed for parallel or chunked folds.

Identity element

f(x, e) == x and f(e, x) == x

xs ++ Nil == xs

Induction

Relate f(x :: xs) to f(xs)

+totalLength(t :: ts) == t.lengthM + totalLength(ts)+

Test oracle

The code under test agrees with a simple, obviously correct (often slower) version

A hand-tuned algorithm against a one-line sortBy implementation.

Homomorphism

The function distributes over an operation

total(a ++ b) == total(a) + total(b)

The second suite applies the shapes to a trailhead case class, with a custom generator.

//> using scala 3.10.0
//> using test.toolkit 0.9.2
//> using test.dep org.scalameta::munit-scalacheck::1.3.1

package trailhead

import munit.ScalaCheckSuite
import org.scalacheck.{Arbitrary, Gen, Prop}
import org.scalacheck.Prop.{forAll, propBoolean}

final case class Trail(name: String, lengthM: Int, climbM: Int)

object Trail:
  /** Text form used on signposts: name;length;climb */
  def render(t: Trail): String = s"${t.name};${t.lengthM};${t.climbM}"

  def parse(s: String): Option[Trail] = s.split(";", -1) match
    case Array(n, l, c) => for li <- l.toIntOption; ci <- c.toIntOption yield Trail(n, li, ci)
    case _              => None

  def totalLength(ts: List[Trail]): Int = ts.map(_.lengthM).sum

  /** Naive and fast versions of the same computation: the fast one is the code under test. */
  def longestNaive(ts: List[Trail]): Option[Trail] = ts.sortBy(-_.lengthM).headOption
  def longest(ts: List[Trail]): Option[Trail] = ts.maxByOption(_.lengthM)

class TrailProps extends ScalaCheckSuite:

  // A custom generator: names without the ';' separator, plausible sizes.
  val nameGen: Gen[String] = Gen.alphaStr.suchThat(_.nonEmpty)
  val trailGen: Gen[Trail] =
    for
      n <- nameGen
      l <- Gen.choose(100, 40_000)
      c <- Gen.choose(0, 3_000)
    yield Trail(n, l, c)

  given Arbitrary[Trail] = Arbitrary(trailGen)

  property("round trip: parse(render(t)) == Some(t)"):
    forAll { (t: Trail) =>
      Trail.parse(Trail.render(t)) == Some(t)
    }

  property("total length is additive (homomorphism)"):
    forAll { (a: List[Trail], b: List[Trail]) =>
      Trail.totalLength(a ++ b) == Trail.totalLength(a) + Trail.totalLength(b)
    }

  property("test oracle: the fast longest agrees on length with the naive one"):
    forAll { (ts: List[Trail]) =>
      Trail.longest(ts).map(_.lengthM) == Trail.longestNaive(ts).map(_.lengthM)
    }

  property("implication: a non-empty list has a longest trail"):
    forAll { (ts: List[Trail]) =>
      ts.nonEmpty ==> Trail.longest(ts).isDefined
    }

  property("induction: adding a trail adds its length"):
    forAll { (t: Trail, ts: List[Trail]) =>
      Trail.totalLength(t :: ts) == t.lengthM + Trail.totalLength(ts)
    }

  property("an explicit generator and classification"):
    forAll(Gen.listOf(trailGen)) { (ts: List[Trail]) =>
      Prop.classify(ts.isEmpty, "empty")(Trail.totalLength(ts) >= 0)
    }

Running it with scala test --server=false . — -v (the arguments after -- go to MUnit; -v prints each test):

Test run trailhead.TrailProps started
trailhead.TrailProps:
trailhead.TrailProps.round trip: parse(render(t)) == Some(t) started
  + round trip: parse(render(t)) == Some(t) 0.04s
trailhead.TrailProps.total length is additive (homomorphism) started
  + total length is additive (homomorphism) 0.045s
trailhead.TrailProps.test oracle: the fast longest agrees on length with the naive one started
  + test oracle: the fast longest agrees on length with the naive one 0.009s
trailhead.TrailProps.implication: a non-empty list has a longest trail started
  + implication: a non-empty list has a longest trail 0.008s
trailhead.TrailProps.induction: adding a trail adds its length started
  + induction: adding a trail adds its length 0.017s
trailhead.TrailProps.an explicit generator and classification started
  + an explicit generator and classification 0.005s
trailhead.TrailProps: finished 0.135s
Test run trailhead.TrailProps finished: 0 failed, 0 ignored, 6 total 0.14s

Three details of this suite are worth explaining.

  • The oracle property compares the lengths of the two answers, not the trails, because two trails can tie for the longest and the two implementations are free to pick different ones. Choosing what to compare is part of designing a property.

  • ==> (from Prop.propBoolean) is an implication: when the left side is false the generated value is discarded rather than counted as a failure. Discarding too many values makes ScalaCheck give up ("Gave up after only N passed tests"); prefer a generator that produces valid values (Gen.nonEmptyListOf, suchThat on a mostly-true condition) to a restrictive ==>.

  • Prop.classify and Prop.collect label the generated data, which is how you check that the generator covers the cases you care about. The labels are printed by the plain ScalaCheck runner (next section); ScalaCheckSuite only reports a failure.

Source: Scala Toolkit: Testing, what else?

Generators

A Gen[A] is a recipe for producing an A from a random seed and a size. The generators compose with map, flatMap and for comprehensions (see For-Comprehensions), which is how trailGen above was built.

Combinator Meaning

Gen.choose(lo, hi)

A number between the bounds, inclusive.

Gen.oneOf(a, b, c), Gen.oneOf(seq)

One of the given values or generators.

Gen.frequency(3 -> g1, 1 -> g2)

A weighted choice between generators.

Gen.const(x), Gen.fail

Always x; never produces a value.

Gen.listOf(g), Gen.nonEmptyListOf(g), Gen.listOfN(n, g)

Lists of generated elements, with a size from the test or a fixed n.

Gen.alphaStr, Gen.alphaNumStr, Gen.stringOf(g)

Strings from letters, letters and digits, or from a character generator.

g.map(f), g.flatMap(f), g.suchThat(p), g.filter(p)

Transform, chain, or restrict a generator; suchThat retries until the predicate holds.

A generator is only a value: gen.sample returns an Option[A] for experimenting in the REPL, and pureApply(params, seed) returns the same result for the same seed. The standalone run below (with plain ScalaCheck, so the labels print) calls the same Gen.listOfN(3, Gen.choose(0, 9)) twice with seed 42 and shows the classification report from Prop.check().

//> using scala 3.10.0
//> using dep org.scalacheck::scalacheck:1.20.0

import org.scalacheck.{Gen, Prop, Test}
import org.scalacheck.Prop.forAll

@main def standalone(): Unit =
  val sizes = forAll(Gen.listOf(Gen.choose(1, 9))) { (xs: List[Int]) =>
    Prop.classify(xs.isEmpty, "empty list")(
      Prop.classify(xs.size > 20, "long list")(xs.reverse.reverse == xs)
    )
  }
  sizes.check()

  val gen = Gen.choose(1, 5).flatMap(n => Gen.listOfN(n, Gen.choose(0, 9)))
  println(gen.sample.map(_.size).exists(n => n >= 1 && n <= 5))
  println(Gen.listOfN(3, Gen.choose(0, 9)).pureApply(Gen.Parameters.default, org.scalacheck.rng.Seed(42L)))
  println(Gen.listOfN(3, Gen.choose(0, 9)).pureApply(Gen.Parameters.default, org.scalacheck.rng.Seed(42L)))
+ OK, passed 100 tests.
> Collected test data:
52% long list
6% empty list
true
List(1, 7, 9)
List(1, 7, 9)

The label percentages show that, with the default size, half of the lists are longer than 20 elements and 6% are empty. If a rare case matters (here, an empty list), the classification is how you notice that the generator rarely reaches it.

The generator controls which bugs can be found. The default Arbitrary[String] produces strings of random characters, so a bug that appears only when a name contains ; is practically never reached (see the next section, where the same property passed 200 runs with the default generator). Write a generator that makes the interesting values likely.

Source: ScalaCheck user guide

Shrinking and reproducing failures

Random failing inputs are usually large and noisy. When a property fails, ScalaCheck shrinks the failing arguments: it repeatedly tries smaller variants (shorter lists, numbers closer to zero, shorter strings) and keeps the smallest one that still fails. The report shows both the shrunk argument (ARG_0) and the original one (ARG_0_ORIGINAL).

The following suite contains a real bug: render does not escape ; in a name, so parse cannot read it back. It also raises the number of runs and shows the parameter hook of ScalaCheckSuite.

//> using scala 3.10.0
//> using test.toolkit 0.9.2
//> using test.dep org.scalameta::munit-scalacheck::1.3.1

package trailhead

import munit.ScalaCheckSuite
import org.scalacheck.Gen
import org.scalacheck.Prop.forAll
import org.scalacheck.Test

final case class Trail(name: String, lengthM: Int)

object Trail:
  def render(t: Trail): String = s"${t.name};${t.lengthM}"
  def parse(s: String): Option[Trail] = s.split(";", -1) match
    case Array(n, l) => l.toIntOption.map(Trail(n, _))
    case _           => None

class ShrinkProps extends ScalaCheckSuite:

  override def scalaCheckTestParameters: Test.Parameters =
    super.scalaCheckTestParameters.withMinSuccessfulTests(200).withWorkers(1)

  // Few characters, one of them the separator, so the bug is reachable.
  val nameGen: Gen[String] = Gen.stringOf(Gen.oneOf('a', 'b', 'c', ';'))

  property("round trip with arbitrary names (this one has a bug)"):
    forAll(nameGen, Gen.choose(0, 40_000)) { (name: String, length: Int) =>
      val t = Trail(name, length)
      Trail.parse(Trail.render(t)) == Some(t)
    }
Test run trailhead.ShrinkProps started
==> X trailhead.ShrinkProps.round trip with arbitrary names (this one has a bug) 0.02s munit.FailException: Shrink.test.scala:32
31:      Trail.parse(Trail.render(t)) == Some(t)
32:    }

Failing seed: R47IbNnrFPOcy0hODql7SQjDWY9MWToynqNR_AS8jvK=
You can reproduce this failure by adding the following override to your suite:

  override def scalaCheckInitialSeed = "R47IbNnrFPOcy0hODql7SQjDWY9MWToynqNR_AS8jvK="

Falsified after 1 passed tests.
> ARG_0: ";"
> ARG_1: 0
> ARG_1_ORIGINAL: 26840
Test run trailhead.ShrinkProps finished: 1 failed, 0 ignored, 1 total 0.029s
Error: munit tests failed.

Source: MUnit: ScalaCheck integration, Reproducing a property failure and ScalaCheck User Guide

The report is the point of the technique. The minimal counterexample is a name that is exactly ; and a length of 0; the original failing length was 26840, which shrank to 0 because the length did not matter. The "failing seed" is a string you paste into the suite (scalaCheckInitialSeed) to rerun exactly the same random sequence while you debug; the seed is different on every run unless you pin it.

Behaviour worth knowing:

  • Shrinking uses a Shrink[A] given. Built-in types have one; a type you define does not, so a failing Trail is not shrunk field by field unless you provide a Shrink[Trail] given. For a case class built by a generator, it is often easier to generate the components (forAll(nameGen, lengthGen)) than the class, as the suite above does.

  • Shrinking can produce values that the generator could never produce (for example a negative number from Gen.choose(1, 9)). Constrain inputs inside the property (==>) if that matters.

  • Without a generator that can hit the bug, nothing is found. The same property with the default String generator (forAll { (name: String, length: Int) => ... }) passed all 200 runs in a separate run on 2026-10-16.

Test parameters

scalaCheckTestParameters (shown above) returns a org.scalacheck.Test.Parameters. The commonly changed settings are:

Setting Meaning

withMinSuccessfulTests(n)

Number of passing runs required (default 100).

withMaxDiscardRatio(r)

How many discarded values per passing run are accepted before giving up (default 5).

withMinSize(n), withMaxSize(n)

Range of the size parameter that grows during a run; bounds the length of generated collections.

withWorkers(n)

Number of parallel workers; use 1 for properties with side effects.

scalaCheckInitialSeed (a suite member)

Pins the seed to reproduce a failure.

Properties in practice

  • Start with the cheapest shapes (round trip, invariant, idempotence) for every encoder, parser and normalizer. A round trip alone finds most escaping and boundary bugs.

  • A property that always passes is as dangerous as a test that asserts nothing. Check the generator (with classify/collect), and try breaking the code on purpose to see the property fail.

  • Keep properties deterministic apart from the seed: no clocks and no shared mutable state. Put that code behind an interface and use an in-memory fake.

  • Failures found by shrinking should become ordinary example tests (test("regression: a name with a separator")), so the case stays covered with a fixed, readable input.

  • Properties take longer than examples. Keep the default 100 runs locally and raise the number on the CI server, where a rare failure is worth hunting.

How it compares

On the JVM, Java projects test with JUnit (see Java Testing) and Groovy’s Spock offers data tables for example-driven parameterization. Scala’s version is the oldest of the family: ScalaCheck was inspired by Haskell’s QuickCheck and many other languages copied it. The Scala difference from a parameterized test is that the inputs are generated and shrunk, not listed.

Where the books are outdated

Older articles and books show the following forms.

  • Properties objects with include(...) and specify("name", forAll { ... }) for classic ScalaCheck. munit-scalacheck or the property integration of another framework is the usual entry now, although Properties still works.

  • import org.scalacheck.Prop.forAll combined with implicit val arbTrail: Arbitrary[Trail]. In Scala 3 write given Arbitrary[Trail] = ....

  • Pre-3.0 snippets using implicit arguments to forAll and other implicit-based ScalaCheck imports. The import used on this page is propBoolean, which provides ==>.

  • ScalaTest GeneratorDrivenPropertyChecks examples: those belong to ScalaTest (a different, third-party framework). They are not the MUnit route used here.

  • The Toolkit documentation lists MUnit only; ScalaCheck is not part of Scala Toolkit 0.9.2, so a property test always adds the munit-scalacheck line shown above.

References