Gradle: Performance Profiling & Build Scans

Every Gradle release carries performance work, and an old version misses it. Upgrading within a major version is low-risk because Gradle keeps backward compatibility between minor versions, and staying current makes the next major upgrade smoother because the deprecation warnings arrive early. The wrapper makes it a one-line change:

The JVM matters too: Gradle runs on it, and newer JDKs bring faster start-up and better garbage collectors. Gradle 9.0.0 raised the JVM needed to run Gradle to 17 (the code you build can still target older versions with release or a toolchain), and 9.8.0 supports Java 27 (see Versions, Compatibility & Release History). The third item is plugins: outdated ones, in particular the Android, Java and Kotlin plugins, can slow a build that a newer version would not. Look at every plugins { } line and check the Plugin Portal. The procedure for a real upgrade (deprecation warnings, --warning-mode=all, plugin updates) is on Upgrading Gradle, and the wrapper is on Gradle Wrapper.

Source: Gradle: Improve the Performance of Gradle Builds, Update Versions

By default Gradle runs one task at a time (a task graph in which several tasks are ready still executes them one after another). --parallel, or org.gradle.parallel=true in gradle.properties, lets tasks of different projects run at the same time, as far as their dependencies allow. The effect depends on the shape of the build: if one project dominates the time, or if the projects form a chain, there is little to gain. tidepool is a good case: tidepool-ports and tidepool-charts are independent of the model/engine/cli chain.

The Groovy DSL copy, run the same way, took 14, 8 and 18 seconds without the option and 8, 10 and 8 with it. The spread is the machine’s load, and the direction is the same as in the Kotlin copy. The speed-up is the expected one: with --parallel the build takes about as long as its longest chain (the model, then the engine and the CLI), not the sum of the three big compile tasks.

Parallel execution assumes that tasks of different projects do not share mutable state. A build that breaks with --parallel has a bug (two tasks writing the same file, a task reading a directory another one is still filling) that was hidden by the sequential order, and the fix is in the build, not in dropping the option. The number of worker threads defaults to the number of CPU cores (10 on this machine); --max-workers or org.gradle.workers.max sets it (see Build Environment & Properties).

Parallel execution also gives the Tooling API clients (the IDEs) parallel model building, which makes an IDE sync faster. Since Gradle 9.4.0 there is a separate property, org.gradle.tooling.parallel, to control that independently of task parallelism; the official page says to set it explicitly (relying on the value derived from org.gradle.parallel is deprecated). That detail is from the documentation and was not run here.

Source: Gradle: Improve the Performance of Gradle Builds, Enable Parallel Execution

The daemon has been on by default since Gradle 3.0 (2016-08-15), and the guide only mentions it for builds that switched it off (org.gradle.daemon=false in some gradle.properties). Every timing on this page was taken with a warm daemon; the first run after a daemon starts pays for the JVM start-up and the class loading (26 seconds for the first Kotlin DSL build against 14 to 16 afterwards). The mechanics, the commands that inspect and stop the daemon, and the memory rules are on The Gradle Daemon.

Source: Gradle: Improve the Performance of Gradle Builds, Re-enable the Gradle Daemon

The build cache stores task outputs keyed by the task’s inputs, so that a task that has already run with the same inputs (on this machine, or on a CI agent that shares the cache) is not run again, and its output is restored. It is off by default; --build-cache enables it for one build and org.gradle.caching=true for all. The incremental build already skips a task whose inputs did not change since its last run in this directory; the build cache also covers the cases where the output is gone (after clean) or the inputs changed and changed back. Details, the remote cache and the debugging of cache misses are on Build Cache.

On tidepool the cache was emptied first, and clean build --build-cache was run three times (the first run stores into the cache, the others use it). The tasks marked FROM-CACHE below come from the second run:

(Lines with NO-SOURCE, UP-TO-DATE and the lifecycle tasks are left out where the listing shows ....) Every compileJava and the test task came out of the cache. What still executed is worth a look, because it is the typical limit of a cache: generateTables is an ad-hoc task that was never declared cacheable, so it re-ran (it produces identical files, which is why the compile tasks that consume them still hit the cache), and the archive tasks (jar, distZip, distTar) ran too. A task is cacheable only when its type says so (the compile, test and many archive tasks of the core plugins do) or when a custom task type is annotated for it; the generator could be, with the annotation described on Writing Custom Tasks.

The Groovy copy took 9, 5 and 4 seconds, and reported the same numbers of tasks. The build cache was introduced as production-ready in Gradle 4.0, and Gradle in Action (written for 1.7) predates it.

Source: Gradle: Improve the Performance of Gradle Builds, Enable the Build Cache and Best Practices, Use the build cache

The configuration cache stores the task graph that the configuration phase built, so that a build whose configuration inputs did not change (build scripts, settings, init scripts, the properties and environment variables they read, files read through providers) skips the phase entirely. The official guide also says that with it enabled, Gradle runs all tasks in parallel, even within one project, and caches the dependency-resolution results. It is off by default in 9.8.1 (it is the "preferred mode" since 9.0, and the default is announced for Gradle 10); enable it for one build with --configuration-cache, or for all with org.gradle.configuration-cache=true. Its requirements (no Project at execution time, providers instead of eager reads) and Isolated Projects are on Configuration Cache & Isolated Projects.

Every example on this page is compatible with it (the generator takes the directory and the package name at configuration time and uses only those in doLast), and the runs below used it where the text says so. The store and the reuse are reported in the output:

The Groovy copy took 4, 8 and 4 seconds for the first command and 2, 2 and 2 for the second. Two things in these numbers deserve a sentence. First, the build itself is tiny, so the configuration phase (0.08 seconds in the profile above) was never the problem and the configuration cache gained little by skipping it; the difference to the baseline (9 to 12 seconds against 14 to 16) is mostly the parallel execution that comes with the cache. On a build with hundreds of projects and heavy plugins the weight is the other way round, and the configuration cache is then the biggest single saving. Second, the combination of the three is where the large drop is: with all three the clean build is three seconds, because the compile and test outputs come from the cache, the tasks that do run do so in parallel, and the configuration is read from the cache.

Source: Gradle: Improve the Performance of Gradle Builds, Enable the Configuration Cache, Gradle: Configuration Cache

Most core tasks are incremental: they declare their inputs and outputs, and Gradle skips them when nothing changed. A custom task participates only if it declares them. The generateTables task above does (an inputs.property and an outputs.dir), and without them it would run on every build and make everything downstream rerun. Run build twice without cleaning, and the second run is nearly free:

The Build Scan "Timeline" view, or the profile report’s task list, shows the tasks that reran when you expected a skip; the usual cause is an input that changes on every build (a time stamp, an absolute path). The mechanism, @Input annotations and the reasons a task is considered out of date are on Incremental Build.

Source: Gradle: Improve the Performance of Gradle Builds, Enable Incremental Build for Custom Tasks

The fastest task is the one that does not run. The guide’s advice is to give each kind of user a task (or a group of tasks) that does only what they need: a lifecycle task that depends on the right subset of tasks, tasks put in a task group so that ./gradlew tasks lists them, and verification deferred until it is wanted. In tidepool a developer who works on the CLI types ./gradlew :tidepool-cli:installDist, and the generated tables of tidepool-ports are never touched. Task naming and grouping are on Tasks.

Source: Gradle: Improve the Performance of Gradle Builds, Create Builds for Specific Developer Workflows

The daemon starts with a maximum heap of 512 MB by default, which is enough for most builds; very large builds keep a big model in memory and may need more. Raise it with org.gradle.jvmargs in gradle.properties of the project or of the Gradle User Home:

Do not raise it by habit: a bigger heap does not make a build faster unless it was short of memory, and a daemon that has to be started again with different arguments costs a start-up. To see what the daemon really got, a task can print it. It reads the value in its action, so it is configuration-cache compatible:

The Groovy DSL copy printed the same two lines. Changing org.gradle.jvmargs starts a second daemon (the command above left two idle daemons, which --status listed), because the JVM arguments are among the settings that must match for a daemon to be reused. The full list of daemon JVM settings is on The Gradle Daemon. The org.gradle.jvmargs value applies to the daemon only; the client JVM is configured through GRADLE_OPTS.

Source: Gradle: Improve the Performance of Gradle Builds, Increase the Heap Size

The configuration phase runs on every build, for every project, whatever tasks were asked for, so a second of work there is a second added to gradle help. The usual culprits, from the guide:

To see the last one, the engine’s script was given a line that prints its compile classpath, outside of any task, so that it runs on every build (a mistake that is easy to make while debugging a classpath):

Running ./gradlew help --profile prints the entries during configuration, in a build whose only task is help:

and the Dependency Resolution tab of that report names the configuration that was resolved, which the same build without the line does not list:

The times are tiny because the dependencies were already in the dependency cache and the build is small; the cost is I/O and grows with the number of configurations and the state of the cache. The tab’s list is the thing to watch: a help or tasks build should resolve nothing. The fix is to defer the resolution to execution time and to capture the file collection (not the project) so that the task stays compatible with the configuration cache:

Both DSLs gave the same output. The Build Scan has the same information on its "Performance" page ("Settings and suggestions" lists the dependencies resolved during configuration) and also shows the plugins and scripts in the order of the time they took to apply, which is where the "apply it only where needed" advice is found.

Source: Gradle: Improve the Performance of Gradle Builds, Optimize Configuration and Avoid dependency resolution during configuration

Resolution contacts remote servers, so the way dependencies are declared decides how often it does. The guide’s list, with the pages that cover each point:

The dynamic-version case has a side effect on the configuration cache that is easy to miss. A dynamic version is a configuration input: when its cached answer expires, the configuration cache entry cannot be reused, and the configuration phase runs again. To see this without waiting 24 hours, the engine was given the dynamic version 1.+ of Commons CSV and a cache time of zero seconds:

Running ./gradlew :tidepool-engine:compileJava --configuration-cache twice in a row, the second run could not reuse the entry:

The Groovy copy printed the same messages. With the default 24 hours the same message appears once a day, which is also the way to find the culprit in a real build: the message names the dynamic dependency. dependencyInsight shows which version a dynamic declaration selected:

The two values (cacheDynamicVersionsFor and cacheChangingModulesFor) should not be lowered in a build or an init script unless there is a reason: lower values make Gradle ask the repositories more often. The other fix is the one that follows from the line that found it: a fixed version, such as the 1.14.1 in the version catalog. Locking the dynamic versions that remain is on Dependency Locking, Verification & Caching.

Source: Gradle: Improve the Performance of Gradle Builds, Optimize dependency resolution and Minimize Dynamic and Snapshot Versions

For a project with the Java plugin the guide has three test suggestions and two compiler ones. The model’s script above shows all of them, and the build was run with each.

Running ./gradlew clean :tidepool-model:test (configuration cache on) with the Test block of the model produced no HTML report, and the XML report was absent too: only the binary results directory was left. Running it again with the property brought them back:

Both DSLs produced the same result with the forked compiler setting in place.

Source: Gradle: Improve the Performance of Gradle Builds, Run tests in parallel and Run the compiler as a separate process

After the measurements, the settings that earned their place in tidepool go into gradle.properties, which is the same for both DSLs:

(The heap line is only there to show where it goes: nothing in this build needs more than the default, and the profile showed no memory problem.) With these properties, ./gradlew clean build took 9 seconds the first time (some compiled outputs were already in the cache from earlier runs, and the configuration cache was stored) and 2 seconds for the next two runs in the Kotlin DSL copy, and 6, 1 and 2 seconds in the Groovy copy, against 14 to 16 seconds at the start. Anyone who clones the project gets the same settings; a developer or CI agent that needs different ones overrides them on the command line (--no-parallel, --no-build-cache) or in the Gradle User Home. Where the cache is shared across machines, who may write to it is a decision of its own (see Build Cache).

The command is the same since Gradle 4.3: add --scan.

At the end of the build Gradle prints the address of the scan. Without a server of your own, the build is published to Gradle’s public Build Scan service (known as scans.gradle.com in older material; the current documentation names it Develocity at gradle.com, and it redirects to docs.develocity.ai). That is where the data leaves your machine, and it requires accepting the terms of use, either when asked on the command line or in the build, with a line that the plugin documentation warns not to commit into a project that other people build:

Nothing on this page publishes a scan. Publishing a scan and accepting the terms of use were not run, and no example here contains the agreement. What the page did run is the plugin itself: --scan makes Gradle apply the Develocity plugin automatically (the version bundled with that Gradle release), but a project can also apply it explicitly, which is how a team normally configures it. The plugin id is com.gradle.develocity (version 4.6.0 is the latest at the time of writing, and works with all Gradle 5.x and later), and it must be applied in the settings file. The scratch copies of the build got this settings file in front of the lines shown earlier; publishing.onlyIf { false } makes publication on-demand only (the plugin’s default is to publish every build), and --no-scan turns it off for one build:

Run with --no-scan, the plugin loaded (it is downloaded from the Gradle Plugin Portal like any plugin), the build ran and was reused from the configuration cache the second time, so the plugin is compatible with it:

The Groovy copy gave the same results (10 seconds, then 585 ms). The plugin also adds a task, buildScanPublishPrevious, which publishes the data captured by the last build, useful for "I should have run that with --scan`". It was not run, but its description is real output of `help:

The plugin documentation also covers publishing only for failed builds, only when the user is authenticated, or only on CI, by giving publishing.onlyIf a predicate; these were not run here.

The plugin documentation lists the captured information in an appendix. In summary: the environment (user home, user name, host names and IP addresses, which can be obfuscated; the JVM and its settings; locale and time zone; operating system; hardware, memory and CPU, network and disk throughput), the build (the invocation options, the console output, failure messages and stack traces, deprecations, the project names and structure), the executed tasks (name, outcome, dependencies, cache operations and keys, input and output property names without their values), the executed tests (names, outcomes, durations, output, failures), the artifact transforms, the applied plugins, the resolved dependencies with their origin repositories, the network downloads, the build cache configuration and the Java toolchains used. The plugin documentation states what it does not capture: the source code being built and the output artifacts. It adds a caveat that matters for privacy: compiler errors and test failures are captured, and these messages can reveal parts of the source code. That is why the obfuscation block above exists (it rewrites the user name, the host names, the IP addresses or the process names before they are sent), and why a team that works on closed-source code should read the appendix before it enables publication, and decide where scans go.

If your organisation runs a Develocity installation, scans go there instead of to the public service, and are visible to all users who can reach the server. There are three ways to point a build at it, in increasing order of how much the project knows about it:

Authentication to the server (access keys) is described in the plugin manual. The commercial product also offers a shared remote build cache, test distribution and other features, which are beyond this section; the plugin manual at docs.develocity.ai/gradle/4.6 is the reference. None of this was run here.

Source: Gradle: Build Scan, Enable Build Scan, Publishing to a specific Develocity server, Captured Information, Develocity Gradle plugin, Connecting to Develocity at Gradle.com, Controlling When a Build Scan Is Published, Publishing Without Modifying Your Gradle Project, Obfuscating Identifying Data and Appendix B, Captured Information

This part of the page is from the official guide; no scan was published to look at. The "Performance" page of a scan shows the same stages as the profile report and some more: startup, configuration, dependency resolution and task execution, with tabs that break each down. The guide points to these views in particular: