Gradle Composite Builds

The manual names three situations. Each one is about a boundary that you do not want to turn into a single build.

Source: Gradle User Manual: Composite Builds

The best-practices chapter goes one step further and recommends an included build-logic build over buildSrc for build logic, because a change in an included build invalidates fewer tasks and because an included build is a complete build that can be opened and built alone. Source: Gradle User Manual: Favor build-logic Composite Builds

The page uses four sibling builds that are cut from the tidepool project of the other pages. The application lives in tidepool-app; the station model, which every other part uses, has been moved out into its own build, tidepool-model; a stand-in for a third-party library that we want to patch is a fourth build, tide-format-fork; and build-logic holds a tiny plugin. No build refers to another one by anything except a directory name in a settings file.

The layout on disk is the co-development layout of the manual: the included builds sit next to the consumer and are included temporarily or permanently by relative path.

Source: Gradle User Manual: Composite Build Layout

The library is an ordinary build with a settings file and a build file. The only thing that matters to a composite is that its project has a group, a name and a version that line up with what consumers declare.

Nothing in these files mentions the composite, and the build works alone: ./gradlew build run inside tidepool-model succeeds with the two tasks it has (compileJava, jar) and nothing else to configure. That independence is what the manual means by an included build being "a complete Gradle build".

These files are this page’s own setup for the examples, not an example taken from the manual.

Source: Gradle User Manual: Composite Builds, requirements for composite builds

The consumer is a normal application build. Its settings file lists the builds to include with Settings.includeBuild(Object); relative paths are resolved from the directory of the settings file that contains the call, and the path must point at the root directory of the included build, the directory that holds its settings file. Absolute paths work but the manual discourages them, because they tie the build to one machine.

The settings file has three kinds of includeBuild, and the rest of the page takes them in turn: a plain one for tidepool-model (substitution by group and name, Dependency substitution), one with a dependencySubstitution block for tide-format-fork (Declaring substitutions explicitly) and one inside pluginManagement for build-logic (Included plugin builds). The pluginManagement block must come first in a settings file, in both DSLs. The repositories are declared once in the settings file, as on the other Gradle pages; they apply to the consumer only.

The application’s build file declares its dependencies the way it would if the library had been published to a repository:

Neither io.tidepool:tidepool-model:1.0.0 nor org.example:tide-format:2.3 exists in any repository the build can reach; the build can only work because the composite provides them. Running it from a clean state on the Gradle 9.8.1 wrapper (with --configuration-cache) shows the included builds being built on demand, interleaved with the consumer’s own tasks:

The task paths show the structure. A task of the root build is :compileJava; a task of an included build is prefixed with the build’s path, :tidepool-model:compileJava. The consumer’s compileJava needs the library’s jar on its class path, so Gradle ran :tidepool-model:jar and everything that jar needs, without anyone asking for it. The example is compatible with the configuration cache (second run in Configuration cache and configuration on demand); the Groovy DSL build prints the same lines.

Source: Gradle User Manual: Defining a Composite Build via the Settings file

--include-build turns the build that is being run into a composite for one invocation, without editing any file. The settings file can then stay free of includeBuild calls, which suits a one-off check of a local library change. Here the includeBuild("../tidepool-model") line was temporarily removed from the settings file of the consumer, and the library was added on the command line instead (the fork and build-logic were still included by the settings file):

The flag can be given more than once (--include-build ../tidepool-model --include-build ../tide-format-fork listed both builds under "Included builds" in ./gradlew projects), and the path in the example is relative to the directory the command was run in. The same substitution happens as with the settings file. For plugins the manual prefers includeBuild in the settings file; see Included plugin builds.

Source: Gradle User Manual: Defining a Composite Build via --include-build

When a build is included, Gradle configures it to find out which external modules it can stand in for. By default it takes the group and the project name of every project in the included build and substitutes a project dependency for any external dependency that matches ${project.group}:${project.name} (the version is not part of the match). tidepool-model has group = "io.tidepool" and the root project name tidepool-model, so it replaces io.tidepool:tidepool-model with no further code. dependencyInsight says where a resolved module came from:

The header project ':tidepool-model' (by composite build) and the line io.tidepool:tidepool-model:1.0.0 -> project ':tidepool-model' are the substitution. Without the includeBuild line the same dependency is looked up in the repository and fails (the settings file was edited to remove the line for this run):

Two details follow from the rule. The version is ignored, so a consumer that asks for 1.0.0 gets whatever version the included build has; and by default only the included builds provide substitutions, not the root build, so subprojects of the consumer are not addressable as group:name unless the root includes itself with includeBuild(".").

Tasks of an included build can be run from the command line with their fully qualified path, build path first: ./gradlew :tidepool-model:jar. This works since Gradle 6.8, which introduced running included-build tasks directly. Build, project and task names can be abbreviated as on any other task path, and --exclude-task takes the qualified path too. A composite does not behave like a multi-project build in one respect that surprises people: a lifecycle task such as build runs in the root build only, and the included builds are built only as far as the root’s tasks need them. build in the consumer ran these tasks (the libraries' own build tasks did not run):

Only compile, processResources, classes and jar of the two libraries ran, because those were needed for the application’s class path. No :tidepool-model:build, :tidepool-model:check or :tidepool-model:test appears. To get that behaviour the composite must say so. Included builds are reached through Gradle.getIncludedBuilds() or Gradle.includedBuild(String), and IncludedBuild.task(String) yields a task reference that can be a dependency. Included builds cannot depend on each other this way, but the root can depend on all of them:

The line ... marks lines left out of the middle of the listing. Two things are worth reading in it. All the included builds' build tasks ran, and :build-logic:build is among them: gradle.includedBuilds also lists the plugin build that was included from pluginManagement, so a buildAll of this kind builds the build logic as well. Filter on the build name if that is not wanted. The manual also shows the same idea in one expression for a task with the same path in every included build (dependsOn(gradle.includedBuilds.map { it.task(":publishMavenPublicationToMavenRepository") })), for example to publish all the libraries of a monorepo. The order of unrelated tasks differs between runs and between the two DSLs, and that is normal.

./gradlew projects lists the included builds beside the projects of the root build:

Change a source file in the library and run the consumer again, and the edit is picked up with no publishing step. A comment was appended to Height.java of the library and the application was run with the configuration cache; the library is recompiled (its jar stays up to date because the class files did not change), and the cache entry is reused because the build’s configuration inputs did not change:

Gradle 9.1 fixed --dry-run for composites: before that release tasks of included builds could still run in the execution phase of a dry run. Tasks of included builds may still run during configuration if a build does that in its own logic. Since 4.1 included builds are executed in parallel, and since 4.10 included builds can be nested; a composite that includes another composite is flattened, so all the builds share one parent.

Source: Gradle User Manual: Executing Tasks, Gradle 6.8 release notes, Gradle 9.1.0 release notes, Gradle 4.1 release notes and Gradle 4.10 release notes

A build that provides plugins is included like any other build, but where it is included decides what the plugin can be applied to. For plugins applied in project build scripts, put includeBuild inside pluginManagement of the settings file (as in tidepool-app/settings.gradle(.kts) above); the plugin is then used from the plugins {} block by its id:

build-logic itself is a plain java-gradle-plugin build that registers tidepool.banner (gradlePlugin { plugins { register("banner") { ... } } } in the Kotlin DSL); how to write that kind of build, and how to share conventions through it, is the subject of Convention Plugins & Sharing Build Logic, Binary Plugins and Testing & Publishing Plugins; the plugins {} block and pluginManagement are on Plugins Overview. A settings plugin, which is applied in the settings file itself, must come from a build included in pluginManagement, because it is needed before projects are configured; the best-practices chapter suggests a separate minimal build for settings plugins so that this early inclusion does not hurt build caching.

The manual calls --include-build less reliable for plugins used in the plugins {} block, especially for unpublished plugins, and recommends the settings file. In this scratch composite the command line worked: with the pluginManagement block removed, ./gradlew --include-build ../build-logic banner printed the same == tidepool-app == and applied the plugin. One run does not make it reliable for every case (the manual warns about unpublished plugins in particular), so the settings file remains the recommended place.

Release history of the feature, each point from the release notes of that Gradle version: composite builds appeared in 3.1 with --include-build; 4.1 ran included builds in parallel and avoided executing included-build tasks twice; 4.10 allowed nesting; 5.1 allowed plugins from included builds in the plugins {} block; 6.8 allowed running their tasks from the command line and made the configuration cache work with composites; 7.0 allowed settings plugins from included builds with pluginManagement { includeBuild(...) }.

Source: Gradle User Manual: Gradle Plugins in Composite Builds, Gradle 5.1 release notes, Gradle 7.0 release notes and Gradle 3.1 release notes

Included builds are executed in isolation. They do not share repositories, plugins, plugin management, version catalogs or user-defined properties with the root or with each other. The settings file of the root build owns the one thing that is shared: the dependency substitution. A practical consequence is that a library can have its own repositories and plugin versions without the consumer knowing, and equally that the consumer’s dependencyResolutionManagement does not reach the library.

Properties show the rule most clearly. The consumer got a gradle.properties with greeting=hello, and both build files printed findProperty("greeting") during configuration. The first run reads the property from the file, the second passes it with -Pgreeting=cli:

The gradle.properties of the root is not visible to the included build, while -P and -D options reach every build. The manual adds that Gradle’s own runtime properties, such as org.gradle.configuration-cache, are read from the root build only and apply to the whole invocation; the same properties in an included build’s gradle.properties are ignored (the runtime-property half was not run here). To share credentials or common configuration across a composite, the manual suggests environment variables or a shared convention plugin included through pluginManagement. The properties are described in Build Environment & Properties.

Source: Gradle User Manual: Configuring a Composite Build

Every build has a build path, derived from the directory name of the included build, and every project is addressed by its build path plus its project path. Two rules keep the addressing unambiguous: no two included builds can have the same build path, and an included build path cannot clash with a project path of the root build. A second tidepool-model directory in another place collides with the first:

The fix is to give one of them a different name in the settings file:

The renaming snippet is from the manual and was not run here; the collision message above was. The name matters when you reference the build: in gradle.includedBuild("...") and in task paths such as :other-tidepool-model:jar.

Source: Gradle User Manual: Requirements for Composite Builds

The runs with --configuration-cache above stored an entry on the first run and reused it on the second (the "Reusing configuration cache" line in the source-edit run). Composites have worked with the configuration cache since Gradle 6.8, and the examples on this page, including the included builds and the substitution, are compatible with it. The cached entry stays valid when only source files of the included library change, as shown. It is not reused when a build file of an included build changes: after a line was appended to tidepool-model/build.gradle.kts the next run printed "Calculating task graph as configuration cache cannot be reused because file '../tidepool-model/build.gradle.kts' has changed." See Configuration Cache & Isolated Projects.

Configuration on demand is a different matter and not run here. The manual says it can hurt a composite: with the default rules Gradle has to inspect every project of an included build to find out which group and name it provides, so the included build is always fully configured; with explicit substitutions Gradle trusts them and configures the included build only when a substituted project is needed. The flip side is that with configuration on demand you must declare substitutions for every project of the included build you want to use, because projects without a rule are treated as external.

Source: Gradle User Manual: Configuration-on-Demand

An IDE imports a Gradle build through the Tooling API (see Init Scripts & the Tooling API), and the manual says composite builds "can be imported into the IDE" like a multi-project build. What an IDE shows (included builds as separate modules or projects next to the root, navigation from the consumer into the library’s sources, a refresh after editing includeBuild) depends on the IDE and its version, and none of it was run here. Two things are independent of the IDE and were run: each included build is a full build that can be opened alone (the standalone ./gradlew build in tidepool-model above), and a change in the library is visible to the consumer without publishing, so the edit-run loop works in the IDE the same way as on the command line. The Gradle 6.8 release notes add that IDE support for running included-build tasks directly could vary by IDE at that time, and suggested creating a Gradle run configuration that runs the task as on the command line. For the cross-tool view of IDE integration see CI & IDE Integration.

Source: Gradle User Manual: Interacting with a Composite Build and Gradle 6.8 release notes

Source: Gradle User Manual: Known Limitations of Composite Builds

Nothing on this page was run on Gradle 8.14.6; the notes below come from the official documentation and the release notes and are labelled "not run here".

Source: Gradle User Manual: Upgrading within Gradle 8.x and the release notes cited above

See Multi-Module Builds & the Reactor, Multi-File Builds: import, include & subant, Ivy: Publishing & Multi-Project Builds and the cross-tool view in Comparing Ant, Maven & Gradle.