Testing & Publishing Gradle Plugins

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

This page is written against Gradle 9.8.1 and was run on JDK 26 on 2026-10-16 (the daemon, the test JVMs and the TestKit builds all ran on JDK 26; the plugin is compiled with options.release = 17, the lowest JVM that runs Gradle 9). Every example was run through the wrapper in scratch copies of a stand-alone plugin project, once with Kotlin DSL build files and once with Groovy DSL build files; the plugin and test sources are Java and are the same in both, and the console output is the same apart from the timings and the order in which independent tasks are listed. The scratch copies pointed their wrapper at a local copy of the same distribution archive; the properties file that a real project commits is

distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionSha256Sum=dce76f55f8e251a3a1f130eb120f30b3d271de2b76c9b0729d316b5a1b6dc01f
distributionUrl=https\://services.gradle.org/distributions/gradle-9.8.1-bin.zip
networkTimeout=10000
retries=0
retryBackOffMs=500
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

The commands use --console=plain. In the listings the blank lines, the "Consider enabling configuration cache" hint, the link to the problems report and the Try: hints of failures are left out, and the scratch directory is shown as /work/tidepool. Examples run with --configuration-cache say so, and a line with ... marks omitted output. Gradle 8.14.x was not installed here and no other Gradle distribution was downloaded, so everything said about it, and everything about the Gradle Plugin Portal itself, comes from the official documentation and is labelled "not run here".

A plugin is code that other builds run, so it deserves the tests any library gets, with one difference: the interesting behaviour only exists inside a Gradle build. Gradle therefore ships two testing tools, ProjectBuilder for checking what a plugin configures and TestKit for running a real build that uses it, and the java-gradle-plugin plugin wires both. Once tested, the plugin is published like any other library, plus one extra publication that lets consumers write plugins { id("...") version "..." }. This page covers that whole path for the tidepool tidepool.station-check plugin. How to write the plugin is on Binary Plugins; the task it registers is the one built on Writing Custom Tasks; how plugins are applied by a consumer is on Plugins Overview. The Maven counterpart, a Mojo tested with the plugin testing harness and the Invoker Plugin, is on Writing Maven Plugins.

The plugin under test

The plugin is a separate project, because a plugin that is published is built and versioned on its own. Its layout:

plugin/
  settings.gradle.kts
  build.gradle.kts
  gradle.properties
  gradlew, gradlew.bat, gradle/wrapper/
  src/main/java/io/tidepool/gradle/
    StationCheckPlugin.java        registers the extension and the checkStations task
    StationCheckExtension.java     stationsFile, minStations
    CheckStationsTask.java         the cacheable task
  src/test/java/io/tidepool/gradle/
    StationCheckPluginTest.java    unit tests (ProjectBuilder)
  src/functionalTest/java/io/tidepool/gradle/
    StationCheckFunctionalTest.java        functional tests (TestKit)
    GradleVersionFunctionalTest.java       the same plugin on a chosen Gradle installation

The plugin class creates the stationCheck extension with conventions, registers the checkStations task lazily, and hooks it into check only when a build has a check task (that is, when the lifecycle-base plugin is present):

public class StationCheckPlugin implements Plugin<Project> {
    @Override
    public void apply(Project project) {
        StationCheckExtension ext = project.getExtensions().create("stationCheck", StationCheckExtension.class);
        ext.getStationsFile().convention(project.getLayout().getProjectDirectory().file("data/stations.csv"));
        ext.getMinStations().convention(1);

        TaskProvider<CheckStationsTask> check = project.getTasks().register("checkStations", CheckStationsTask.class, t -> {
            t.setGroup("verification");
            t.setDescription("Validates the tidepool station file.");
            t.getStationsFile().set(ext.getStationsFile());
            t.getMinStations().set(ext.getMinStations());
            t.getReport().set(project.getLayout().getBuildDirectory().file("reports/stations.txt"));
        });
        project.getPluginManager().withPlugin("lifecycle-base",
                p -> project.getTasks().named("check").configure(c -> c.dependsOn(check)));
    }
}

CheckStationsTask is an abstract class with @InputFile (PathSensitivity.NONE), @Input and @OutputFile properties, annotated @CacheableTask; its @TaskAction fails with a GradleException such as duplicate station id SF or expected at least 5 stations but found 3, and otherwise logs 3 stations OK. The reason for each annotation is on Writing Custom Tasks; here they matter because validatePlugins insists on them (see below).

The build file of the plugin project applies java-gradle-plugin (which applies java-library, adds gradleApi() to the compileOnlyApi configuration and registers the plugin descriptor, the plugin-under-test metadata and the validation) and maven-publish (for publishing). Two source sets of tests are declared with the JVM Test Suite plugin, which is the subject of the later page Testing JVM Projects; the functional suite is told to gradlePlugin.testSourceSets(...) so that TestKit finds the plugin.

Kotlin DSL
plugins {
    `java-gradle-plugin`
    `maven-publish`
}

group = "io.tidepool"
version = "0.1.0"

repositories {
    mavenCentral()
}

java {
    withSourcesJar()
    withJavadocJar()
}

tasks.withType<JavaCompile>().configureEach {
    options.release = 17
}

tasks.javadoc {
    (options as StandardJavadocDocletOptions).addStringOption("Xdoclint:none", "-quiet")
}

gradlePlugin {
    plugins {
        create("stationCheck") {
            id = "tidepool.station-check"
            implementationClass = "io.tidepool.gradle.StationCheckPlugin"
            displayName = "Tidepool station check"
            description = "Validates the tidepool station CSV file."
        }
    }
}

testing {
    suites {
        named<JvmTestSuite>("test") {
            useJUnitJupiter("6.1.3")
            dependencies {
                implementation(gradleApi())
            }
        }
        register<JvmTestSuite>("functionalTest") {
            useJUnitJupiter("6.1.3")
            dependencies {
                implementation(project())
            }
        }
    }
}

tasks.withType<Test>().configureEach {
    systemProperty("tidepool.gradleHome", providers.gradleProperty("gradleHome").getOrElse(""))
    testLogging {
        events("passed", "failed", "skipped")
        showStandardStreams = true
    }
}

gradlePlugin.testSourceSets(sourceSets["functionalTest"])

tasks.named("check") {
    dependsOn(testing.suites.named("functionalTest"))
}

publishing {
    repositories {
        maven {
            name = "local"
            url = uri(layout.buildDirectory.dir("repo"))
        }
    }
}
Groovy DSL
plugins {
    id 'java-gradle-plugin'
    id 'maven-publish'
}

group = 'io.tidepool'
version = '0.1.0'

repositories {
    mavenCentral()
}

java {
    withSourcesJar()
    withJavadocJar()
}

tasks.withType(JavaCompile).configureEach {
    options.release = 17
}

tasks.javadoc {
    options.addStringOption('Xdoclint:none', '-quiet')
}

gradlePlugin {
    plugins {
        stationCheck {
            id = 'tidepool.station-check'
            implementationClass = 'io.tidepool.gradle.StationCheckPlugin'
            displayName = 'Tidepool station check'
            description = 'Validates the tidepool station CSV file.'
        }
    }
}

testing {
    suites {
        test {
            useJUnitJupiter('6.1.3')
            dependencies {
                implementation gradleApi()
            }
        }
        functionalTest(JvmTestSuite) {
            useJUnitJupiter('6.1.3')
            dependencies {
                implementation project()
            }
        }
    }
}

tasks.withType(Test).configureEach {
    systemProperty 'tidepool.gradleHome', providers.gradleProperty('gradleHome').getOrElse('')
    testLogging {
        events 'passed', 'failed', 'skipped'
        showStandardStreams = true
    }
}

gradlePlugin.testSourceSets(sourceSets.functionalTest)

tasks.named('check') {
    dependsOn testing.suites.functionalTest
}

publishing {
    repositories {
        maven {
            name = 'local'
            url = layout.buildDirectory.dir('repo')
        }
    }
}

The pieces worth knowing:

  • options.release = 17 is not decoration. Without it the plugin is compiled for the JDK that runs the build (26 here), the published Gradle Module Metadata says org.gradle.jvm.version: 26, and a consumer on JDK 17 cannot resolve the plugin. Gradle 9 runs on JVM 17 and later, so 17 is the lowest useful target.

  • The javadoc setting only silences "no comment" warnings of the small example; real plugins document their public types.

  • useJUnitJupiter("6.1.3") adds the JUnit Jupiter API and engine (and, since Gradle 9 requires it, the platform launcher) to the suite; JUnit 6 needs JVM 17 or later.

  • systemProperty("tidepool.gradleHome", ...) and testLogging apply to both test tasks: the first lets a test receive the directory of a Gradle installation from -PgradleHome=... (used below), and the second prints one line per test.

  • publishing.repositories points at build/repo, a file repository inside the project, so that publishing is safe to run anywhere.

Source: Gradle User Manual: Java Gradle Plugin, Usage and Java Gradle Plugin, Tasks

Three test levels

The Gradle manual maps the testing pyramid onto plugins like this:

Level Tool What it can show

Unit test

Any JVM test framework, plus ProjectBuilder for code that needs a Project

That the plugin registers an extension and tasks, that conventions and wiring are right. ProjectBuilder builds an in-memory Project; it does not run tasks, so it checks configuration only.

Integration test

Any JVM test framework

That a class works with the outside world it talks to (files, HTTP, a database). It does not need Gradle at all.

Functional test

Any JVM test framework, plus TestKit (GradleRunner)

That a real build applies the plugin, runs its tasks, and produces the files, outcomes and messages a user sees.

Source: Gradle User Manual: Testing Gradle Plugins, Tooling support

The practical advice of the manual is to push as much logic as possible out of classes that use the Gradle API into plain classes with plain unit tests, and to keep a few functional tests for the wiring. Manual testing, which the manual lists first, is done by using the plugin from a composite build (see Composite Builds) while it is being written.

The two Gradle tools run in different processes, which decides what each can see:

flowchart LR subgraph tj["Test JVM (the test task, JUnit)"] unit["Unit test: ProjectBuilder, an in-memory Project (no task actions run)"] gr["Functional test: GradleRunner"] end meta["plugin-under-test-metadata.properties (written by pluginUnderTestMetadata)"] -->|withPluginClasspath| gr gr -->|Tooling API| daemon["TestKit daemon: runs a real build"] daemon --> proj["Temporary project: settings.gradle, build.gradle, data/stations.csv"] daemon --> res["BuildResult: output, task outcomes"] res --> gr

Because the build runs in a separate process, a breakpoint in the plugin is not hit by a test that merely runs in debug mode; TestKit has a debug option for that (GradleRunner.withDebug(true) or the org.gradle.testkit.debug system property).

Source: Gradle User Manual: TestKit, Debugging build logic

Unit tests with ProjectBuilder

ProjectBuilder.builder().build() returns a Project that exists only in memory, with no build file and no task execution. Applying the plugin by id exercises the plugin descriptor too, which a test that instantiates the plugin class does not:

class StationCheckPluginTest {
    @Test
    void registersTaskAndExtensionWithConventions() {
        Project project = ProjectBuilder.builder().build();
        project.getPluginManager().apply("tidepool.station-check");

        assertNotNull(project.getExtensions().findByName("stationCheck"));
        CheckStationsTask task = (CheckStationsTask) project.getTasks().getByName("checkStations");
        assertEquals(1, task.getMinStations().get());
        assertTrue(task.getStationsFile().get().getAsFile().getPath().endsWith("data/stations.csv"));
    }

    @Test
    void extensionValuesFlowIntoTheTask() {
        Project project = ProjectBuilder.builder().build();
        project.getPluginManager().apply(StationCheckPlugin.class);
        StationCheckExtension ext = project.getExtensions().getByType(StationCheckExtension.class);
        ext.getMinStations().set(5);

        CheckStationsTask task = project.getTasks().withType(CheckStationsTask.class).getByName("checkStations");
        assertEquals(5, task.getMinStations().get());
    }

    @Test
    void checkDependsOnCheckStationsWhenLifecycleBaseIsApplied() {
        Project project = ProjectBuilder.builder().build();
        project.getPluginManager().apply("base");
        project.getPluginManager().apply("tidepool.station-check");

        assertTrue(project.getTasks().getByName("check").getTaskDependencies().getDependencies(null)
                .stream().anyMatch(t -> t.getName().equals("checkStations")));
    }
}

The second test shows what unit tests are good at: the extension value is read by the task only through a lazy Property, so setting it after the plugin was applied still reaches the task (see Lazy Configuration & Providers). The third shows the withPlugin reaction: with base applied the dependency exists, without it the plugin does not touch check.

Run the suite (Kotlin DSL build file, with --configuration-cache; the Groovy DSL build file printed the same lines apart from the order of two independent tasks):

$ ./gradlew test --configuration-cache
Calculating task graph as no cached configuration is available for tasks: test
> Task :processTestResources NO-SOURCE
> Task :pluginDescriptors
> Task :processResources
> Task :compileJava
> Task :classes
> Task :pluginUnderTestMetadata
> Task :compileTestJava
> Task :testClasses
> Task :test
StationCheckPluginTest > checkDependsOnCheckStationsWhenLifecycleBaseIsApplied() PASSED
StationCheckPluginTest > extensionValuesFlowIntoTheTask() PASSED
StationCheckPluginTest > registersTaskAndExtensionWithConventions() PASSED
BUILD SUCCESSFUL in 5s
6 actionable tasks: 6 executed
Configuration cache entry stored.

The order of the test methods is not deterministic. ProjectBuilder has limits that the manual states plainly: it runs no tasks and is not the real configuration lifecycle, so it cannot tell whether the plugin works with the configuration cache, with a real settings file or with plugins {} resolution. That is the job of the next level.

Source: Gradle User Manual: Testing Gradle Plugins, Implementing unit tests and Gradle Javadoc: ProjectBuilder

The unit test cannot see the Gradle API

The first version of the build file above had no implementation(gradleApi()) in the test suite, and the unit tests failed before running a single test:

> Task :test FAILED
JUnit Jupiter > initializationError FAILED
    org.junit.platform.launcher.core.DiscoveryIssueException
1 test completed, 1 failed
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///work/tidepool/build/reports/tests/test/index.html
BUILD FAILED in 3s

The cause is in the XML test report, not on the console: Cause: java.lang.NoClassDefFoundError: org/gradle/api/Task while JUnit inspected StationCheckPluginTest. java-gradle-plugin puts gradleApi() on compileOnlyApi, which is on the compile classpath of the tests but not on their runtime classpath. By default the plugin adds gradleTestKit() (which brings the Gradle API with it) to the test source set, and that is what normally hides the problem; but once gradlePlugin.testSourceSets(functionalTest) points TestKit to another source set, as in this build, the test suite is no longer given it. A run of the same build without the testSourceSets line and without gradleApi() passed. Declaring implementation(gradleApi()) for the unit suite is the explicit fix.

Functional tests with TestKit

TestKit is GradleRunner: it runs a build in a project directory with the arguments you give, in a separate Gradle daemon (a dedicated daemon that is not shared with your normal builds and is stopped when the tests finish), and returns a BuildResult. Add gradleTestKit() to the dependencies, or let java-gradle-plugin add it to the source sets named in testSourceSets, which is what this build does for functionalTest.

class StationCheckFunctionalTest {
    @TempDir
    Path projectDir;

    @BeforeEach
    void setUp() throws IOException {
        Files.writeString(projectDir.resolve("settings.gradle"), "rootProject.name = 'demo'\n");
        Files.writeString(projectDir.resolve("build.gradle"), """
                plugins {
                    id 'base'
                    id 'tidepool.station-check'
                }
                stationCheck { minStations = 2 }
                """);
        Files.createDirectories(projectDir.resolve("data"));
        Files.writeString(projectDir.resolve("data/stations.csv"), """
                id,name,latitude,longitude,constituents
                SF,San Francisco,37.8063,-122.4659,M2:0.55:120
                BOS,Boston,42.3539,-71.0503,M2:1.35:15
                """);
    }

    private GradleRunner runner(String... args) {
        return GradleRunner.create()
                .withProjectDir(projectDir.toFile())
                .withPluginClasspath()
                .withArguments(args);
    }

    @Test
    void checksAValidFile() throws IOException {
        BuildResult result = runner("checkStations", "--console=plain").build();

        assertEquals(TaskOutcome.SUCCESS, result.task(":checkStations").getOutcome());
        assertTrue(result.getOutput().contains("2 stations OK"));
        assertEquals("2 stations OK" + System.lineSeparator(),
                Files.readString(projectDir.resolve("build/reports/stations.txt")));
    }

    @Test
    void isUpToDateOnTheSecondRun() {
        runner("checkStations").build();
        BuildResult second = runner("checkStations").build();

        assertEquals(TaskOutcome.UP_TO_DATE, second.task(":checkStations").getOutcome());
    }

    @Test
    void failsOnADuplicateId() throws IOException {
        Files.writeString(projectDir.resolve("data/stations.csv"), """
                id,name,latitude,longitude,constituents
                SF,San Francisco,37.8063,-122.4659,M2:0.55:120
                SF,Again,1,2,M2:1:1
                """);

        BuildResult result = runner("checkStations").buildAndFail();

        assertEquals(TaskOutcome.FAILED, result.task(":checkStations").getOutcome());
        assertTrue(result.getOutput().contains("duplicate station id SF"));
    }
}

The elements of the API, in the order the test uses them:

Element Meaning

GradleRunner.create()

A new runner. By default it runs the Gradle distribution that loaded the GradleRunner class, which, in a test task run by Gradle, is the distribution that runs the build: the plugin is tested on the version it is built with.

withProjectDir(File)

The directory with the test build. Use a temporary directory per test (@TempDir in JUnit 5 and 6, @TempDir in Spock) and write settings.gradle(.kts), the build file and the inputs into it.

withPluginClasspath()

Injects the plugin under test into the build, so that plugins { id 'tidepool.station-check' } resolves without a repository and without a version. See the next section.

withArguments(...)

The command line: task names and options, exactly as typed after ./gradlew.

build() / buildAndFail()

Run and expect success or failure; the other outcome throws UnexpectedBuildFailure or UnexpectedBuildSuccess with the full output in its message. run() does not check.

BuildResult

getOutput() (the console output), task(":path") and getTasks() (with getOutcome(): SUCCESS, FAILED, UP_TO_DATE, SKIPPED, FROM_CACHE, NO_SOURCE), and taskPaths(outcome).

forwardOutput()

Also prints the build’s output to the test’s standard output, useful when debugging a failing test.

Source: Gradle User Manual: TestKit, Functional testing with the Gradle Runner and Gradle Javadoc: GradleRunner

Run the whole verification (check runs the unit tests, validates the plugin and runs the functional tests), Kotlin DSL build file, with the configuration cache:

$ ./gradlew check --configuration-cache -PgradleHome=/opt/gradle-9.8.1
Calculating task graph as no cached configuration is available for tasks: check
> Task :processFunctionalTestResources NO-SOURCE
> Task :processTestResources NO-SOURCE
> Task :pluginDescriptors
> Task :processResources
> Task :compileJava
> Task :classes
> Task :pluginUnderTestMetadata
> Task :jar
> Task :compileFunctionalTestJava
> Task :functionalTestClasses
> Task :compileTestJava
> Task :testClasses
> Task :validatePlugins
> Task :functionalTest
GradleVersionFunctionalTest > runsOnTheGivenGradleInstallation() STANDARD_OUT
    Gradle 9.8.1
> Task :test
StationCheckPluginTest > checkDependsOnCheckStationsWhenLifecycleBaseIsApplied() PASSED
StationCheckPluginTest > extensionValuesFlowIntoTheTask() PASSED
StationCheckPluginTest > registersTaskAndExtensionWithConventions() PASSED
> Task :functionalTest
GradleVersionFunctionalTest > runsOnTheGivenGradleInstallation() PASSED
StationCheckFunctionalTest > isRestoredFromTheBuildCacheAfterClean() PASSED
StationCheckFunctionalTest > isUpToDateOnTheSecondRun() PASSED
StationCheckFunctionalTest > checksAValidFile() PASSED
StationCheckFunctionalTest > failsOnADuplicateId() PASSED
StationCheckFunctionalTest > streamsTheOutputWithoutBufferingIt() PASSED
> Task :check
BUILD SUCCESSFUL in 9s
10 actionable tasks: 10 executed
Configuration cache entry stored.

The Groovy DSL build file printed the same lines. The -PgradleHome= value was the directory of the unpacked 9.8.1 distribution in the scratch area, shown here as /opt/gradle-9.8.1; it is explained below. :functionalTest appears twice because its header is printed again when the output of another task comes in between, and the order of such lines is not deterministic. Both test tasks ran, the build is compatible with the configuration cache (a second run reuses the entry), and the functional suite has three more tests, discussed below, in addition to the three shown.

How the plugin gets into the test build

The test build is a separate build in a temporary directory, with its own classpath: it does not share a class loader with the test JVM, so the plugin classes are not visible to it by themselves. withPluginClasspath() closes that gap:

  1. java-gradle-plugin registers the pluginUnderTestMetadata task. It writes plugin-under-test-metadata.properties, a file with an implementation-classpath entry that lists the plugin’s class directories and runtime dependencies (the main source set’s output and its runtime classpath).

  2. The same plugin adds that file to the classpath of the testSourceSets (and the gradleTestKit() dependency).

  3. withPluginClasspath() with no argument reads the file from the test classpath and puts those entries on the classpath of the plugin resolution of the test build. Because the plugin descriptor META-INF/gradle-plugins/tidepool.station-check.properties is among them, the plugins {} block finds the id.

The metadata file of this build contains one line, with the project’s paths:

implementation-classpath=/work/plugin/build/classes/java/main\:/work/plugin/build/resources/main

and the descriptor in the plugin JAR is

implementation-class=io.tidepool.gradle.StationCheckPlugin

Two limits of this mechanism are in the manual. It works only when the test build applies the plugin with the plugins {} block (not with apply plugin:), and it does not work for a plugin that depends on another plugin’s classes with compileOnly: the injected classpath lacks those classes and the test fails with a ClassNotFoundException. For that case publish the plugin to a local repository first and resolve it from there, as in the publishing section. In an IDE, run assemble once so that the metadata exists.

Source: Gradle User Manual: TestKit, Getting the plugin-under-test into the test build and Automatic injection with the Java Gradle Plugin Development plugin

Forgetting withPluginClasspath

Removing .withPluginClasspath() from the runner turns every functional test into a failing build, and the message is the one a user gets when a plugin id cannot be resolved:

org.gradle.testkit.runner.UnexpectedBuildFailure: Unexpected build execution failure in /tmp/junit1 with arguments [checkStations]

Output:

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/junit1/build.gradle' line: 1

* What went wrong:
Plugin [id: 'tidepool.station-check'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Included Builds (No included builds contain this plugin)
- Plugin Repositories (plugin dependency must include a version number for this source)

BUILD FAILED in 5s

The temporary directory of the real run was a longer path under the system temp directory and the lines after BUILD FAILED were trimmed; the rest is verbatim. "plugin dependency must include a version number" is the hint: a plugin resolved from a repository needs version, a plugin on the injected classpath must not have one.

The build environment of a test build

TestKit runs each build in an isolated environment. The directory is a fresh one under java.io.tmpdir, so the user’s ~/.gradle/gradle.properties, init scripts and settings are not used (a mirror or credentials configured only there will not reach the test build, and a test that needs external dependencies must configure repositories itself). TestKit keeps its working storage in a directory of its own, which also serves as the Gradle user home of the test builds unless a test passes -g; you can place it with the org.gradle.testkit.dir system property or GradleRunner.withTestKitDir(File), for example inside the project’s build directory. It is shared by the tests, so state such as the local build cache survives between tests. The manual says that TestKit does not currently let you select the JDK of the test build.

Source: Gradle User Manual: TestKit, Controlling the build environment and Gradle Javadoc: GradleRunner

Testing the build cache

CheckStationsTask is @CacheableTask, so there is a test that the cache really restores it. It overwrites settings.gradle to put the local build cache inside the temporary project (the manual’s advice: the TestKit user home is shared, so a shared cache directory would leak results from one test to the next), runs the task with --build-cache, runs clean, and runs it again:

@Test
void isRestoredFromTheBuildCacheAfterClean() throws IOException {
    Files.writeString(projectDir.resolve("settings.gradle"), """
            rootProject.name = 'demo'
            buildCache {
                local { directory = file('local-cache') }
            }
            """);

    BuildResult first = runner("checkStations", "--build-cache").build();
    assertEquals(TaskOutcome.SUCCESS, first.task(":checkStations").getOutcome());

    runner("clean", "--build-cache").build();
    BuildResult second = runner("checkStations", "--build-cache").build();
    assertEquals(TaskOutcome.FROM_CACHE, second.task(":checkStations").getOutcome());
}

It passes (see the listing above). This is the cheapest way to find out that a task annotated @CacheableTask is missing an input: a cache restore that should not happen, or one that does not happen, shows up as the wrong TaskOutcome. The cache itself is on the later page Build Cache, and the up-to-date part on Incremental Build.

Source: Gradle User Manual: TestKit, Testing with the Build Cache

Streaming the output since Gradle 9.3

BuildResult.getOutput() returns the complete console output as one String, which costs memory for a verbose build. Gradle 9.3.0 (2026-01-16) added BuildResult.getOutputReader() (incubating), a BufferedReader over the output that the caller must close:

@Test
void streamsTheOutputWithoutBufferingIt() throws IOException {
    BuildResult result = runner("checkStations").build();
    try (var reader = result.getOutputReader()) {
        assertTrue(reader.lines().anyMatch(l -> l.contains("2 stations OK")));
    }
}

On Gradle 8.14 the method does not exist (not run here); a test that must run against both lines uses getOutput().

Source: Gradle 9.3.0 Release Notes, Stream TestKit output

Testing against several Gradle versions

By default the plugin is tested with the Gradle that builds it. A published plugin is used by builds on other versions, so the manual shows a test that repeats the same build over a list of versions, with three runner methods:

Method What it does

withGradleVersion("8.14.6")

Uses that release. Unless it was downloaded before, the distribution is downloaded from Gradle’s distribution servers and cached beneath the Gradle user home (gradle.user.home, GRADLE_USER_HOME or ~/.gradle).

withGradleInstallation(File)

Uses an unpacked distribution on disk. Nothing is downloaded.

withGradleDistribution(URI)

Uses a distribution archive at a URI, for example a company mirror or a file: URL.

Source: Gradle User Manual: TestKit, Setting the Gradle version used to test

Only Gradle 9.8.1 was available here, so the runnable example uses withGradleInstallation. The test reads the directory from a system property, which the build file fills from -PgradleHome=..., and is skipped when the property is empty so that the normal build needs no installation:

@Test
void runsOnTheGivenGradleInstallation() throws IOException {
    String home = System.getProperty("tidepool.gradleHome", "");
    assumeFalse(home.isEmpty(), "no -PgradleHome=<installation directory> given");
    // ... write settings.gradle, build.gradle (plugins { id 'tidepool.station-check' }) and data/stations.csv ...

    GradleRunner runner = GradleRunner.create()
            .withProjectDir(projectDir.toFile())
            .withGradleInstallation(new File(home))
            .withPluginClasspath();

    BuildResult version = runner.withArguments("--version").build();
    System.out.println(version.getOutput().lines().filter(l -> l.startsWith("Gradle ")).findFirst().orElse("?"));

    BuildResult result = runner.withArguments("checkStations").build();
    assertEquals(TaskOutcome.SUCCESS, result.task(":checkStations").getOutcome());
    assertTrue(result.getOutput().contains("1 stations OK"));
}

In the check run above it printed Gradle 9.8.1 and passed; without -PgradleHome it is reported as SKIPPED. The same test, with withGradleVersion("8.14.6") in place of the installation (not run here), would be the form to run against the last 8.x release. Three things are worth planning for, all from the official documentation and not tried here: the TestKit client is compatible with JVM 8, but the Gradle it starts must run on a JVM it supports, and Gradle 8.14 runs on JVM versions up to 24 (JVM 25 needs Gradle 9.1.0 and JVM 26 needs 9.4.0), so the test task has to run on a JDK no newer than 24 when it targets 8.14, which a javaLauncher from a toolchain on the Test task provides, while the JDK 26 of this page cannot; a plugin written for Gradle 9 APIs, such as the lazy Property types used by the task, may not exist in an older Gradle, so compatibility has to be a decision (the plugin’s minimum Gradle version) and the test list should start at it; and the first run of each version downloads a distribution into the Gradle user home. The release history behind these limits is on Versions, Compatibility & Release History.

validatePlugins

java-gradle-plugin also registers validatePlugins and makes check depend on it. It reads the public task and artifact transform types of the project and fails when their properties are annotated wrongly or not at all. This is the same check that Gradle performs at runtime for any task (see Incremental Build), but done at build time of the plugin so that a user never meets the problem. The first version of CheckStationsTask had no @CacheableTask, and check failed:

> Task :compileJava UP-TO-DATE
> Task :pluginDescriptors UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :validatePlugins FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':validatePlugins' (registered by plugin 'org.gradle.java-gradle-plugin').
> Plugin validation failed with 1 problem
  Not cacheable without reason
    Type 'io.tidepool.gradle.CheckStationsTask' must be annotated either with @CacheableTask or with @DisableCachingByDefault
      The task author should make clear why a task is not cacheable
      For more information, please refer to https://docs.gradle.org/9.8.1/userguide/validation_problems.html#disable_caching_by_default in the Gradle documentation.
      Possible solutions:
        1. Add @DisableCachingByDefault(because = ...).
        2. Add @CacheableTask.
        3. Add @UntrackedTask(because = ...).
BUILD FAILED in 1s
4 actionable tasks: 1 executed, 3 up-to-date

The three solutions are the three honest answers: the task can be cached (@CacheableTask, which needs all of its inputs and outputs declared), it can opt out of caching with a reason (@DisableCachingByDefault), or it can opt out of up-to-date checking and caching altogether (@UntrackedTask). The unit and functional tests above did not notice the omission, which is why the validation is a separate step. The other rule it enforces is that every getter of a task type has an input, output or @Internal/@Console/@Nested annotation.

Source: Gradle User Manual: Java Gradle Plugin, Tasks and Validation Problems, Disable caching by default

Integration tests and manual tests

An integration test for a plugin is an ordinary test of a class that talks to the outside world: a parser that reads a file, a client that makes an HTTP call. It needs no Gradle at all, and the manual’s own example calls a URL, which makes the need for a network a property of the test, not of the plugin. Separate it from the unit tests by source set and suite (src/integrationTest), so that test stays fast and offline, the way functionalTest is separate here. Nothing of this kind is in the example plugin, because its only external resource is a CSV file, and for that file the functional tests above are the integration tests.

Manual testing, the base of the manual’s pyramid, uses the plugin from another build while you change it. The convenient way is a composite build: the consumer includes the plugin build with includeBuild in pluginManagement, and a change of the plugin is used by the next build of the consumer without publishing (see Composite Builds and the build-logic pattern of Convention Plugins & Sharing Build Logic).

Source: Gradle User Manual: Testing Gradle Plugins, Implementing integration tests and Best Practices for Testing: Test your custom Task and Plugins with TestKit

The best-practices page says it briefly: prototype a task or plugin in a build file, extract it to buildSrc or a plugin project when it stabilises, and from then on keep functional tests for it with TestKit.

Publishing the plugin to a Maven repository

A Gradle plugin is a library plus a way to resolve it from the plugins {} block. The block does not take a dependency notation: it takes a plugin id and a version, and Gradle looks for an artifact named after the id, the plugin marker. When java-gradle-plugin and maven-publish are both applied, java-gradle-plugin configures two kinds of publication (in an afterEvaluate hook, and only if you have not defined them yourself):

  • a main publication named pluginMaven, built from the java component (the JAR, sources, Javadoc, POM and Gradle Module Metadata), at group:artifactId:version, here io.tidepool:tidepool-station-check-plugin:0.1.0;

  • a marker publication for every plugin declared in gradlePlugin {}, named <pluginName>PluginMarkerMaven, here stationCheckPluginMarkerMaven. It is a POM-only artifact with groupId equal to the plugin id and artifactId equal to the id plus .gradle.plugin, here tidepool.station-check:tidepool.station-check.gradle.plugin:0.1.0, whose only dependency is the main artifact.

Setting automatedPublishing = false in gradlePlugin {} turns the automation off. The build file already has a repository named local that points at build/repo, so publish writes there (Kotlin DSL; the Groovy run printed the same):

$ ./gradlew publish --configuration-cache
Calculating task graph as no cached configuration is available for tasks: publish
> Task :pluginDescriptors UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :compileJava UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :generatePomFileForPluginMavenPublication
> Task :generatePomFileForStationCheckPluginMarkerMavenPublication
> Task :jar UP-TO-DATE
> Task :sourcesJar
> Task :publishStationCheckPluginMarkerMavenPublicationToLocalRepository
> Task :javadoc
> Task :javadocJar
> Task :generateMetadataFileForPluginMavenPublication
> Task :publishPluginMavenPublicationToLocalRepository
> Task :publish
BUILD SUCCESSFUL in 4s
12 actionable tasks: 8 executed, 4 up-to-date
Configuration cache entry stored.

The repository that came out (the .md5, .sha1, .sha256 and .sha512 checksum files that sit next to each of these files are left out):

build/repo/
  io/tidepool/tidepool-station-check-plugin/
    maven-metadata.xml
    0.1.0/
      tidepool-station-check-plugin-0.1.0.jar
      tidepool-station-check-plugin-0.1.0-sources.jar
      tidepool-station-check-plugin-0.1.0-javadoc.jar
      tidepool-station-check-plugin-0.1.0.module
      tidepool-station-check-plugin-0.1.0.pom
  tidepool/station-check/tidepool.station-check.gradle.plugin/
    maven-metadata.xml
    0.1.0/
      tidepool.station-check.gradle.plugin-0.1.0.pom

The marker is the file that makes the plugins {} block work:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>tidepool.station-check</groupId>
  <artifactId>tidepool.station-check.gradle.plugin</artifactId>
  <version>0.1.0</version>
  <packaging>pom</packaging>
  <name>Tidepool station check</name>
  <description>Validates the tidepool station CSV file.</description>
  <dependencies>
    <dependency>
      <groupId>io.tidepool</groupId>
      <artifactId>tidepool-station-check-plugin</artifactId>
      <version>0.1.0</version>
    </dependency>
  </dependencies>
</project>

Its name and description come from displayName and description in gradlePlugin {}; its dependency leads to the main artifact. The main POM is empty apart from the coordinates (with Gradle’s marker comment that the .module file should be preferred), and the .module file carries the real model: variants apiElements, runtimeElements and the sources and Javadoc variants, with org.gradle.jvm.version set to 17 thanks to options.release = 17 (it was 26 before that line was added). Gradle resolves the plugin from the .module file, Maven-only tools from the POM. The same repository layout is described for libraries on the later page Publishing, and the cross-tool view of repositories on Repositories & Publishing Across Tools.

Source: Gradle User Manual: Java Gradle Plugin, Maven Publish Plugin and Publishing a project as module (Maven Publish plugin)

Using the published plugin

A consumer finds the plugin by naming the repository in pluginManagement in its settings file, then applies it by id and version. The consumer sits next to the plugin project here, so the repository path is relative. It applies base too, which is what gives it a check task and so activates the withPlugin hook of the plugin.

Kotlin DSL (settings.gradle.kts and build.gradle.kts)
// settings.gradle.kts
pluginManagement {
    repositories {
        maven {
            url = uri("../plugin/build/repo")
        }
        gradlePluginPortal()
    }
}

rootProject.name = "tidepool-consumer"

// build.gradle.kts
plugins {
    base
    id("tidepool.station-check") version "0.1.0"
}

stationCheck {
    minStations = 3
}
Groovy DSL (settings.gradle and build.gradle)
// settings.gradle
pluginManagement {
    repositories {
        maven {
            url = uri('../plugin/build/repo')
        }
        gradlePluginPortal()
    }
}

rootProject.name = 'tidepool-consumer'

// build.gradle
plugins {
    id 'base'
    id 'tidepool.station-check' version '0.1.0'
}

stationCheck {
    minStations = 3
}

Both consumers behave the same, with the configuration cache, first run and second run:

$ ./gradlew check --configuration-cache
Calculating task graph as no cached configuration is available for tasks: check
> Task :checkStations
3 stations OK
> Task :check
BUILD SUCCESSFUL in 629ms
1 actionable task: 1 executed
Configuration cache entry stored.

$ ./gradlew check --configuration-cache
Reusing configuration cache.
> Task :checkStations UP-TO-DATE
> Task :check UP-TO-DATE
BUILD SUCCESSFUL in 538ms
1 actionable task: 1 up-to-date
Configuration cache entry reused.

The consumer’s check ran checkStations because of the withPlugin("lifecycle-base") hook, and the cached entry was reused, which is the practical meaning of "the plugin is compatible with the configuration cache". gradlePluginPortal() is listed so that other plugins keep resolving: declaring pluginManagement { repositories { ... } } replaces the default (the Portal), it does not add to it. Two failures show the other side. Without the repository in pluginManagement the plugin cannot be found, and the message lists where Gradle looked:

FAILURE: Build failed with an exception.
* Where:
Build file '/work/tidepool/build.gradle.kts' line: 1
* What went wrong:
Plugin [id: 'tidepool.station-check', version: '0.1.0'] was not found in any of the following sources:
- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Included Builds (No included builds contain this plugin)
- Plugin Repositories (could not resolve plugin artifact 'tidepool.station-check:tidepool.station-check.gradle.plugin:0.1.0')
  Searched in the following repositories:
    Gradle Central Plugin Repository
BUILD FAILED in 1s

When the repository is declared but the plugin was not published to it yet (here the build directory had just been deleted), the same message names the repository, with the scratch path shortened:

- Plugin Repositories (could not resolve plugin artifact 'tidepool.station-check:tidepool.station-check.gradle.plugin:0.1.0')
  Searched in the following repositories:
    maven(file:/work/plugin/build/repo)
    Gradle Central Plugin Repository

The artifact it could not resolve is the marker, tidepool.station-check:tidepool.station-check.gradle.plugin:0.1.0, which is how to diagnose a plugin published without it. With minStations = 5 in the consumer, the plugin’s own check fails the build:

> Task :checkStations FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkStations' (registered by plugin 'tidepool.station-check').
> expected at least 5 stations but found 3
BUILD FAILED in 9s
1 actionable task: 1 executed

Note that ./gradlew tasks --group verification in a consumer that does not apply base lists checkStations but no check, and ./gradlew check there runs checkStations anyway: Gradle’s task-name abbreviation resolved check to the only task whose name starts with that word (the abbreviation rules are on Command-Line Interface).

A plugin published without a marker (for example one built by hand, without java-gradle-plugin) can still be applied by id through a resolutionStrategy in pluginManagement that maps the id to a module, useModule("org.example:custom-plugin:${requested.version}") (not run here). The Groovy and Kotlin forms are in the manual.

Source: Gradle User Manual: Publishing Plugins, Plugins published without Gradle Plugin Portal

Publishing to the Gradle Plugin Portal

The Gradle Plugin Portal is the default plugin repository: a build that applies id("x") version "y" with no pluginManagement block resolves it from there (see Plugins Overview). To publish to it, apply the com.gradle.plugin-publish plugin, which is itself hosted on the Portal. Since version 1.0.0 it applies java-gradle-plugin and maven-publish for you; version 2.0.0 needs Gradle 7.4 or later. It creates and publishes the sources and Javadoc JARs (so the withSourcesJar() and withJavadocJar() lines of the plugin build are dropped here), signs the artifacts when the signing plugin is applied, and needs three things beyond the build of the plugin: a website, a source repository, and per-plugin displayName, description and tags. Since version 2.1.0 it also expects the plugin to declare which opt-in Gradle features it supports; publishing without that declaration is deprecated and warns, and the Portal plans to reject such submissions. Because the plugin was run with --configuration-cache above, declaring configurationCache = true is truthful (the Portal can revoke a badge that does not match reality). The relevant parts of the build file (the example.com addresses are placeholders):

Kotlin DSL
import org.gradle.plugin.compatibility.compatibility

plugins {
    `java-gradle-plugin`
    `maven-publish`
    id("com.gradle.plugin-publish") version "2.2.1"
}

group = "io.tidepool"
version = "0.1.0"

gradlePlugin {
    website = "https://example.com/tidepool"
    vcsUrl = "https://example.com/tidepool.git"
    plugins {
        create("stationCheck") {
            id = "tidepool.station-check"
            implementationClass = "io.tidepool.gradle.StationCheckPlugin"
            displayName = "Tidepool station check"
            description = "Validates the tidepool station CSV file."
            tags = listOf("tidepool", "validation", "csv")
            compatibility {
                features {
                    configurationCache = true
                }
            }
        }
    }
}
Groovy DSL
plugins {
    id 'java-gradle-plugin'
    id 'maven-publish'
    id 'com.gradle.plugin-publish' version '2.2.1'
}

group = 'io.tidepool'
version = '0.1.0'

gradlePlugin {
    website = 'https://example.com/tidepool'
    vcsUrl = 'https://example.com/tidepool.git'
    plugins {
        stationCheck {
            id = 'tidepool.station-check'
            implementationClass = 'io.tidepool.gradle.StationCheckPlugin'
            displayName = 'Tidepool station check'
            description = 'Validates the tidepool station CSV file.'
            tags = ['tidepool', 'validation', 'csv']
            compatibility(it) {
                features {
                    configurationCache = true
                }
            }
        }
    }
}

The Kotlin DSL needs the explicit import of the extension function, and the Groovy DSL passes it as the receiver of compatibility. Both build files were evaluated here and the plugin resolved from the Portal (version 2.2.1 is the latest on the day of the run). The tasks it adds are in the group "Plugin Portal":

$ ./gradlew tasks --all
...
Plugin Portal tasks
-------------------
login - Update the gradle.properties files so this machine can publish to the Gradle Plugin portal.
publishPlugins - Publishes this plugin to the Gradle Plugin portal.
...
$ ./gradlew help --task publishPlugins
> Task :help
Detailed task information for publishPlugins
Path
     :publishPlugins (registered by plugin 'com.gradle.plugin-publish')
Type
     PublishTask (com.gradle.publish.PublishTask)
Options
     --validate-only     Do not publish plugins, only validate they can be published
     --no-validate-only     Disables option --validate-only.
     --rerun     Causes the task to be re-run even if up-to-date.
Description
     Publishes this plugin to the Gradle Plugin portal.
Group
     Plugin Portal
BUILD SUCCESSFUL in 1s

Credentials and the publish command (not run here)

publishPlugins needs an API key and secret from a Plugin Portal account. The manual gives three ways to supply them: the Gradle properties gradle.publish.key and gradle.publish.secret in the user’s gradle.properties, the environment variables GRADLE_PUBLISH_KEY and GRADLE_PUBLISH_SECRET (the choice for CI), or the experimental login task, which writes the properties file. Keep them out of the project: the user-level file is read by every build of the machine (see Build Environment & Properties), and a CI secret store injects the variables. A placeholder ~/.gradle/gradle.properties looks like this; the values are not real and must never be committed:

gradle.publish.key=${PLACEHOLDER_PORTAL_KEY}
gradle.publish.secret=${PLACEHOLDER_PORTAL_SECRET}

The commands, as given in the manual, are

./gradlew publishPlugins --validate-only
./gradlew publishPlugins

and none of them was run against the Portal here: publishing needs a real account and a real, public website and repository, and an upload cannot be taken back, because the Portal treats every version as fixed. Running the validation without keys fails before it contacts anything, and that failure was run (Kotlin DSL, Groovy DSL gave the same):

$ ./gradlew publishPlugins --validate-only
> Task :compileJava
> Task :pluginDescriptors
> Task :processResources
> Task :classes
> Task :jar
> Task :javadoc
> Task :javadocJar
> Task :sourcesJar
> Task :assemble
> Task :generateMetadataFileForPluginMavenPublication
> Task :generatePomFileForPluginMavenPublication
> Task :publishPlugins FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':publishPlugins' (registered by plugin 'com.gradle.plugin-publish').
> Missing publishing keys. Please set gradle.publish.key/gradle.publish.secret system properties or GRADLE_PUBLISH_KEY/GRADLE_PUBLISH_SECRET env variables or login using the login task.
BUILD FAILED in 4s

This shows what --validate-only does and does not do: the whole build of the artifacts runs first, and --validate-only stops at the upload; it still wants the keys, so it cannot be used as a keyless dry run, and it was therefore not run further. The rules of the Portal that matter before the first upload, from its documentation and not tried here:

  • The version cannot end in SNAPSHOT; every published version is fixed and cannot be replaced.

  • A com.github group prefix is rejected; use io.github for plugins hosted on GitHub.

  • The first version of a plugin may be held for manual approval by the Portal administrators, who check that the owner can be established (for example a linked GitHub account, or a DNS record for an organization domain), that the plugin is not a fork of a still-maintained plugin, and that the website and repository URLs point to public documentation. The process can take a few days.

  • The plugin should follow the Gradle code of conduct, and the person publishing must accept the Portal’s terms of use.

Source: Gradle User Manual: Publishing Plugins, Adding the Plugin Publish Plugin, Configuring the Plugin Publish Plugin, Publishing the plugin, Declaring compatibility with Gradle features and Plugin Portal: How do I publish my plugin?

Plugins that must stay private are published to a Maven repository that the organization runs, exactly as in the previous section, and consumed with that repository in pluginManagement. Publishing to a repository that needs credentials is covered by the credentials and secrets sections of Build Environment & Properties; the Maven Central route, which needs a signed and checksummed publication, is described for libraries on Publishing and, for the Maven build tool, on Publishing to Maven Central.

Changed since Gradle 8.14 and since the Gradle 1.x book

Nothing on this page was run on Gradle 8.14.6; the entries below come from the official documentation and release notes and are labelled "not run here".

Item Status

BuildResult.getOutputReader()

New in Gradle 9.3.0 (and marked incubating); a TestKit test for 8.14 uses getOutput(). Not run here on 8.14.

Running Gradle 9

Needs JVM 17 or later (the TestKit client stays compatible with JVM 8). A plugin tested with TestKit on 8.14 needs a test JVM no newer than 24, because the build under test runs on it. Not run here on 8.14.

Declaring compatibility with Gradle features

com.gradle.plugin-publish 2.1.0 added the compatibility { features { ... } } block; publishing without it warns, and the Portal plans to reject it. Isolated Projects can be declared from 2.2.1.

com.gradle.plugin-publish 2.0.0

Requires Gradle 7.4 or later. Since 1.0.0 the mavenCoordinates block is gone (the coordinates are those of the Maven publication) and the former pluginBundle block is merged into gradlePlugin {} (the merged form needs Gradle 7.6 or later).

Kotlin DSL val x by getting(Type::class) and val x by registering(Type::class)

All Kotlin DSL property delegates are deprecated and will be removed in Gradle 10; use named<Type>("x") and register<Type>("x"). The first version of the build file above used the delegates and printed the deprecation with --warning-mode all. Upgrading to Gradle 9 lists the replacements.

Plugins tested with ProjectBuilder only (Gradle 1.x book)

TestKit GradleRunner is the main tool for functional tests; ProjectBuilder stays for fast configuration checks.

apply plugin: 'x' and no Plugin Portal (book)

The plugins {} block and the Portal; TestKit’s classpath injection works for the plugins {} block only.

Standalone plugin published to a hand-made repository (book)

java-gradle-plugin plus maven-publish publishes the main artifact and the marker; com.gradle.plugin-publish for the Portal.

maven plugin and uploadArchives (book)

Removed in Gradle 7.0; use maven-publish.

Source: Gradle 9.3.0 Release Notes, Upgrading to Gradle 9, Kotlin DSL delegated properties and Plugin Portal: Breaking changes

Classic mistakes

Symptom Cause and fix

Every functional test fails with Plugin [id: 'tidepool.station-check'] was not found in any of the following sources

withPluginClasspath() is missing, or the test build applies the plugin with apply plugin: instead of plugins {}, or the plugin builds with compileOnly dependencies on another plugin.

JUnit Jupiter > initializationError with NoClassDefFoundError: org/gradle/api/Task in the report

The unit-test source set has no Gradle API on its runtime classpath; add implementation(gradleApi()) or leave testSourceSets at its default.

Plugin validation failed with 1 problem in :validatePlugins

A task type lacks an annotation (@CacheableTask, @DisableCachingByDefault, @UntrackedTask, or an input/output annotation on a getter).

A consumer on JDK 17 cannot resolve the plugin that was built on JDK 26

org.gradle.jvm.version in the module metadata is 26; compile with options.release = 17, or use a toolchain with a language version of 17.

A test passes alone and fails in the suite

State shared through the TestKit user home (the local build cache, the daemon), or a fixed project directory; use a @TempDir per test and a build cache directory inside it.

publishPlugins fails with Missing publishing keys

gradle.publish.key/gradle.publish.secret or GRADLE_PUBLISH_KEY/GRADLE_PUBLISH_SECRET are not set; this applies to --validate-only too.

Publication to the Portal is rejected

Version ending in SNAPSHOT, a com.github group, or missing website/vcsUrl; see the rules above.

The consumer’s check runs only checkStations

The consumer has no check task, because base or a language plugin is not applied; the name was abbreviated to the one task that starts with check.

The same idea in the other two tools

Concept Gradle Maven Ant

Unit test of the extension

A JUnit test with ProjectBuilder

A JUnit test with the plugin testing harness (@MojoTest)

A JUnit test that instantiates the task class

Run a real build in a test

TestKit GradleRunner in a temporary project

The Maven Invoker Plugin runs projects under src/it

A test that launches ant or the Project API on a scratch build file

Plugin metadata for tests

pluginUnderTestMetadata and withPluginClasspath()

The plugin descriptor generated by maven-plugin-plugin

A taskdef with the class path of the task JAR

Publish

maven-publish with the marker, or com.gradle.plugin-publish

mvn deploy, or the Central publishing flow

A JAR placed in a repository by hand or with Ivy or the Resolver Ant Tasks

See Writing Maven Plugins, Writing Custom Ant Tasks and the cross-tool view in Comparing Ant, Maven & Gradle.

References