Ivy: Module Descriptors & Configurations

Three small modules were built from the tidepool sources: tidepool-model (no dependencies of its own), tidepool-engine (depends on the model and on org.apache.commons:commons-csv) and tidepool-cli (depends on the engine and on info.picocli:picocli). tidepool-model and tidepool-engine were published with ivy:publish into a file repository repo/ next to the project, so that the next module could depend on them by coordinate. The settings file that lets Ivy see both that repository and Maven Central is below; its elements are explained on the settings page.

Most of the experiments are small descriptors resolved by one generic build file, which resolves a descriptor named on the command line and then retrieves the result into one directory per configuration; the lists of files in the output below are ls of those directories. The (-[classifier]) in the pattern is an optional part that appears only for artifacts that have a Maven classifier.

It is run as ant -lib ivy-2.6.0.jar -Ddescriptor=m1.xml. In every output block the two banner lines (Apache Ivy 2.6.0 ... and loading settings ...) and the Buildfile, BUILD SUCCESSFUL and timing lines are removed, because they never change; the Ivy cache was already warm (no download lines). Everything else is as printed.

The files in this section are this page’s own setup for the examples that follow, not taken from the documentation.

An Ivy file is an XML file, usually named ivy.xml. The simplest one has a root and an info:

The root, ivy-module, has a required version (still 2.0 with Ivy 2.6.0). Its children are, in the order the XML schema requires, info (once), then optionally configurations, publications, dependencies and the deprecated conflicts. The same shape, with the children expanded, is the hierarchical index of the reference:

Ivy expects no XML namespace for its own elements and attributes, so you cannot use a prefix for them; your own extra attributes and elements live in a namespace of your own (the info, conf, dependency and artifact elements accept extra attributes, since 1.4). For editor completion, the schema (ivy.xsd, which is inside the JAR at org/apache/ivy/plugins/parser/xml/ivy.xsd) can be declared as xsi:noNamespaceSchemaLocation.

A descriptor plays two roles. During development it is dynamic: it may use dynamic revisions such as latest.release and Ivy variables. At publication it must be resolved: fixed revisions and no variables, so that the published revision always produces the same result. ivy:deliver turns the first kind into the second; both are shown below.

This is the complete descriptor of tidepool-engine, the one used for most of the page (the comments are explained in the sections that follow):

Resolved (first resolve, with the middle of the list trimmed):

The compile configuration holds four modules: the model (from the local repository), commons-csv, and the two modules commons-csv needs. runtime holds the same four, because the commons-csv POM has no runtime-only dependency. test adds JUnit and its dependencies (nine more modules), and master holds nothing, because it exists only to name the module’s own artifact. The rest of the page explains why each of those numbers is what it is.

Source: Ivy documentation: Ivy files

Source: Ivy documentation: Hierarchical index

info identifies the module and carries its metadata.

When a descriptor has no revision, Ivy gives the module the pseudo-revision working@<hostname>: that is the working@<host> you see in the output of the experiments below, for instance io.tidepool#exp-mapping;working@<host>. The status matters for dynamic revisions: latest.release selects the latest revision whose status is release, and Ivy’s ivy:publish and ivy:deliver take a status of their own at publication.

Child elements: license (name and URL, the licence of the module), ivyauthor, repository (where the module can be found publicly), description (free text with an optional homepage), and extends (inherit from a parent descriptor, below). After the description you can add elements in your own namespace. The descriptors that Ivy derives from POMs fill license and description from the POM; the converted commons-csv descriptor in the cache has, for instance, <license name="Apache-2.0" url="https://www.apache.org/licenses/LICENSE-2.0.txt"/> and a description with the project’s homepage.

Source: Ivy documentation: info

A configuration is the idea that makes Ivy different from Maven’s fixed scopes. The reference defines it as a way to use or construct a module: the same module has different dependencies depending on how you use it, and each distinct set is a configuration. Hibernate’s jta.jar is needed only outside an application server; a build needs the compiler plug-in at build time and not at run time. A configuration is a view of the module that selects some of its artifacts and some of its dependencies. Everything else in Ivy (resolving a part, retrieving a part, mapping to a dependency’s part) is expressed in configurations. If you declare none, the module has one public configuration called default, and that is enough for a simple module.

The configurations element contains conf elements (and include`s). A `conf has these attributes:

Writing conf="compile->compile(*),master(*)" on every dependency is tedious, and three attributes remove the repetition. They sit on configurations or on dependencies; the last definition of each wins.

The documentation shows how they interact in a table:

The table is the one on the configurations page. A descriptor can also pull the whole configurations element from a file with include; the included file has the same structure, with its own defaultconfmapping. The next example, i1.xml, includes inc-confs.xml (which declares compile and runtime and maps both to default), adds test, and sets defaultconf="runtime" on the dependencies:

There is no listing for compile: commons-csv has no conf, so it got defaultconf, runtime, and only that configuration (and the ones that extend it) receives the dependency. When ivy:deliver publishes an included file, it inlines it, so the delivered descriptor no longer depends on the external file.

confmappingoverride is easiest to see with a dependency whose runtime view is bigger than its compile view, as JUnit Jupiter’s is (the API at compile time, the engine at run time). Both descriptors have defaultconfmapping="compile->compile;runtime->runtime", runtime extends compile, and the dependency is declared once with conf="compile". Without the override, runtime only gets what compile brought; with it, runtime applies its own mapping as well:

With the default (false) the runtime configuration silently lacks junit-jupiter-engine and junit-platform-engine, a classic way for an Ivy build to compile and then fail at run time. That is why the engine descriptor above sets confmappingoverride="true". The documented in-memory form is conf1->other1;conf2->other2, which is what the reference page shows.

Source: Ivy documentation: configurations, defaultconfmapping

Source: Ivy documentation: Configuration mappings details

Source: Ivy documentation: include

Source: Ivy documentation: dependencies

publications lists the files the module publishes and in which public configurations each is found. A consumer who depends on a configuration gets the artifacts published in it (and in the configurations it extends). defaultconf on publications (default *, all public configurations) sets where artifacts go when they name none.

If the element is missing, the module is taken to publish one artifact, named like the module, in every configuration. A module that publishes no artifact at all (a virtual module that only groups other modules) needs an empty publications element. <artifact/> alone is the default artifact. The tidepool modules publish one JAR, in master (the module alone, no dependencies) and in compile (so that a consumer who maps to compile gets the module’s own JAR together with its compile dependencies):

Resolved from tidepool-cli, whose descriptor has conf="compile->compile;runtime->runtime" on the engine, this is why the engine’s own JAR is on the CLI’s classpath without a separate mapping (descriptor and listing below). The CLI’s descriptor, which also has a processor configuration used for the annotation processor, and the result of resolving it:

(The 1 in the search and dwnlded columns is tidepool-engine itself, which that cache had not seen before; the listing lines are the CLI’s retrieved directories.) Three things worth reading: the engine reached the CLI together with the model and commons-csv, which is transitivity through the compile mapping; JUnit and the engine’s own test configuration are absent from compile and runtime, thanks to the private visibility of the engine’s test; and processor holds only picocli, a configuration that exists so that the processorpath of javac can be fed from it without putting the processor on the compile classpath. That is the Ivy counterpart of Maven’s annotationProcessorPaths and Gradle’s annotationProcessor configuration.

Extra artifacts with a Maven classifier use the extra attribute m:classifier, where m is the namespace http://ant.apache.org/ivy/maven; the converted commons-csv descriptor publishes its sources that way: <artifact name="commons-csv" type="source" ext="jar" conf="sources" m:classifier="sources"/>.

Source: Ivy documentation: publications

Source: Ivy documentation: artifact

Source: Ivy tutorial: Using Ivy module configurations

dependencies holds the dependency elements and, after them in this order, the exclude, override and conflict elements; the XSD requires that order. A dependency says: this module needs that module, in that revision, and these configurations of it in these of mine.

The children are conf (a nested form of the mapping), artifact and include (restrict which artifacts are used) and exclude.

exclude removes artifacts, modules or organisations from the graph. It has two homes: as a child of a dependency, where it applies to what that dependency brings in, and as a child of dependencies, where it applies to the whole module (since 2.0). Its attributes are org, module, artifact, type, ext, matcher and conf, all defaulting to "anything"; matcher is exact (the default since descriptors of version 1.3) or regexp, and conf limits the exclusion to some of your configurations.

On a dependency, removing commons-codec while keeping commons-csv and commons-io:

On the whole module, but only in compile, with test unchanged:

The module-level form is the one to use when many dependencies drag in a module you never want (a logging bridge, for instance). Excluding a module is a statement that nothing needs it; if something does, you find out at run time.

Source: Ivy documentation: exclude

override changes the revision (and/or branch) requested for a transitive dependency, before any conflict management and without declaring a dependency on it (since 2.0). This is the right tool when a direct dependency brings a module whose version you want to pin, and the module is not really yours to depend on. The override is applied in the phase called dependency descriptor mediation; afterwards the transitive dependency behaves exactly as if it had been declared with the new value. Attributes: org, module, branch, rev, matcher (default exact; org and module can be expressions with a matcher such as regexp).

commons-csv wants commons-io 2.20.0; with <override org="commons-io" module="commons-io" rev="2.19.0"/> after the dependencies:

(The 1 in search and dwnlded is the first download of commons-io 2.19.0 in that cache.) Nothing was evicted, because there was no conflict: the override changed what was requested. In the converted commons-csv descriptor the entries of the BOM that its POM imports (dependencyManagement) appear as override elements; the first is <override org="org.junit.jupiter" module="junit-jupiter" matcher="exact" rev="5.13.1"/>.

Since 2.6.0 the post-resolve tasks (retrieve, cachepath and the others) also accept an override child, so a build can adjust revisions when it gathers files. The child works with the resolve that the task performs itself from nested dependency elements (not in the inline mode, and not with file; the task rejects both with a message such as 'file' not allowed with child elements). This ran:

The 2.6.0 release also makes makepom write an override into the generated POM’s dependencyManagement (IVY-1663), so published POMs keep the pin. That one was not run here.

Source: Ivy documentation: override

Source: Ivy documentation: post resolve tasks

conflict (a child of dependencies, since 2.0) names the conflict manager to use for one or several modules, selected with org, module and matcher, either by manager (the name of a manager) or by rev (a comma-separated list of revisions to select, for a fixed manager). It replaces the old top-level conflicts element, which is deprecated. The default, when nothing is specified, is latest-revision for all modules. The strict manager does the opposite of resolving: it fails if two revisions of a module are required. With the same commons-io 2.18.0 and 2.20.0 from the force example, and <conflict org="commons-io" module="commons-io" manager="strict"/>:

Ivy conflict managers (latest-revision, latest-time, latest-compatible, strict, all, nearest since 2.6.0, and fixed ones) are the subject of Ivy: Conflicts & Versions. For Maven’s mediation, see Maven dependencies, scopes and mediation; for Gradle’s, Gradle dependency resolution, conflicts and capabilities.

Source: Ivy documentation: conflict

Source: Ivy documentation: conflicts

A short checklist that follows from the above, and the scheme the tidepool modules use:

The cross-tool picture is in Comparing Ant, Maven & Gradle. The tasks that act on all of this (resolve, retrieve, cachepath, report, deliver, publish) are on Ivy Ant Tasks.