Fortran Package Manager (fpm)

The heatflow project from the earlier pages is an fpm project with a library, one application, one example and one test program. The manifest is the one from Getting Started with auto-examples and an explicit [[test]] table added:

Source: fpm: Manifest reference

The first build also fetches the dev-dependency. The compile order is computed from the use and submodule statements: the submodules heatflow_ftcs and heatflow_jacobi are compiled after their parent module heatflow_solver, although fpm starts the files in an order that respects those dependencies.

The git init messages that precede the From line in the real output are omitted. fpm test, fpm run and fpm run --example then give:

The lines marked ... stand for fpm’s progress messages. Listing the targets before running them:

Source: fpm: First steps with fpm

fpm run and fpm test take target names, with simple globbing: ? matches one character and * any string, quoted against the shell. --all is an alias for --target '*', and in fpm 0.13.0 a bare fpm run also runs every executable (the help text says a list is printed when several candidates exist and no name is given, but the version tested ran them all). In a project with app/main.f90, app/tools/probe.f90 and two test programs the commands behave like this:

Everything after -- goes to the program. The heatflow application takes the name of a namelist file, and its error stop 2 for a missing file is reported by fpm as a failed execution:

The same applies to fpm test: a test program signals failure with a nonzero exit status, for example through error stop, and fpm test itself then exits with status 1 (checked with a one-line test program that executes error stop "boom"). That is the contract that makes fpm usable in continuous integration; the statements are described in Error Handling and Program Termination.

--runner CMD puts a command in front of the program path. It is available for run and test, which makes it the hook for debuggers, valgrind, mpiexec and tracing tools. Without a value the runner is echo, which prints the paths of the executables instead of running them. --runner-args passes arguments to the runner itself, as opposed to -- ARGS, which goes to the program. A value that begins with a dash is read by fpm’s argument parser as options, so write --runner-args=" -np 4" with an equals sign and a leading space:

The first command ran the test program under valgrind and printed the same two PASSED lines, the second printed build/gfortran_DF6D0BB80A7053CC/app/heatflow, and the third started the MPI program of the mpi metapackage example below on three ranks. With the value -np 4 and no equals sign the container’s fpm answered UNKNOWN COMPOUND SHORT KEYWORD:n in -np 4 followed by a table of its options.

Source: fpm: Built-in dependencies (metapackages)

fpm decides whether a file must be rebuilt from the content of the source, not from its timestamp, and it rebuilds every file that depends on a changed module. Touching a file changes nothing:

A real change in a deep module cascades through everything that uses it, directly or not (here a comment was appended to heatflow_kinds.f90, which every other module uses; only heatflow_boundary, which does not use it, was left alone):

The same comment appended to a submodule, heatflow_jacobi.f90, rebuilds only that file, relinks the archive and the application (do_concurrent_stencil does not use the solver and is not relinked):

That measurement is the whole argument for splitting implementations into submodules, discussed in Modules and Submodules: fpm cannot tell that a comment does not change a module’s interface, so a changed module file recompiles its users, but a changed submodule has no users.

Objects, .mod files, archives and executables live in subdirectories of build/ named after the compiler and a hash (build/gfortran_DD609D2CDA6636FC). Builds with different compilers, profiles or flags normally get different directories and do not overwrite each other, and a cache of the resolved dependencies is kept in build/cache.toml. With --build-dir DIR or the environment variable FPM_BUILD_DIR the whole directory moves (fpm build --build-dir /tmp/mybuild produced cache.toml, compile_commands.json, dependencies and the hashed directories there).

The module files are written to the hashed directory itself and the objects to a subdirectory named after the package; the directory is passed with -J and -I, which is the "module directory" of the compiler options in Compilers and Compiler Options. To see exactly what fpm runs, use --verbose, or read build/compile_commands.json (see compile_commands.json and editors). For heatflow_grid.f90 with --profile release:

The last four options before -I././include come from the manifest, not from the profile: -fimplicit-none and -Werror=implicit-interface follow from implicit-typing = false and implicit-external = false (the defaults), and -ffree-form from source-form = "free". With implicit-typing = true and implicit-external = true and source-form = "default", the same file was compiled with none of the three. A missing implicit none is therefore caught in fpm projects by the flag (see The fortran table).

Pruning ("tree-shaking"): in the default monolithic library mode, the archive contains only the objects that the applications, tests and examples actually need; the others are still compiled. --no-prune keeps all of them.

A plain fpm build in fpm 0.13.0 passed no optimisation, warning or run-time checking options at all, whatever the documentation suggests about an "implicit debug build": the compile line for first_steps.f90 was gfortran -c ././src/first_steps.f90 -fimplicit-none -Werror=implicit-interface -ffree-form -J ... -I... -o ..., and fpm build --verbose printed an empty COMPILER OPTIONS line. The profile has to be named. With the GNU compiler the two built-in profiles gave:

The exact lists belong to fpm and may change between versions; read them with --verbose. -fcoarray=single is one of the three values gfortran accepts (none, single, lib) and compiles coarray code for a single image, see Coarrays, Images and Synchronization. The C and C++ options are derived in the same way for projects with C sources (-Wall -Wextra -fPIC -fmax-errors=1 -g for debug).

A consequence worth knowing: because the debug profile includes -fPIC and a plain build does not, a plain build cannot link a shared library (see Library types and installation).

--flag FFLAGS is added to the options of the named profile; without a profile it is the only option fpm passes, replacing the defaults. It has the strongest use in continuous integration, to impose strict standard checking whatever the profile does:

For --profile release --flag "-march=native" the compile line becomes gfortran -c ././src/heatflow_grid.f90 -march=native -O3 -Wimplicit-interface -fPIC -fmax-errors=1 -funroll-loops ..., with the user flag first. Every setting has an environment variable for use in scripts and CI; the command-line option wins when both are set.

For MPI, fpm additionally reads MPIFC, MPIF90, MPIF77, MPICC and MPICXX (and the I_MPI_* variables for Intel MPI) to find compiler wrappers, see Metapackages.

--compiler NAME or FPM_FC picks the compiler by executable name, and fpm identifies the family from the name. Building the whole heatflow project with Flang worked unchanged; the compiler name only has to be in the PATH:

The results are identical to the gfortran build, and the build directories are named flang-23_... next to the gfortran_... ones. Two caveats were measured. First, with the versioned name flang-23, or even with a symbolic link named flang, --profile debug and --profile release passed no options at all (COMPILER OPTIONS was empty), so on Flang use --flag, or a feature with a flang section (see Features and profiles). Second, the openmp metapackage also probes the C compiler; in the container, which has Flang but no clang, fpm run --compiler flang stopped with failed command clang -c ... -fopenmp, so OpenMP with Flang is compiled with --flag "-fopenmp" instead, see OpenMP and OpenACC. Flang rejects -std=f2023: it prints error: Only -std=f2018 is allowed currently.

The compiler names fpm knows (for the per-compiler sections of the manifest) are: gfortran, ifort, ifx, nvfortran, nagfor, flang, amdflang, lfortran, f95, ftn95, caf, pgfortran, xlf90, crayftn and lfc. Since 0.13.0 the name flang means LLVM Flang, and amdflang is supported. No other compiler (Intel ifx, NAG, Cray, NVHPC) was available for this page.

Source: fpm: Features and Profiles

Source: fpm: Manifest reference

The library target is generated when src/ or include/ exists. Its [library] table moves them:

Source: fpm: Manifest reference

source-dir is relative to the project root, with / as the separator on all platforms. include-dir takes one directory or a list and is added to the include path of the compiler for the package and its dependents (it does not allow pre-built .mod files). type is covered in Library types and installation.

Executables, tests and examples are alike. Each is declared with an array-of-tables entry; name is required, and source-dir (default app, test or example) and main (the file with the program) are optional:

Source: fpm: Manifest reference

The last two entries show the syntax of the reference and were not built here. Each entry may also declare its own link libraries and its own dependencies, which are used only by that target and are not exported. For many targets an inline array is shorter: executable = [ { name = "a-prog" }, { name = "app-tool", source-dir = "tool" } ], which must come before the first [table] header of the file.

To see an explicit executable at work, here is a second fpm project, heatflow-cli, that depends on heatflow by path (the dependency forms are in Dependencies) and has its own application. app/hot_spot.f90:

Source: fpm: Manifest reference

and its manifest:

(The executable is hot-spot, named in the manifest, not after the file.) The program compiles without warnings under the strict gfortran options, passed with --flag, and builds and runs identically with --compiler flang. Note that the dev-dependency test-drive of heatflow was not fetched for this consumer: build/dependencies stayed empty, because development dependencies are not exported.

Source: fpm: Manifest reference

Link order is the classic trap. LAPACK depends on BLAS, so link = ["lapack", "blas"] is the correct order. With the reference BLAS and LAPACK shared libraries of the container both orders happened to link and print ddot = 14.0, because shared libraries record their own dependencies; with static libraries the wrong order fails. A library that does not exist is a linker error, not an fpm error:

external-modules is for modules that fpm cannot find among the sources. fpm must not report them as missing, but it does not locate their files or their libraries: the include paths and libraries are the user’s responsibility. Compare three cases for a program that says use omp_lib. A module that is missing everywhere is an fpm error (the program here uses a module named nosuch_module):

omp_lib is a compiler-provided module, so gfortran finds it by itself, the fpm scan lets it through, and the failure comes at link time because nothing links OpenMP:

With external-modules = "omp_lib" in [build] and fpm run --flag "-fopenmp" it builds and prints OpenMP threads available: 10. The openmp metapackage (below) does all of this without the manual steps.

Source: fpm: Manifest reference

implicit-typing (default false) and implicit-external (default false) control whether fpm omits -fimplicit-none and -Werror=implicit-interface; source-form is "free" (default), "fixed" or "default", which lets the compiler decide from the file extension. The defaults mean that modern code gets the safe flags automatically and legacy code must opt out with implicit-typing = true and implicit-external = true, which is also how a fixed-form legacy package is described (with fpm 0.13.0 such a package also needed an explicit [[executable]] entry with main = "main.f", because app/main.f was not discovered automatically), see Legacy Features and Modernization. Writing implicit none in the source is still the recommended style, because it does not depend on the build tool.

[install] library = true makes fpm install also install the library archive and the module files; without it only executables are installed. module-dir (since 0.13.0) changes the subdirectory of the prefix for the .mod files (default include). See Library types and installation.

Third-party tools keep their own configuration in [extra], which fpm itself never evaluates; the convention is one subtable per tool (extra.fpm.<plugin> for fpm plugins), keys in lowercase with dashes.

[preprocess.cpp] enables the C preprocessor for the package, with the keys suffixes (file suffixes to preprocess), directories (limit it to directories) and macros (a list of NAME or NAME=VALUE). See Preprocessing and conditional compilation.

A path is relative to the fpm.toml that contains it and uses / on all platforms. It is the form for several packages in one repository, and for the heatflow-cli example above. After building, build/cache.toml records what fpm resolved:

Source: fpm: Manifest reference

A git dependency names a repository and at most one of branch, tag or rev. Without one, the default branch is used.

Source: fpm: Manifest reference

All three were fetched and built, and build/cache.toml recorded obj and the full commit that obj resolved to (rev = "c506771aefd594e7e372240a8027b3fd06d61264" for the tag v0.6.1, and for branch = "main" the commit of the main branch on the day, which changes). The errors are:

Pin to a tag or a full rev in anything that must be reproducible; a branch (or no reference) moves under you. A dependency is cloned into build/dependencies/<name>, built with the same compiler and flags as the main package, and reused until fpm update or a changed reference. fpm update takes dependency names, --fetch-only to fetch without updating existing projects, --clean to ignore the cache, and --dump FILE to write the resolved dependency tree (TOML, or JSON for a .json name). Dependencies of dependencies are resolved recursively: the stdlib example below brought in test-drive version 0.4.0, which the stdlib package depends on, next to stdlib itself.

A dependency can be given more control with a dependency-specific preprocessor table, since fpm 0.9.1: fftpack = { git = "https://github.com/fortran-lang/fftpack.git", preprocess.cpp.macros = ["REAL32"] }. fpm does not check for collisions with the dependency’s own macros.

A package from the registry names a namespace and, optionally, a version:

Without v the newest version is used. The registry settings live in the global configuration file, ~/.local/share/fpm/config.toml on Unix-like systems and %APPDATA%\local\fpm\config.toml on Windows (or the file given with --config-file): [registry] url selects another registry that implements the same API, [registry] path a local directory, and cache_path moves the cache of downloaded packages (default ~/.local/share/fpm/dependencies). A local registry is laid out as <registry>/<namespace>/<package>/<version>/fpm.toml, and fpm then neither downloads nor uses the cache.

This was not usable on the day of the check. The URL that fpm 0.13.0 requested for a registry dependency, https://fpm-registry.vercel.app/packages/toml-f/jonquil/0.3.0, answered HTTP 500 (FUNCTION_INVOCATION_FAILED), and fpm reported:

The official documentation itself says the online registry is under active development. Today git dependencies are what the ecosystem uses, and the curated package index at https://fortran-lang.org/packages/ is a catalogue of repositories, not a registry that fpm queries.

Source: fpm: Registry

The heatflow manifest keeps test-drive in [dev-dependencies]: it is fetched and built for fpm test, but a package that depends on heatflow does not get it. The pin is a tag because a test framework must not change under the project. A dependency needed by just one program, such as a command-line parser used only by the application, belongs in that target’s table, so the library keeps its dependency list short:

Source: fpm: Adding dependencies

Features live in the [features] table, written with dotted keys:

A feature can set compiler flags (flags, c-flags, cxx-flags, link-time-flags), build, Fortran and library settings (build.link, fortran.source-form, library.type), dependencies, dev-dependencies, extra executable, test and example targets, preprocess macros, and install and description entries. Compiler names are the list in Selecting the compiler; operating systems are linux, macos, windows, cygwin, solaris, freebsd and openbsd.

When several variants of a feature match, fpm merges them in this order: the base properties, then the compiler’s, then the operating system’s, then the combined compiler-and-system’s. Flags (flags, c-flags, cxx-flags, link-time-flags) are concatenated with spaces, the lists (dependencies, dev-dependencies, executable, test, example, preprocess) are appended, and the tables build, install, fortran and library are exclusive: defining the same one in two variants of the same feature is an error.

Profiles group features in [profiles]. debug and release exist as built-in profiles with the flags of the table above; if the manifest defines debug or release, its definition replaces the built-in one completely. A profile named default holds features that are applied together with debug and release, but not with a custom profile or with --features.

Here is a manifest for heatflow, added to the one shown earlier, with a ci profile for strict checking and a production profile for speed with OpenMP. The flang sections show how one feature serves two compilers:

Selecting them:

The COMPILER OPTIONS line of the first and the last build was identical, and the tests passed under the ci profile with the same output as before:

Source: fpm: Features and Profiles

For the production profile the compile line of heatflow_grid.f90 was gfortran -c ././src/heatflow_grid.f90 -O3 -march=native -funroll-loops -O3 -march=native -funroll-loops -fopenmp -O3 -march=native -funroll-loops -O3 -march=native -funroll-loops -fopenmp -fimplicit-none ...: the executable linked libgomp.so.1 and gave the same results as before, but the -O3 -march=native -funroll-loops group appeared four times and -fopenmp twice, an inefficiency (not an error) of fpm 0.13.0 that is worth knowing about when reading --verbose output. The same manifest with --compiler flang --features strict passed -pedantic to Flang and built heatflow successfully.

--profile and --features cannot be used together, and an unknown profile is an error:

Name a profile that contains the features you want (--profile ci), or list all features (--features release,mpi). The --help text of fpm build describes the first form as "feature profiles group multiple features together". A dependency can be built with its own features, my-lib = { git = "...", features = ["mpi"] }, or profile, my-lib = { git = "...", profile = "parallel-mpi" }, but not both.

Source: fpm: Features and Profiles

[library] type selects the output (since 0.12.0), and since 0.13.0 it may be an array:

With type = ["static", "shared"], a plain fpm build failed in the link of the shared library, because the objects were built without -fPIC (the aarch64 linker named the relocation R_AARCH64_ADR_PREL_PG_HI21 and asked to recompile with -fPIC):

With --profile release, which includes -fPIC, the build produced both libheatflow.a and libheatflow.so, and fpm install --profile release --prefix /tmp/i2 installed both to lib/. The installed executable linked the shared library (ldd printed libheatflow.so => not found): fpm sets the library search path itself for fpm run, but an installed executable needs LD_LIBRARY_PATH or an rpath. On Windows fpm also generates an import .lib for every .dll.