OpenMP and OpenACC

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

This page was checked on 2026-10-19 with GNU Fortran 16.2 and LLVM Flang 23.1.4. Every OpenMP example for the host was compiled with -fopenmp, run with both compilers, and the output shown is real output (identical for both unless stated). The sources compile warning-free with gfortran -Wall -Wextra -pedantic -fimplicit-none -fcheck=all -fopenmp; the strict -std=f2023 switch is not applied to them because the directives are comments to the standard (see Why directives are not Fortran). GPU offloading cannot be run here: no accelerator was available, so the target and OpenACC examples are labelled below as checked against the documentation and the compilers' syntax checking, and as executing only on the host fallback.

OpenMP and OpenACC are not part of the Fortran standard. They are separate, vendor-neutral specifications, maintained by the OpenMP Architecture Review Board and by the OpenACC organization, that add directives to Fortran, C and C++. As of 2026-10-19 the OpenMP page lists OpenMP 6.0 (November 2024) as the latest released specification, with a 6.0 errata of November 2025; a 6.1 version exists only as a public comment draft (Technical Report 15, July 2026), so nothing on this page relies on 6.1. The OpenACC site offers OpenACC 3.4 as the latest specification. What a given compiler implements is usually an earlier edition, as the version checks in Compiler support and flags show.

Two other ways to use several processors are described elsewhere: do concurrent and GPU Offload (a standard statement), Coarrays, Images and Synchronization (a standard SPMD model) and MPI with mpi_f08 (distributed memory). The examples here continue the heatflow diffusion code: a 41 by 41 grid with a hot spot, advanced with an explicit five-point stencil. The programs are self-contained so each can be pasted into a file, but the loop is the one in heatflow_kernels.

Why directives are not Fortran

A directive is a specially formatted comment. A compiler that does not know about OpenMP sees !$omp parallel do as an ordinary comment starting with ! and ignores it, so the same source is a valid serial Fortran program. This is the central design decision of both specifications: the program must give the same answer with the directives removed. A directive that changes the meaning (a data race, a missing reduction) is a bug in the program, not something the compiler can reject; with the directives ignored, the serial result is what you compare against.

The consequences:

  • the standard-conformance switches of the compiler (-std=f2023) say nothing about the directives;

  • a directive is only checked when the flag that enables it is given (-fopenmp, -fopenacc), after which the compiler checks the clauses and often the data-sharing rules, as in [_default_none_and_diagnostics];

  • directives follow the comment rules of the source form: in free form the sentinel !$omp (or !$acc) may start in any column but must be the first non-blank thing on the line, with no blank between ! and the letters $omp, and a space after it; a directive is continued by ending the line with & and starting the next with the sentinel, optionally followed by & (as in the stencil below);

  • fixed-form sources use c$omp, *$omp or !$omp in column 1; this page uses free form only (see Legacy Features and Modernization).

Sentinels and conditional compilation

The sentinel !$ followed by a blank is the conditional compilation sentinel of OpenMP: the line is an ordinary statement when OpenMP is enabled, and a comment when it is not. It lets a program use omp_lib without breaking a serial build.

program sentinel
  !$ use omp_lib
  implicit none
  integer :: nthreads
  nthreads = 1
  !$ nthreads = omp_get_max_threads()
  print '(a,i0)', 'max threads: ', nthreads
  !$ print '(a)', 'compiled with OpenMP'
  !$omp parallel
  !$omp single
  print '(a,i0)', 'team size: ', nthreads_in_region()
  !$omp end single
  !$omp end parallel
contains
  integer function nthreads_in_region()
    nthreads_in_region = 1
    !$ nthreads_in_region = omp_get_num_threads()
  end function
end program sentinel

Compiled with and without -fopenmp, run with four threads where OpenMP is enabled:

with -fopenmp (OMP_NUM_THREADS=4), gfortran and Flang:
  max threads: 4
  compiled with OpenMP
  team size: 4

without -fopenmp, gfortran:
  max threads: 1
  team size: 1

Source: GNU libgomp manual: Enabling OpenMP

The !$omp parallel / !$omp single pair starts a team of threads and lets one of them execute the print; without -fopenmp the directives are comments and the program runs once with the values set before the conditional lines. The sentinel !$ is only recognised with the flag, so you do not need the preprocessor for this pattern. A real preprocessor (.F90 extension) additionally defines the macro _OPENMP, whose value is the release date of the implemented specification (yyyymm), a way to ask for the version at compile time.

Compiler support and flags

Compiler Flag to enable OpenMP Notes

GNU Fortran

-fopenmp

links libgomp automatically; -fopenmp-simd enables only the simd family of directives with no runtime library

LLVM Flang

-fopenmp

links the LLVM OpenMP runtime; the Debian image used here needs the libomp development package installed

Intel ifx

-qopenmp

the spelling of Intel’s compilers, named in the ifx developer guide; not run here

NVIDIA nvfortran

-mp

named in the NVIDIA HPC SDK documentation; not run here

The -fopenmp spelling is shared by gfortran and Flang; Intel’s -qopenmp is the exception. Build systems hide the difference: CMake’s find_package(OpenMP) and OpenMP::OpenMP_Fortran target (see Building with Make, CMake and Meson), and fpm’s openmp metapackage, described in The fpm openmp metapackage.

What the compilers report about the specification version on 2026-10-19 (openmp_version from omp_lib, and the _OPENMP macro):

program version
  use omp_lib, only: openmp_version
  implicit none
  print '(a,i0)', 'omp_lib openmp_version = ', openmp_version
#ifdef _OPENMP
  print '(a,i0)', '_OPENMP = ', _OPENMP
#endif
end program version
gfortran 16.2 -fopenmp:  omp_lib openmp_version = 202111
                         _OPENMP = 202111
Flang 23.1.4 -fopenmp:   omp_lib openmp_version = 201611
                         _OPENMP = 201107

Source: GNU libgomp manual: OpenMP Implementation Status

GNU Fortran reports 202111 (the OpenMP 5.2 date), although its implementation-status pages also list features of 6.0. Flang’s two numbers disagree with each other (201611 is an older date than 202111, and 201107 is the date of OpenMP 3.1), and its documentation describes OpenMP support as complete for 3.1 and partial for later versions up to 6.0. Do not read these constants as a statement that every feature of that release works: check the implementation status page of the compiler for the clause you intend to use.

If you compile a file that does use omp_lib without -fopenmp, the module is still found by both compilers, and the failure comes at link time, because the runtime library is not linked:

gfortran:  undefined reference to `omp_get_max_threads_'
Flang:     undefined reference to `omp_get_max_threads'

The parallel do directive

parallel do creates a team of threads and divides the iterations of the immediately following do loop among them. The construct ends with !$omp end parallel do, which is optional but recommended. The listing is the heatflow explicit stencil step, with the maximum change reduced across threads:

program stencil
  use, intrinsic :: iso_fortran_env, only: dp => real64
  implicit none
  integer, parameter :: nx = 41, ny = 41, nsteps = 200, ic = 21
  real(dp), parameter :: alpha = 1.0_dp, dt = 5.0e-5_dp
  real(dp) :: u(nx, ny), unew(nx, ny), dx, dy, residual
  integer :: i, j, n
  dx = 1.0_dp/(nx - 1); dy = 1.0_dp/(ny - 1)
  u = 0.0_dp
  u(ic, ic) = 100.0_dp
  unew = u
  do n = 1, nsteps
    residual = 0.0_dp
    !$omp parallel do collapse(2) default(none) shared(u, unew, dx, dy) private(i, j) &
    !$omp& reduction(max: residual)
    do j = 2, ny - 1
      do i = 2, nx - 1
        unew(i, j) = u(i, j) + alpha*dt*( &
                     (u(i - 1, j) - 2.0_dp*u(i, j) + u(i + 1, j))/dx**2 + &
                     (u(i, j - 1) - 2.0_dp*u(i, j) + u(i, j + 1))/dy**2)
        residual = max(residual, abs(unew(i, j) - u(i, j)))
      end do
    end do
    !$omp end parallel do
    u(2:nx - 1, 2:ny - 1) = unew(2:nx - 1, 2:ny - 1)
  end do
  print '(a,es10.3,a,f8.3)', 'residual=', residual, ' max=', maxval(u)
end program stencil

Output:

residual= 2.514E-03 max=   0.499

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), parallel do construct and reduction clause

Both compilers print exactly this, and so does the serial program compiled without -fopenmp. That equality is what to test for. The clauses in the directive are the core of OpenMP programming for loops.

Clause Meaning

default(none)

every variable referenced in the construct must appear in a data-sharing clause or be a loop index; the compiler then reports forgotten ones ([_default_none_and_diagnostics]). Loop indices of the associated loops are private automatically, but listing them in private is harmless

shared(list)

one copy, visible to all threads; the right choice for arrays that are read, or written in disjoint elements

private(list)

each thread has its own uninitialised copy; the original is unchanged after the construct

firstprivate(list)

private, initialised from the original value on entry

lastprivate(list)

private, and the value from the sequentially last iteration is copied back on exit

reduction(op: list)

each thread accumulates into a private copy initialised to the identity of op; the copies are combined into the original on exit. Operators: +, *, .and., .or., .eqv., .neqv., and the intrinsics max, min, iand, ior, ieor

collapse(n)

merges the n perfectly nested loops into one iteration space, which is divided among the threads; essential for the short outer loop of a stencil

schedule(kind[, chunk])

how iterations are handed out; see below

num_threads(n)

size of the team for this construct

if(scalar-logical)

run serially unless the condition holds, for loops too small to pay for the team start

nowait

on the end directive of a worksharing loop inside a parallel region: do not wait at the implied barrier

Fortran’s column-major layout decides which loop to parallelise. In the stencil the inner loop runs over i, the first (contiguous) index, and collapse(2) hands out pairs of j, i. Parallelising only the inner loop would start a team for each j; collapsing, or placing the directive on the outer j loop, divides the work over columns, which also avoids two threads writing into one cache line (false sharing). The layout is described in Performance and Memory Layout.

Schedules

The schedule clause chooses how the iterations are distributed. With twelve iterations and three threads, recording the number of the thread that executed each one shows the effect of static:

program schedule_demo
  use omp_lib
  implicit none
  integer, parameter :: n = 12
  integer :: owner(n), i
  !$omp parallel do num_threads(3) schedule(static, 2)
  do i = 1, n
    owner(i) = omp_get_thread_num()
  end do
  !$omp end parallel do
  print '(a,12i2)', 'static,2 : ', owner
  !$omp parallel do num_threads(3) schedule(static)
  do i = 1, n
    owner(i) = omp_get_thread_num()
  end do
  !$omp end parallel do
  print '(a,12i2)', 'static   : ', owner
end program schedule_demo

Output:

static,2 :  0 0 1 1 2 2 0 0 1 1 2 2
static   :  0 0 0 0 1 1 1 1 2 2 2 2

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), schedule clause

The output is the same with both compilers because a static schedule is deterministic by specification.

Kind Behaviour

static

iterations are divided into chunks assigned to threads in round-robin order at loop entry; without a chunk size one contiguous chunk per thread. Lowest overhead, the right choice when every iteration costs the same, as in a stencil

dynamic

chunks are handed to whichever thread is free; for iterations with unequal cost, at the price of synchronisation per chunk

guided

like dynamic, with chunk sizes that decrease; usually fewer synchronisations than dynamic

auto

the compiler or runtime decides

runtime

taken from the OMP_SCHEDULE environment variable or omp_set_schedule; for experiments without recompiling

If schedule is absent the choice is implementation-defined, so a loop whose performance matters should say what it wants.

Data sharing, atomic and critical

The data-sharing clauses decide whether threads see one variable or many. The program below shows firstprivate, lastprivate and reduction on one loop, then two ways of protecting an update that cannot be a reduction:

program data_sharing
  use omp_lib
  implicit none
  integer :: i, offset, last, total, hits
  offset = 100
  total = 0
  hits = 0
  !$omp parallel do num_threads(4) firstprivate(offset) lastprivate(last) reduction(+: total)
  do i = 1, 10
    offset = offset + i          ! each thread has its own copy, initialised to 100
    last = i
    total = total + i
  end do
  !$omp end parallel do
  print '(a,i0,a,i0,a,i0)', 'total=', total, ' last=', last, ' offset after loop=', offset
  !$omp parallel do num_threads(4)
  do i = 1, 1000
    !$omp atomic update
    hits = hits + 1
  end do
  !$omp end parallel do
  print '(a,i0)', 'hits=', hits
  !$omp parallel num_threads(4)
  !$omp critical
  hits = hits + 1
  !$omp end critical
  !$omp end parallel
  print '(a,i0)', 'after critical: ', hits
end program data_sharing

Output:

total=55 last=10 offset after loop=100
hits=1000
after critical: 1004

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), data-sharing attribute clauses, atomic and critical constructs

offset is private in each thread and starts at 100, but the original is untouched, so the program prints 100; last receives the value of the sequentially last iteration (10), and total is the sum 1 to 10. The two protected updates:

  • !$omp atomic update makes one memory update indivisible, with hardware support where available; it applies to a single simple statement such as x = x + expr or x = max(x, expr). The construct has the forms read, write, update and capture;

  • !$omp critical serialises a block of statements; an optional name (critical (name)) lets unrelated critical sections run concurrently. It costs much more than atomic and should guard larger sections only.

Without either, the 1000 increments would race and the final value would vary from run to run: a data race is undefined behaviour under the specification, and neither compiler detects it. Prefer a reduction whenever the update is a combination of values, as with the max in the stencil; it needs no synchronisation in the loop at all.

Array reductions

A Fortran reduction list item may be a whole array (it is reduced element by element), which makes a histogram a one-line directive:

program array_reduction
  implicit none
  integer, parameter :: n = 10000, nbins = 5
  integer :: hist(nbins), i, bin
  hist = 0
  !$omp parallel do reduction(+: hist) private(bin)
  do i = 1, n
    bin = mod(i, nbins) + 1
    hist(bin) = hist(bin) + 1
  end do
  !$omp end parallel do
  print '(a,5i6)', 'histogram: ', hist
end program array_reduction

Output:

histogram:   2000  2000  2000  2000  2000

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), reduction clause

Each thread gets a private copy of hist initialised to zero and the copies are summed at the end. The cost is one array copy per thread, so a histogram with many bins and many threads costs memory.

default(none) and diagnostics

With default(none) the compiler can enforce that every shared variable was thought about. A missing variable is a compile-time error with -fopenmp; deliberately wrong example:

program bad
  implicit none
  integer :: i
  real :: a(10), s
  a = 1.0
  s = 0.0
  !$omp parallel do default(none) shared(a)
  do i = 1, 10
    s = s + a(i)
  end do
  !$omp end parallel do
  print *, s
end program bad

Diagnostics:

gfortran -fopenmp:
11_default_none.f90:9:16:
    9 |     s = s + a(i)
      |                ^
Error: 's' not specified in enclosing 'parallel'

Flang -fopenmp:
error: Semantic errors in 11_default_none.f90
11_default_none.f90:9:5: error: The DEFAULT(NONE) clause requires that 's' must be listed in a data-sharing attribute clause

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), default clause

The program as written would also be wrong with shared(a, s): all threads would update s unsynchronised. The correct clause is reduction(+: s). Use default(none) on every directive in new code, because the alternative, the implicit sharing rules (variables from an enclosing scope are shared, variables declared inside a called procedure are private to the call), hides precisely the mistakes that cause races.

SIMD

!$omp simd asks the compiler to vectorise a loop, promising that the iterations are independent as far as vector lanes go. It can be combined with threads in parallel do simd:

program simd_demo
  use, intrinsic :: iso_fortran_env, only: dp => real64
  implicit none
  integer, parameter :: n = 1000
  real(dp) :: x(n), y(n), s
  integer :: i
  x = [(real(i, dp), i = 1, n)]
  y = 1.0_dp
  s = 0.0_dp
  !$omp simd reduction(+: s)
  do i = 1, n
    s = s + x(i)*y(i)
  end do
  print '(a,f10.1)', 'dot = ', s
  !$omp parallel do simd reduction(+: s)
  do i = 1, n
    s = s + x(i)
  end do
  print '(a,f10.1)', 'dot + sum = ', s
end program simd_demo

Output:

dot =   500500.0
dot + sum =  1001000.0

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), simd construct

-fopenmp-simd in gfortran honours simd directives without linking a threading runtime (the program above gives the same output with that flag). Clauses such as safelen, aligned, linear and simdlen give the compiler more information, and !$omp declare simd makes a procedure callable from a vector loop. The standard attribute pure/elemental, and plain array syntax, often give the same vectorisation without any directive, with the compiler’s own optimisation reports as the check.

Tasks

A task is a unit of work, created by !$omp task, that any thread of the team may run now or later. Tasks suit recursion and irregular work that has no regular loop. The recursive Fibonacci below creates two tasks per call and waits for both with taskwait; below a cut-off the recursion is sequential, as task creation has a cost:

program tasks_demo
  implicit none
  integer :: r
  !$omp parallel num_threads(4)
  !$omp single
  r = fib(20)
  !$omp end single
  !$omp end parallel
  print '(a,i0)', 'fib(20) = ', r
contains
  recursive function fib(n) result(f)
    integer, intent(in) :: n
    integer :: f, a, b
    if (n < 2) then
      f = n
    else if (n < 12) then
      f = fib(n - 1) + fib(n - 2)
    else
      !$omp task shared(a) firstprivate(n)
      a = fib(n - 1)
      !$omp end task
      !$omp task shared(b) firstprivate(n)
      b = fib(n - 2)
      !$omp end task
      !$omp taskwait
      f = a + b
    end if
  end function fib
end program tasks_demo

Output:

fib(20) = 6765

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), task and taskwait constructs

The !$omp single around the first call makes one thread create the root task while the others wait to execute the tasks that appear. shared(a) is needed because a is private to the call of fib and would otherwise be copied into the task (firstprivate), so the parent would never see the result after taskwait. Dependences between tasks are expressed with depend(in: x) and depend(out: x) clauses, and the taskloop construct divides a loop into tasks.

Array syntax with workshare

Fortran array assignments can be shared among threads by the workshare construct, which divides the evaluation of array assignments, where statements and some intrinsics into units that are executed once each:

program workshare_demo
  use, intrinsic :: iso_fortran_env, only: dp => real64
  implicit none
  real(dp) :: a(8), b(8), c(8)
  integer :: i
  a = 1.0_dp; b = [(real(i, dp), i = 1, 8)]
  !$omp parallel workshare num_threads(2)
  c = a + 2.0_dp*b
  !$omp end parallel workshare
  print '(8f5.1)', c
end program workshare_demo

Output:

  3.0  5.0  7.0  9.0 11.0 13.0 15.0 17.0

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), workshare construct

The directive is Fortran-specific. A compiler may execute parts of it in a single thread, so measure it before relying on it; an explicit parallel do over the elements is predictable.

Offloading with target

target constructs execute a region on a device such as a GPU, copying (mapping) the data the region uses between host memory and device memory. The teams distribute parallel do combination uses the whole device: a league of teams, each with threads. The map clause says how arrays travel: to (copy in), from (copy out), tofrom, alloc and delete.

program target_demo
  use omp_lib
  use, intrinsic :: iso_fortran_env, only: dp => real64
  implicit none
  integer, parameter :: n = 1000
  real(dp) :: x(n), y(n), a
  integer :: i
  logical :: on_host
  a = 2.0_dp
  x = 1.0_dp
  y = 3.0_dp
  on_host = .true.
  !$omp target teams distribute parallel do map(to: x) map(tofrom: y) firstprivate(a)
  do i = 1, n
    y(i) = y(i) + a*x(i)
  end do
  !$omp end target teams distribute parallel do
  !$omp target map(from: on_host)
  on_host = omp_is_initial_device()
  !$omp end target
  print '(a,f6.1,a,l1)', 'y(1)=', y(1), ' ran on host: ', on_host
  print '(a,i0,a,i0)', 'devices: ', omp_get_num_devices(), ' default: ', omp_get_default_device()
end program target_demo

Checked against the documentation and run only on the host. The machine has no accelerator and neither compiler was built with an offload target, so the construct fell back to the host: omp_get_num_devices() is 0 and omp_is_initial_device() is true inside the region. That the program compiles and gives the correct result checks the syntax and the semantics of the mapping, not the behaviour or the performance on a GPU. Output of both compilers:

y(1)=   5.0 ran on host: T
devices: 0 default: 0

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), target and teams distribute parallel do constructs, map clause

Offload needs a compiler built for the device (GCC with the nvptx or amdgcn offload target packages, LLVM with its offload runtime, or the vendor compilers) and is selected with flags that differ per compiler (for example -foffload=... in GCC and -fopenmp-targets=... in LLVM); the manual of the compiler is the reference. Mapping each time costs transfers across the PCIe bus, so loops that are called repeatedly keep the data on the device with an unstructured data region:

module scale_mod
contains
  subroutine scale(v, a)
    use, intrinsic :: iso_fortran_env, only: dp => real64
    real(dp), intent(inout) :: v(:)
    real(dp), intent(in) :: a
    integer :: i
    !$omp declare target
    do i = 1, size(v)
      v(i) = a*v(i)
    end do
  end subroutine scale
end module scale_mod

program target_data_demo
  use, intrinsic :: iso_fortran_env, only: dp => real64
  use scale_mod, only: scale
  implicit none
  integer, parameter :: n = 8
  real(dp) :: u(n), r
  integer :: i, step
  u = [(real(i, dp), i = 1, n)]
  !$omp target enter data map(to: u)
  do step = 1, 3
    !$omp target
    call scale(u, 2.0_dp)
    !$omp end target
  end do
  !$omp target update from(u)
  !$omp target exit data map(delete: u)
  r = sum(u)
  print '(a,f8.1)', 'sum after three doublings = ', r
end program target_data_demo

Output:

sum after three doublings =    288.0

Source: OpenMP Architecture Review Board: Specifications (OpenMP 6.0), target enter data, target update and target exit data constructs, declare target directive

target enter data allocates and copies u once, the three kernels reuse the device copy, target update from brings the result back, and target exit data frees the device copy. declare target marks scale as one that must also exist on the device. Both compilers print the same output (host fallback again).

The omp_lib module

use omp_lib provides the OpenMP runtime routines and constants; it is a module that comes with the compiler, and the include file omp_lib.h is the legacy alternative for fixed-form sources.

Routine Purpose

omp_get_max_threads(), omp_get_num_threads(), omp_get_thread_num()

upper bound for the next team, size of the current team (1 outside parallel regions) and the thread number (0 is the primary thread)

omp_set_num_threads(n), environment OMP_NUM_THREADS

set the team size; the environment variable is the usual way, as in the commands above

omp_get_wtime(), omp_get_wtick()

wall-clock seconds from an arbitrary origin and its resolution; the routine to time parallel code (cpu_time adds the time of all threads)

omp_in_parallel(), omp_get_level()

query nesting

omp_get_num_devices(), omp_get_default_device(), omp_is_initial_device()

offload queries, used above

omp_init_lock, omp_set_lock, omp_unset_lock, omp_destroy_lock

locks (kind omp_lock_kind)

openmp_version

the version constant shown above

The environment variables that matter most are OMP_NUM_THREADS, OMP_SCHEDULE, OMP_PROC_BIND and OMP_PLACES (thread pinning, important on NUMA machines), and OMP_DISPLAY_ENV=true (prints what the runtime decided).

The fpm openmp metapackage

The Fortran Package Manager knows OpenMP as a metapackage: declaring it in fpm.toml adds the right compile and link flags for the compiler in use.

name = "ompdemo"

[dependencies]
openmp = "*"

[[executable]]
name = "ompdemo"
source-dir = "app"

Source: Fortran Package Manager documentation

With the sentinel program above as app/main.f90, fpm run with gfortran in the container printed compiled with OpenMP, and a team of as many threads as there are cores (10 here), and OMP_NUM_THREADS=2 fpm run printed max threads: 2. Selecting Flang with fpm run --compiler flang-23 failed on 2026-10-19 in this container: fpm’s metapackage check compiles a small C file with clang -fopenmp, which was not available, and stopped with Invalid command line. The fallback is to pass the flags by hand, fpm run --flag "-fopenmp" --link-flag "-fopenmp", as the heatflow notes do. See Fortran Package Manager (fpm).

OpenACC

OpenACC is a directive model aimed at accelerators. Where OpenMP describes how work is divided among threads, OpenACC describes what may be run in parallel and lets the compiler choose the mapping onto gangs, workers and vector lanes. The sentinel is !$acc. The OpenACC site lists 3.4 as the latest specification.

The same SAXPY, written with OpenACC data and loop constructs:

program acc_demo
  use, intrinsic :: iso_fortran_env, only: dp => real64
  implicit none
  integer, parameter :: n = 1000
  real(dp) :: x(n), y(n), a, s
  integer :: i
  a = 2.0_dp
  x = 1.0_dp
  y = 3.0_dp
  s = 0.0_dp
  !$acc data copyin(x) copy(y)
  !$acc parallel loop
  do i = 1, n
    y(i) = y(i) + a*x(i)
  end do
  !$acc parallel loop reduction(+: s)
  do i = 1, n
    s = s + y(i)
  end do
  !$acc end data
  print '(a,f6.1,a,f8.1)', 'y(1)=', y(1), ' sum=', s
end program acc_demo

Syntax-checked, not run on an accelerator. gfortran supports -fopenacc; with it the file compiled warning-free under -std=f2023 -Wall -Wextra -pedantic -fcheck=all and ran on the host (acc_device_host), printing:

y(1)=   5.0 sum=  5000.0

Source: OpenACC: Specification (3.4), data and loop constructs

The same output is produced without -fopenacc, as the directives are then comments, which is the "same answer without the directives" rule again. GCC’s openacc module reports the implemented release:

program acc_rt
  use openacc
  implicit none
  print '(a,i0)', 'openacc_version = ', openacc_version
  print '(a,i0)', 'host devices    = ', acc_get_num_devices(acc_device_host)
  print '(a,l1)', 'on host         = ', acc_on_device(acc_device_host)
end program acc_rt
openacc_version = 201711
host devices    = 1
on host         = T

The value 201711 is the macro date of OpenACC 2.6, so GCC 16.2 implements an earlier release than the 3.4 of the specification site, which is the situation with every compiler: read the compiler’s release notes, not the site, for what works. LLVM Flang 23.1.4 has an OpenACC option, but both programs above were rejected by it when compiling for the host:

error: unsupported OpenACC operation: acc.reduction.recipe     (09_acc.f90, with reduction)
error: unsupported OpenACC operation: acc.private.recipe       (13_acc_noreduce.f90, parallel loop only)
error: failed to create the LLVM module

Source: OpenACC: Specification (3.4), runtime library routines

so on this setup OpenACC code can be checked with gfortran, and OpenACC offload is mainly used with NVIDIA’s nvfortran (flag -acc). nvfortran was not available here, so nothing about its behaviour is claimed beyond its documentation.

The constructs used:

Directive or clause Meaning

!$acc data …​ !$acc end data

a structured data region: arrays named in clauses are placed on the device for the extent of the region

copyin, copyout, copy, create

data clauses: copy to the device on entry, back on exit, both, or only allocate. present asserts the data is already there

!$acc parallel loop

starts parallel execution on the device and divides the loop; kernels asks the compiler to find parallelism itself

reduction(+: s)

as in OpenMP

gang, worker, vector, collapse(n), independent

loop mapping and assertions; independent is the programmer’s promise that iterations do not depend on each other, like do concurrent

!$acc enter data, exit data, update

unstructured data movement, equivalent to the OpenMP forms above

async, wait

queues for overlapping transfers and computation

OpenMP and do concurrent compared

Standard Fortran has its own statement, do concurrent, with locality specifiers local, local_init, shared, default(none) and the reduce specifier added in Fortran 2023 (R1130 in J3/23-007r1, within 11.1.7). The same loop in that form:

program dc_demo
  use, intrinsic :: iso_fortran_env, only: dp => real64
  implicit none
  integer, parameter :: n = 1000
  real(dp) :: x(n), y(n), s
  integer :: i
  x = 1.0_dp; y = 3.0_dp; s = 0.0_dp
  do concurrent (i = 1:n) reduce(+: s)
    y(i) = y(i) + 2.0_dp*x(i)
    s = s + y(i)
  end do
  print '(a,f8.1)', 'sum=', s
end program dc_demo

Output:

sum=  5000.0

Source: J3/23-007r1 (Fortran 2023), 11.1.7 DO CONCURRENT construct

The program compiled and ran with -std=f2023 in gfortran 16.2 and with Flang 23, which accept reduce. The detail pages are do concurrent and GPU Offload.

Aspect do concurrent OpenMP parallel do

Defined by

the Fortran standard (compilers must accept it everywhere)

a separate specification; ignored if the flag is absent

Threads

not mentioned: the statement states that iterations may run in any order, and the compiler decides whether to use threads, vectors or a GPU

explicit team, schedule, binding and number of threads

Without a flag

usually runs serially; gfortran has -ftree-parallelize-loops=N, the LLVM Flang documentation describes an option that maps it to OpenMP, and NVIDIA documents nvfortran -stdpar for GPUs (not run here; see the offload page)

serial

Data sharing

locality specifiers, with variables shared by default

many more clauses, including firstprivate, lastprivate, linear

Reductions

the reduce specifier, Fortran 2023

the reduction clause, any version, including user-defined reductions and array reductions

Tasks, SIMD, devices

none

tasks, simd, target, teams

Portability of results

the loop body must be legal in any order, enforced by constraints on what may be referenced

the programmer is responsible for races; the compiler checks little

A good rule: use do concurrent for loops that really are independent and that you want to be vectorised or offloaded on any compiler that supports it, and OpenMP or OpenACC when you need control over threads, scheduling, placement, tasks or device data. They mix: an OpenMP parallel do can contain a do concurrent loop in a procedure called from it.

Pitfalls

  • Data races. Writes to a shared scalar, or to overlapping array elements, from several threads. The program may print the right answer on a laptop and the wrong one on a server; use reduction, atomic or private.

  • Forgetting -fopenmp. The directives are silently comments and the program is serial but correct. Check the thread count at run time with omp_get_max_threads guarded by !$.

  • use omp_lib without the flag compiles and fails at link time (see above).

  • Wrong loop is parallel. Parallelising the inner, contiguous loop of a column-major nest starts a team per outer iteration; use collapse or place the directive on the outer loop.

  • Thread-unsafe procedures. Local variables with the save attribute, implicitly saved variables (initialised in the declaration) and module variables are shared by all threads.

  • Non-deterministic reductions. Floating-point sums combined in a different order give slightly different results; compare with a tolerance, not equality.

  • Nested parallelism and oversubscription. A parallel loop calling a library that is itself threaded (OpenBLAS, MPI with threads) multiplies the thread count.

  • Offload mapping mistakes. Using an array on the device without mapping it, or mapping an array section with the wrong bounds, gives wrong results or runtime errors that only the device run shows. Compile with the vendor’s -Minfo or an equivalent report.

  • Assuming a specification release. The version constants above show that a compiler’s OpenMP and OpenACC release can be several editions behind the latest.

  • Thread count in containers and batch queues. omp_get_max_threads may report the number of cores of the host, not the cores granted to you (10 in the container used for this page); set OMP_NUM_THREADS.

For the combination of MPI processes and OpenMP threads see MPI with mpi_f08; for the flags of the compilers see Compilers and Compiler Options; for other common mistakes Common Gotchas and Pitfalls.

The same idea in other languages

OpenMP’s counterparts elsewhere are Rayon’s parallel iterators in Rust, std::thread/OpenMP in C and C++, and Go goroutines; none of them is a directive on an ordinary loop, which is why Fortran code is easily annotated in place. See Rust and Go for the models of those languages.

Where the books are outdated

Area Outdated in the books, current approach

Parallel features in the language

parallel execution described through forall (obsolescent) and do concurrent as the only parallel statement; Fortran 2018 and 2023 add locality specifiers and reduce, and coarrays and collectives cover distributed memory

OpenMP and OpenACC

not covered, or mentioned only as external compiler extensions; OpenMP 6.0 is the current release (November 2024) and OpenACC 3.4 the current specification

GPU offload

not covered; target and OpenACC, and do concurrent offload in LLVM and NVIDIA compilers

Compiler context

no information on -fopenmp or on the versions each compiler implements; this page measured them on 2026-10-19

Tools

no fpm metapackage, no CMake FindOpenMP; both are used above

Standard version provenance and compiler support

Feature Introduced Checked on 2026-10-19

!$omp directives, omp_lib

OpenMP (not Fortran): OpenMP 6.0 is the latest released edition

parallel do, reduction, collapse, schedule, simd, task, workshare, atomic, critical, array reductions: compiled and run with GNU Fortran 16.2 and LLVM Flang 23.1.4, identical output

target, teams distribute parallel do, map, declare target, target enter data

OpenMP 4.0 and later

syntax and host fallback with both compilers; no accelerator available, so offload not run

OpenACC parallel loop, data

OpenACC (not Fortran), 3.4 on the specification site

gfortran 16.2 -fopenacc: compiled, run on the host; Flang 23.1.4: rejected the two examples; nvfortran not available

do concurrent, local, shared, default(none)

Fortran 2008 and 2018

both compilers

reduce locality specifier

Fortran 2023

both compilers accepted it with -std=f2023 in gfortran

Not checked here, because no compiler for them is installed: Intel ifx, NAG, HPE Cray and NVIDIA HPC compilers; and the accelerator targets of any compiler. See History, Standards and Compilers for the per-compiler support table.

References