Pipes and the Native Pipe
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. The reference material consulted while preparing these pages is listed in the section bibliography. |
Checked against R 4.6.1 (2026-06-24) on 2026-10-20; R 4.6.2 is scheduled for 2026-10-29. This page is about the pipe:
what |> really is, the three things it learned in R 4.1, 4.2 and 4.3, the errors that its few rules produce, the
anonymous-function idiom that makes it flexible, and how it differs from the older %>% of the magrittr
package. The native pipe needs no package, so every example in the first sections uses base R only. The sections on the
package pipe were checked against magrittr 2.0.5, the style section against lintr 3.4.0 (two linters), and one line of
the older-tutorials section against dplyr 1.2.1 (it shows that dplyr re-exports the magrittr pipe). Those three
packages are community packages, not part of R, and are labelled as such wherever they appear.
Every example below is a complete script that was run with Rscript on R 4.6.1 with warnings printed as they happen
(options(warn = 1)), and the lines starting with #> are what it printed. The scripts also pass the default lintr
linters. Where an idiom being shown is exactly what a default linter dislikes (the %>% pipe, the %<>% assignment
pipe, ->), the script carries a # nolint comment, in the magrittr scripts as a # nolint start / # nolint end
pair, so that the linter does not hide the lesson. The names books, loans, shelf_today, loan_days come from the
scenario.R file of the running library-shelf scenario described on the
section index: a script that uses them starts with source("scenario.R"),
which reads books.csv and loans.csv with read.csv(). Several scripts call parse() on a string so that a syntax
error, which would otherwise stop the whole script, can be caught and its real message printed.
Why a pipe: reading code in the order it runs
A data analysis is a sequence of steps, each one working on the result of the previous one. Without a pipe R offers two spellings of a sequence. Nested calls are written from the inside out, so the first step is the one that is hardest to find. A chain of assignments to temporary variables reads in order, but every step needs a name. A pipe is a third spelling in which the value flows from left to right and each step is written once, in the order it runs.
The help page of the operator (?pipeOp) states the rule in one sentence: the left-hand side
lhs is inserted as the first argument of the call on the right, so x |> f(y) is interpreted as f(x, y)
(R documentation: Forward Pipe Operator). The same page
says that the operator is motivated by the pipe of the magrittr package but is more streamlined, and that it is similar
to the pipe or pipeline operators of F#, Julia and JavaScript. Three NEWS entries give the history.
| Version | What changed |
|---|---|
R 4.1.0 |
The native forward pipe |
R 4.2.0 |
A named argument of the right-hand call may be the placeholder |
R 4.3.0 |
The placeholder may be the head of a chain of extractions such as |
The Warning section of the help page adds the practical consequence: code that uses the pipe is not parsed as intended,
probably with an error, by a version of R before 4.1.0, so a package or script that uses it should say so in its
Depends: R (>= 4.1.0) field or its documentation (the _ placeholder needs 4.2.0, an extraction chain 4.3.0).
A syntax transformation, not a function
The most important fact about the native pipe is that it does not exist when the code runs. The parser rewrites
lhs |> f(args) into f(lhs, args) while it reads the text, and everything after that point, evaluation,
printing, debugging and error messages, sees only the rewritten call. The help page says that, for now, pipe operations are
implemented as syntax transformations. The first script below shows the rewrite with quote(), which returns
the parsed expression without evaluating it, and shows that there is no object named |> to be found.
source("scenario.R")
# The three spellings of one computation give the same value.
nested <- nrow(head(subset(books, year > 2000), 3L))
piped <- books |> subset(year > 2000) |> head(3L) |> nrow()
identical(nested, piped)
#> [1] TRUE
# The parser rewrites the pipe: the lhs becomes the first argument of the call.
quote(books |> head(3L))
#> head(books, 3L)
quote(books |> subset(year > 2000) |> head(3L) |> nrow())
#> nrow(head(subset(books, year > 2000), 3L))
identical(quote(books |> head(3L)), quote(head(books, 3L)))
#> [1] TRUE
# The rewritten expression is an ordinary call, so deparse() shows it.
deparse(quote(books |> head(3L)))
#> [1] "head(books, 3L)"
class(quote(books |> head(3L)))
#> [1] "call"
as.list(quote(books |> head(3L)))
#> [[1]]
#> head
#>
#> [[2]]
#> books
#>
#> [[3]]
#> [1] 3
#>
# There is no pipe object or function: the name exists only in the grammar.
exists("|>")
#> [1] FALSE
tryCatch(`|>`(books, head), error = conditionMessage)
#> [1] "could not find function \"|>\""
# A function defined with a pipe contains the rewritten call.
newest <- function(d, n = 3L) d |> head(n)
body(newest)
#> head(d, n)
# The callee sees the lhs expression as written, exactly as in a nested call.
label <- function(x) deparse(substitute(x))
books |> label()
#> [1] "books"
books |> subset(year > 2000) |> label()
#> [1] "subset(books, year > 2000)"
Reading the output from the top:
-
quote(books |> head(3L))printshead(books, 3L): the expression is already the nested call, andidentical()withquote(head(books, 3L))isTRUE. A chain of three steps becomes three nested calls, with the first step innermost (R documentation: quote). -
The class of the result is
"call"andas.list()shows a call object as a list of the function and its arguments, as for any call;deparse()turns it back into text (R documentation: deparse). Nothing in the expression tells a reader of the object that a pipe was ever written. -
exists("|>")isFALSEand calling the name as a function fails withcould not find function "|>". The token belongs to the grammar of the language, asifandfunctiondo, and unlike+, which is also an ordinary function thatReduce()can be given. A pipe therefore cannot be passed toReduce()orvapply(), redefined, masked by a package or looked up withget(). -
The body of a function that contains a pipe holds the rewritten call. The function is defined and printed with
head(d, n), notd |> head(n). Withoptions(keep.source = TRUE), the default in an interactive session, the original text is still available through the source reference, so printing a function shows the pipe in an interactive session; the body shows the call. -
A function that looks at its arguments with
substitute()sees the same expressions as in the nested spelling. Forbooks |> subset(year > 2000) |> label()the expression issubset(books, year > 2000), the full upstream call. This is why model objects, such as the one printed near the end of this page, show the nested data expression in theirCall:line, and whystop()andwarning()in a function called through a pipe name the rewritten call.
The consequences for debugging are mostly good news. There is no extra frame in the call stack, no hidden environment and no
run-time cost for the pipe itself, because there is no pipe at run time. The price is that a long chain is one deeply
nested expression for the debugger: a failing step is reported as the call that failed, for example
head(subset(books, year > 2000), 3L), rather than as the line of the chain that was written. Splitting a long chain into named intermediate objects, as the later sections show, is
the usual way to localise a failure. The page on Debugging covers the
tools themselves.
Precedence, evaluation order and visibility
Because the pipe is syntax, its position in the precedence table decides how much of the surrounding expression becomes
the left-hand side. The R Language Definition lists |> on the line of the special %xyz% operators, below
:, unary plus and minus, ^, [, $ and @, and above *, /, binary + and - and the comparison operators;
operators of equal precedence group from left to right
(The R Language Definition: Infix and prefix operators,
R documentation: Syntax).
| Written | Parsed as | Because |
|---|---|---|
|
|
|
|
|
unary minus binds tighter than |
|
|
|
|
|
|
|
|
|
|
|
parentheses on the lhs are kept as written |
The practical rule is to parenthesise any left-hand side that is an arithmetic or comparison expression:
(a + b) |> f(), not a + b |> f(), which applies f() to b alone. The next script prints the
rewrites. It then shows that evaluation follows the nested form too: an argument is a promise and is evaluated only
when the callee uses it, so a left-hand side that the callee never touches is never evaluated, and a pipeline can end in
tryCatch() to handle an error raised upstream. The help page says exactly this: pipeline code is written
sequentially, but regular R evaluation semantics apply and the piped expressions are evaluated only when first used.
source("scenario.R")
# |> has the precedence of %any%: tighter than * and +, looser than unary -.
quote(1 + 4 |> sqrt())
#> 1 + sqrt(4)
quote(2 * 4 |> sqrt())
#> 2 * sqrt(4)
quote(-4 |> sqrt())
#> sqrt(-4)
quote(a %in% b |> f())
#> f(a %in% b)
quote(1:3 |> rev())
#> rev(1:3)
quote(x == y |> f())
#> x == f(y)
quote(books$year |> head(2L))
#> head(books$year, 2L)
# Parentheses around the lhs are kept, so the call shows the extra pair.
quote((1 + 3) |> sqrt())
#> sqrt((1 + 3))
(1 + 3) |> sqrt()
#> [1] 2
# Regular evaluation rules apply: an argument is evaluated only when it is used.
ignore <- function(...) "the lhs was never used"
stop("never evaluated") |> ignore()
#> [1] "the lhs was never used"
# A pipeline can be wrapped in a handler for the same reason.
stop("evaluated now") |> tryCatch(error = conditionMessage)
#> [1] "evaluated now"
# Visibility follows the nested call: a trailing invisible() hides the result.
1 |> sqrt() |> invisible()
(1 |> sqrt() |> invisible())
#> [1] 1
The last two lines of the script show a visible consequence of the nested form. The value of invisible() is returned
without printing, and because the pipe is only a nested call, a trailing invisible() makes the whole pipeline
invisible; wrapping the pipeline in parentheses prints it. See
Lazy Evaluation and Promises for why promises make
the stop-early behaviour work, and
R documentation: invisible for the visibility rule.
The placeholder
By default the left-hand side becomes the first argument. Many functions take the data as a later argument, lm()
takes the formula first, gsub() the pattern first. R 4.2.0 added a way to say where the left-hand side goes: write the
placeholder _ as the value of a named argument of the right-hand call
(R NEWS: Changes in R 4.2.0). The parser then does not
insert the left-hand side as the first argument; it replaces the placeholder with the left-hand side. The script below
fits a regression of pages on year for the books that have both values, and then prints the rewritten forms and every
error the rules can produce.
source("scenario.R")
# By default the lhs becomes the first argument. lm() wants the formula first,
# so the data frame goes to the named argument `data` through the placeholder.
books |>
subset(!is.na(year) & !is.na(pages)) |>
lm(pages ~ year, data = _) |>
coef()
#> (Intercept) year
#> 2067.0926829 -0.8945014
# The placeholder is only a marker for the parser; the call is rewritten.
quote(books |> lm(pages ~ year, data = _))
#> lm(pages ~ year, data = books)
quote(books$title |> gsub("quiet", "calm", x = _))
#> gsub("quiet", "calm", x = books$title)
# Naming the other arguments is the alternative that needs no placeholder.
identical(
quote(books |> lm(formula = pages ~ year)),
quote(lm(books, formula = pages ~ year))
)
#> [1] TRUE
# Every misuse is a parse error; parse() lets the script show the message.
parse_error <- function(code) {
tryCatch(
{
parse(text = code)
"parsed"
},
error = conditionMessage
)
}
parse_error("x |> f(_)")
#> [1] "pipe placeholder can only be used as a named argument (<input>:1:6)"
parse_error("x |> f(a = _, b = _)")
#> [1] "pipe placeholder may only appear once (<input>:1:6)"
parse_error("x |> f(a = g(_))")
#> [1] "invalid use of pipe placeholder (<input>:1:0)"
parse_error("f(a = _)")
#> [1] "invalid use of pipe placeholder (<input>:1:0)"
parse_error("y <- _")
#> [1] "invalid use of pipe placeholder (<input>:1:0)"
parse_error("x |> f(a = _)")
#> [1] "parsed"
The rules are exactly three, and each has a message of its own.
| Rule | Example that breaks it | Real message (R 4.6.1) |
|---|---|---|
The placeholder must be the value of a named argument |
|
|
The placeholder can appear once in the right-hand call |
|
|
The placeholder must be directly the argument, not nested in another call, and it cannot be used outside a pipe |
|
|
The text in brackets is the position of the error in the parsed text: <input> is the name that parse(text = ) gives
to a string, followed by the line and column. For the invalid-use message the parser has no usable position and reports column 0. Since R 4.3.0 the error object
also carries the line and column as fields (see the common errors section).
Two consequences follow from the rules. First, a placeholder is not a variable: _ outside a pipe is a syntax
error, as the script shows. Second, the one-use limit means the left-hand side appears once in the rewritten call; to use the value in two places,
use a lambda or a temporary variable (next sections). Naming the other arguments instead of using the placeholder gives the same call without it:
books |> lm(formula = pages ~ year) is lm(books, formula = pages ~ year), since the first unmatched
argument of lm() is data. The script shows the two forms are identical expressions. The placeholder is clearer when
the position matters, and naming arguments is clearer when the argument is a well-known one.
Extraction chains (experimental)
R 4.3.0 let the placeholder be the head of a chain of extractions. The NEWS entry presents it as an experimental feature
and describes _$coef and, more generally, _$coef[[2]]
(R NEWS: Changes in R 4.3.0). The help page of the operator
documents the feature without the label, listing the extraction functions that may form the chain: $, [, [[ and
@. Because the NEWS calls it experimental, this page treats an extraction chain as a feature that R Core Team may
still change, and recommends it only for short throwaway pipelines; the stable spelling is a call, for example
|> with(title) or a lambda. The script below extracts from a fitted model and from the shelf data.
source("scenario.R")
# EXPERIMENTAL since R 4.3.0: the placeholder may start a chain of extractions.
fit <- lm(pages ~ year, data = books)
fit |> _$coefficients
#> (Intercept) year
#> 2067.0926829 -0.8945014
fit |> _$coefficients[["year"]]
#> [1] -0.8945014
books |> subset(year > 2000) |> _$title
#> [1] " the quiet streets " "Loud Roofs" "Midnight Ledger"
#> [4] "A Map of Salt" "Rivers of Glass" "THE LONG TABLE"
#> [7] "Paper Moons" "Notes on Gravity" "Orchard Logic"
#> [10] "The Slow Comet" "Ten Winters" "Salt and Static"
# The rewrite is a plain extraction on the lhs.
quote(fit |> _$coefficients[["year"]])
#> fit$coefficients[["year"]]
quote(books |> _[["title"]][1:2])
#> books[["title"]][1:2]
books |> _[["title"]][1:2]
#> [1] "The Dispossessed" "Pride and Prejudice"
# Only $, [, [[ and @ may follow the placeholder, and it may not be called.
parse_error <- function(code) {
tryCatch(
{
parse(text = code)
"parsed"
},
error = conditionMessage
)
}
parse_error("x |> _$f(1)")
#> [1] "pipe placeholder cannot be used in the RHS function (<input>:1:6)"
parse_error("x |> f(a = _$b)")
#> [1] "invalid use of pipe placeholder (<input>:1:0)"
parse_error("x |> _")
#> [1] "The pipe operator requires a function call as RHS (<input>:1:6)"
parse_error("x |> _$a[[2]]")
#> [1] "parsed"
parse_error("x |> _@slot")
#> [1] "parsed"
The rewrite is the plain extraction on the left-hand side: fit |> _$coefficients[["year"]] is
fit$coefficients[["year"]]. The errors show how narrow the feature is. The extraction functions are not calls to
_ in the sense of the other rules: x |> _$f(1) would call the extracted element, and the parser
rejects it with pipe placeholder cannot be used in the RHS function. A bare x |> _ is a right-hand side
that is not a call. The placeholder inside an extraction in an ordinary argument, f(a = _$b), is
invalid use of pipe placeholder. See
R documentation: Extract for the four extraction
operators.
The right-hand side must be a call
The first rule of the operator is in its argument list: the right-hand side rhs is a call expression. The parser checks this when it
reads the pipe, and a right-hand side that is not a call is a syntax error, not a run-time error. The commonest
mistake, from habit with the magrittr pipe, is to write a bare function name.
source("scenario.R")
parse_error <- function(code) {
tryCatch(
{
parse(text = code)
"parsed"
},
error = conditionMessage
)
}
# The right-hand side must be a call, so the parentheses are not optional.
parse_error("books |> head")
#> [1] "The pipe operator requires a function call as RHS (<input>:1:10)"
parse_error("books |> head()")
#> [1] "parsed"
# A function expression, a lambda or a parenthesised name is not a call.
parse_error("books |> function(d) head(d)")
#> [1] "function 'function' not supported in RHS call of a pipe (<input>:1:10)"
parse_error("books |> \\(d) head(d)")
#> [1] "function 'function' not supported in RHS call of a pipe (<input>:1:10)"
parse_error("books |> (head)")
#> [1] "function '(' not supported in RHS call of a pipe (<input>:1:10)"
parse_error("books |> {head(books)}")
#> [1] "function '{' not supported in RHS call of a pipe (<input>:1:10)"
# Special forms and operators cannot be the function that is called.
parse_error("books |> `+`(1)")
#> [1] "function '+' not supported in RHS call of a pipe (<input>:1:10)"
parse_error("books |> if (TRUE) head()")
#> [1] "function 'if' not supported in RHS call of a pipe (<input>:1:10)"
parse_error("books |> base::head")
#> [1] "function '::' not supported in RHS call of a pipe (<input>:1:10)"
parse_error("books |> base::head()")
#> [1] "parsed"
# A computed function is fine: namespaces, extraction, parentheses.
quote(books |> base::head(2L))
#> base::head(books, 2L)
quote(books |> (head)(2L))
#> (head)(books, 2L)
quote(books |> fns$head(2L))
#> fns$head(books, 2L)
parse_error("books |> head(2L)()")
#> [1] "parsed"
There are two different messages, and they mean different things.
| Right-hand side | Message | Meaning |
|---|---|---|
|
|
A name is not a call: add the parentheses, |
|
|
A function expression is a call to the special form |
|
|
Parentheses and braces are calls to the special forms |
|
|
|
|
|
Operators and control-flow words are syntactically special and cannot be the callee. |
The help page gives the reason: to avoid ambiguities, functions in the right-hand call may not be syntactically special,
such as + or if. A callee that is a name, a namespace-qualified name such as base::head(), an extraction such as
fns$head(), a parenthesised expression that yields a function, (head)(2L), or even a call that returns a
function, head(2L)(), is accepted, because in every case the callee is a call that is not special. The
second half of the script shows that these forms parse and are rewritten to the call you expect. Hence the answer to
“how do I pipe into +?” is: you do not; call an ordinary function such as sum(), or wrap the operator in a lambda as
the next section shows. The magrittr pipe does accept operators as the called function (a script further down
pipes into +), which is one of the places where the two pipes differ.
Anonymous functions, the lambda shorthand and Reduce
When a step is not already a function with the data as its first argument, a pipe needs an anonymous function. R 4.1.0
introduced \(x) as a shorter spelling of function(x), in the same release as the pipe
(R NEWS: Changes in R 4.1.0,
R documentation: function). Because the callee of a
pipe cannot be a function expression, the lambda has to be parenthesised and called: x |> (\(d) ...)(). The
parentheses turn the function expression into a value, and the empty argument list is the call that receives the
left-hand side. The rewrite is (function(d) ...)(x), a call whose callee is a parenthesised function.
source("scenario.R")
# \(x) is only a shorter spelling of function(x) since R 4.1.0.
quote(\(x) x + 1)
#> function(x) x + 1
(\(x) x + 1)(2)
#> [1] 3
# A lambda is called by putting it in parentheses and adding an argument list.
books |>
(\(d) d[!is.na(d$year) & d$year > 2000, c("title", "year")])() |>
head(3L)
#> title year
#> 3 the quiet streets 2019
#> 4 Loud Roofs 2021
#> 5 Midnight Ledger 2016
# The parenthesised function is the head of the rewritten call.
quote(books |> (\(d) d[d$year > 2000, ])())
#> (function(d) d[d$year > 2000, ])(books)
quote(books |> (function(d) nrow(d))())
#> (function(d) nrow(d))(books)
# A lambda can also name the position of the data, which is the job of `_` in
# a call that takes the data frame by name; both forms give the same result.
a <- books |> (\(d) lm(pages ~ year, data = d))() |> coef()
b <- books |> lm(pages ~ year, data = _) |> coef()
identical(a, b)
#> [1] TRUE
# A list of steps is applied with Reduce(); a pipe cannot loop over functions.
steps <- list(
\(d) subset(d, !is.na(year)),
\(d) d[order(d$year), ],
\(d) head(d, 3L),
\(d) d$title
)
Reduce(\(acc, step) step(acc), steps, books)
#> [1] "Pride and Prejudice" "The Dispossessed" "The Lamp at Noon"
# The same chain written with the pipe, for comparison.
books |>
subset(!is.na(year)) |>
(\(d) d[order(d$year), ])() |>
head(3L) |>
_$title
#> [1] "Pride and Prejudice" "The Dispossessed" "The Lamp at Noon"
# Reduce() can also build one function from the list, which a pipe cannot do.
compose <- function(...) {
fs <- list(...)
\(x) Reduce(\(acc, f) f(acc), fs, x)
}
oldest_titles <- compose(steps[[1L]], steps[[2L]], steps[[3L]], steps[[4L]])
oldest_titles(books)
#> [1] "Pride and Prejudice" "The Dispossessed" "The Lamp at Noon"
Source: R documentation: Reduce
When there is a list of steps, a pipe is the wrong tool because its steps are fixed in the source text. Reduce() applies
a function repeatedly, so folding a list of functions over a starting value with \(acc, step) step(acc) is a data-driven
pipeline. The same fold builds a single function from a list of steps (compose in the script). R-devel, the
development version that will become R 4.7.0, lists new higher-order functions Compose(), Funcall() and
Partial() in its NEWS; they are not part of R 4.6.1, they were not used here, and they are mentioned only so that a
reader who finds them in newer code knows where they come from
(R NEWS: Changes in R-devel).
A lambda is also the way to use the left-hand side twice, or in a position that the placeholder cannot reach, for example inside a nested call. The placeholder rules are strict on purpose; a lambda is a normal function, so it has none of these limits:
-
x |> (\(d) f(d, g(d)))()usesdtwice. -
x |> (\(d) d[order(d$year), ])()writes an expression in whichdis nested in indexing. -
x |> (\(d) d + 1)()is how an operator is applied to the left-hand side.
Pipes on the shelf
The next script uses the shelf data for what pipes are for: a sequence of base-R steps on one data frame, one step per
line. The books chain trims titles, keeps the books after 2000, sorts them by year and takes the first four. The loans
chain computes the number of days each returned loan was out with transform(), keeps the ones longer than the loan
period loan_days, and counts them per member with aggregate() and table(). aggregate() takes the formula first
and data second, so this is the case where the placeholder is needed.
source("scenario.R")
# Books published after 2000, newest first, one step per line.
books |>
transform(title = trimws(title)) |>
subset(!is.na(year) & year > 2000, select = c(title, author, year)) |>
sort_by(~ year, decreasing = TRUE) |>
head(4L)
#> title author year
#> 10 THE LONG TABLE Iris Fenwick 2023
#> 15 Ten Winters Priya Raman 2022
#> 4 Loud Roofs Mara Quill 2021
#> 13 Orchard Logic Elias Moreau 2020
# Loans: how many days was each returned loan out, and which ones are overdue?
overdue <- loans |>
transform(
member = tolower(trimws(member)),
days_out = as.integer(return_date - loan_date)
) |>
subset(!is.na(days_out) & days_out > loan_days)
nrow(overdue)
#> [1] 25
# Overdue loans per member: the placeholder sends the data to `data =`.
overdue |>
aggregate(loan_id ~ member, data = _, FUN = length) |>
sort_by(~ loan_id, decreasing = TRUE)
#> member loan_id
#> 1 ana ruiz 6
#> 3 chloe martin 5
#> 6 grace liu 4
#> 7 hugo silva 4
#> 2 ben okafor 3
#> 4 dev patel 2
#> 5 eva lindqvist 1
# The same as a table of counts, then the loans still out beyond the period.
overdue |> with(table(member))
#> member
#> ana ruiz ben okafor chloe martin dev patel eva lindqvist
#> 6 3 5 2 1
#> grace liu hugo silva
#> 4 4
loans |>
subset(is.na(return_date) & shelf_today - loan_date > loan_days) |>
nrow()
#> [1] 7
# The nested spelling reads from the inside out.
nrow(subset(
loans,
is.na(return_date) & shelf_today - loan_date > loan_days
))
#> [1] 7
# Intermediate objects with names are better when a step is worth a name.
returned <- subset(loans, !is.na(return_date) & !is.na(loan_date))
returned$days_out <- as.integer(returned$return_date - returned$loan_date)
nrow(subset(returned, days_out > loan_days))
#> [1] 25
Source: R documentation: aggregate
transform(), subset() and with() evaluate their arguments inside the data frame
(R documentation: transform,
R documentation: subset,
R documentation: with). They take the data frame as the first
argument, which is why they fit a pipe without any placeholder. sort_by() was added in R 4.4.0 and its formula
interface (~ year) also evaluates in the data frame, which makes it a good citizen of a pipeline
(R documentation: sort_by,
R NEWS: Changes in R 4.4.0). The 25 overdue returned loans
and the 7 loans still out beyond the loan period are the figures that the other pages of this reference use for the
shelf, and the last lines of the script show the same answers from nested calls and from named intermediate objects.
There is no single right spelling: the pipe is best when each step is a small transformation of one object, and a
named intermediate object is better when a step is worth a name, when two objects are combined, or when a failing
step needs inspecting.
Common errors and pitfalls
Almost every error of the native pipe is raised while R parses the code, which means a misuse is found even in code that is not executed, and it is reported with the position. Since R 4.3.0 these errors have classes, so that a program (an editor, a linter, a teaching tool) can tell the misuses apart without reading the message.
# Since R 4.3.0 the parser signals classed errors for the pipe, so code can
# tell the misuses apart without matching message text.
codes <- c(
"x |> f",
"x |> function(d) d",
"x |> f(_)",
"x |> f(a = _, b = _)",
"x |> f(a = g(_))",
"f(a = _)",
"x |> _$f(1)"
)
first_class <- vapply(codes, \(code) {
err <- tryCatch(parse(text = code), error = identity)
class(err)[[1L]]
}, character(1L))
data.frame(code = codes, class = unname(first_class))
#> code class
#> 1 x |> f RHSnotFnCall
#> 2 x |> function(d) d unsupportedInPipe
#> 3 x |> f(_) placeholderNotNamed
#> 4 x |> f(a = _, b = _) tooManyPlaceholders
#> 5 x |> f(a = g(_)) invalidPlaceholder
#> 6 f(a = _) invalidPlaceholder
#> 7 x |> _$f(1) placeholderInRHSFn
Source: R NEWS: Changes in R 4.3.0
The classes are the ones observed on R 4.6.1; the help pages of 4.6.1 do not document them (the R-devel NEWS says the
condition classes that the parser and other internals signal are now being documented in ?conditions), so rely on the
message and the first class only as an implementation detail. The next script collects the other mistakes that
are worth knowing by their symptoms.
source("scenario.R")
parse_error <- function(code) {
tryCatch(
{
parse(text = code)
"parsed"
},
error = conditionMessage
)
}
# A pipe at the start of a line: the previous line is already a complete
# expression, so the parser meets a stray operator.
parse_error("books\n |> head()")
#> [1] "<text>:2:3: unexpected '|>'\n1: books\n2: |>\n ^"
parse_error("books |>\n head()")
#> [1] "parsed"
# %>% needs magrittr (or a package that re-exports it) to be attached.
tryCatch(
eval(quote(1:3 %>% sum())), # nolint: pipe_consistency_linter.
error = conditionMessage
)
#> [1] "could not find function \"%>%\""
# The native pipe is part of the language and needs no package.
1:3 |> sum()
#> [1] 6
# The error class of the parse errors lets a program tell them apart.
err <- tryCatch(parse(text = "books |> head"), error = identity)
class(err)
#> [1] "RHSnotFnCall" "parseError" "error" "condition"
c(line = err$lineno, column = err$colno)
#> line column
#> 1 10
# Piping into a function whose first argument is not the data.
tryCatch(
books |> lm(pages ~ year),
error = conditionMessage
)
#> [1] "cannot coerce class ‘\"formula\"’ to a data.frame"
books |> subset(!is.na(year) & !is.na(pages)) |> lm(pages ~ year, data = _)
#>
#> Call:
#> lm(formula = pages ~ year, data = subset(books, !is.na(year) &
#> !is.na(pages)))
#>
#> Coefficients:
#> (Intercept) year
#> 2067.0927 -0.8945
#>
| Symptom | Cause | Fix |
|---|---|---|
|
The previous line is already a complete expression, so the parser reaches the pipe with nothing on its left (the same rule as for any binary operator, see Syntax and Lexical Structure). |
End the line with |
|
The code uses the magrittr pipe, but neither magrittr nor a package that re-exports the pipe (dplyr, the tidyverse core packages) is attached. |
Attach it, or write |
|
A bare name on the right. |
Add the parentheses, |
|
A function expression, a parenthesised name, braces, an operator or a control-flow word on the right. |
Parenthesise a lambda and call it, or call a named function. |
|
The left-hand side went into the first argument, which for |
Use |
A pipeline whose result prints nothing |
The last step returned its value invisibly. |
Wrap the pipeline in parentheses or assign the result. |
The last row of the script shows something that surprises readers of model output: the printed Call: of the lm()
fit contains data = subset(books, ...), the nested expression, not the pipe, because that is the call the function
received. Nothing about the model records that a pipe was used.
The ordinary pitfalls of lazy evaluation apply unchanged to pipes (the
Idioms and Common Pitfalls page collects the R FAQ
entries): a function that needs the left-hand side to have been evaluated, for example to test a side effect, must call
force() on its argument, exactly as it would when called with nested calls.
The magrittr pipe (package)
Before R 4.1.0 the pipe came from the magrittr package, a community package of the tidyverse project (installed here as
magrittr 2.0.5). Its %>% is an ordinary function that captures its two arguments with substitute() and
evaluates the right-hand side at run time; the package documentation describes the right-hand side as a function call
that follows the magrittr semantics
(magrittr package documentation: pipe). Its rules are therefore more
permissive than the native pipe’s, and the permissiveness is where the differences are. The script below shows the
rules, with the dot placeholder.
source("scenario.R")
library(magrittr)
# nolint start: pipe_consistency_linter.
# %>% is an ordinary function in the package namespace, not syntax.
class(`%>%`)
#> [1] "function"
environmentName(environment(`%>%`))
#> [1] "magrittr"
# A bare function name works, and the lhs is the first argument.
books %>% nrow
#> [1] 16
books %>% subset(year > 2000) %>% head(2L) %>% nrow()
#> [1] 2
# Operators can be the function that is called, which the native pipe forbids.
1:3 %>% `+`(1)
#> [1] 2 3 4
# The dot places the lhs; it can be used several times and as `.$name`.
books %>% lm(pages ~ year, data = .) %>% coef()
#> (Intercept) year
#> 2067.0926829 -0.8945014
books %>% .$title %>% head(2L)
#> [1] "The Dispossessed" "Pride and Prejudice"
books %>%
subset(!is.na(pages)) %>%
{
paste(nrow(.), "books, the longest has", max(.$pages), "pages")
}
#> [1] "14 books, the longest has 432 pages"
c(3, 4) %>% paste(., "of", length(.))
#> [1] "3 of 2" "4 of 2"
# The first-argument rule: a dot used only inside a nested call does not stop
# the lhs from also being inserted as the first argument.
c(3, 4) %>% paste(max(.))
#> [1] "3 4" "4 4"
# Braces switch the rule off, so the dot is exactly where it is written.
c(3, 4) %>%
{
paste(max(.))
}
#> [1] "4"
c(3, 4) %>%
{
c(min(.), max(.))
}
#> [1] 3 4
# A lambda is a parenthesised function; no call parentheses are needed.
books %>% (\(d) nrow(d))
#> [1] 16
# A dot as the lhs builds a function (a "functional sequence").
first_titles <- . %>%
subset(!is.na(year)) %>%
head(2L) %>%
extract2("title")
class(first_titles)
#> [1] "fseq" "function"
first_titles(books)
#> [1] "The Dispossessed" "Pride and Prejudice"
# nolint end
The package documentation describes each rule.
-
A bare name works:
x %>% fisf(x). A lambda is any parenthesised function expression, so the call parentheses are not needed. If more than one expression is needed in the body, braces turn the right-hand side into a lambda with the argument named.. -
The dot
.marks where the left-hand side goes. It can be used several times and in nested calls (.$title,length(.)), unlike the native_. -
The first-argument rule: if the dot appears only in nested calls, the left-hand side is also inserted as the first argument, so
c(3, 4) %>% paste(max(.))ispaste(c(3, 4), max(c(3, 4))), the output"3 4" "4 4"above. Braces switch the rule off. This implicit insertion is the main source of surprise when moving to the native pipe, which never does it. -
A dot as the left-hand side builds a function (a functional sequence),
. %>% f %>% g, which the native pipe has no equivalent for; write a lambda or a named function instead. -
Operators may be the called function (the script pipes into
+), and the package provides named aliases for them such asadd(),multiply_by()andextract2(), shown in the next script.
The package has three more pipes. The next script runs each of them, and the table after it gives the native replacement.
source("scenario.R")
library(magrittr)
# nolint start: pipe_consistency_linter.
# %<>% pipes the value and assigns the result back to the lhs.
pages <- c(210, 380, 95)
pages %<>% sqrt() %>% round(1L) # nolint: assignment_linter.
pages
#> [1] 14.5 19.5 9.7
# %T>% runs a side effect on the value and passes the original value on.
books %T>%
{
cat("rows:", nrow(.), "\n")
} %>%
subset(year > 2020) %>%
nrow()
#> rows: 16
#> [1] 3
# The native spelling: a lambda that returns its argument.
books |>
(\(d) {
cat("rows:", nrow(d), "\n")
d
})() |>
subset(year > 2020) |>
nrow()
#> rows: 16
#> [1] 3
# %$% exposes the columns of the lhs, like with().
books %$% mean(pages, na.rm = TRUE)
#> [1] 268.2143
books %$% table(genre)
#> genre
#> Classic fiction Fiction Fiction Mystery
#> 2 1 2 1 2
#> Non-fiction Poetry science fiction Science fiction
#> 3 2 1 2
# Aliases for operators, so that they can be used as the rhs of a pipe.
c(1, 2, 3) %>% multiply_by(2) %>% add(1)
#> [1] 3 5 7
list(a = 1, b = 2) %>% extract2("b")
#> [1] 2
# nolint end
The three operators are documented on their own pages of the package site: the compound assignment pipe (compound), the tee pipe (tee) and the exposition pipe (exposition). The operator aliases are on the aliases page.
| magrittr | What it does | Native replacement |
|---|---|---|
|
Pipes |
|
|
Runs |
Split the pipeline: call the side effect on its own line, or use a lambda that returns its argument, |
|
Exposes the names of |
|
Run time versus parse time
The deep difference between the two pipes is when they exist. The native pipe is gone after parsing, the magrittr pipe is a function call at run time, and everything else in the table that follows comes from that. The script below shows three of the consequences, the call stack, the body of a function and the call recorded in an error, and then two behaviours that changed in magrittr 2.0.0 or that remain different.
source("scenario.R")
library(magrittr)
# nolint start: pipe_consistency_linter.
# A function that returns the call stack as text shows who ran what. The frames
# that exist before the call (zero under Rscript) are dropped.
base_depth <- sys.nframe()
stack <- function(x) {
calls <- sys.calls()
calls <- calls[seq_along(calls) > base_depth]
vapply(calls, \(cl) paste(deparse(cl), collapse = " "), character(1L))
}
1 %>% stack()
#> [1] "1 %>% stack()" "stack(.)"
1 |> stack()
#> [1] "stack(1)"
# The magrittr pipe stays in the code as a call to %>%; the native pipe is gone.
body(function(d) d %>% head(2L))
#> d %>% head(2L)
body(function(d) d |> head(2L))
#> head(d, 2L)
# An error raised in the called function reports the call as it saw it.
fails <- function(x) stop("no good", call. = TRUE)
conditionCall(tryCatch(books %>% fails(), error = identity))
#> fails(.)
conditionCall(tryCatch(books |> fails(), error = identity))
#> fails(books)
# Both pipes are lazy since magrittr 2.0.0: an unused lhs is never evaluated,
# and a trailing handler can catch an error raised on the left.
ignore <- function(...) "the lhs was never used"
stop("never evaluated") %>% ignore()
#> [1] "the lhs was never used"
stop("never evaluated") |> ignore()
#> [1] "the lhs was never used"
stop("evaluated now") %>% tryCatch(error = conditionMessage)
#> [1] "evaluated now"
stop("evaluated now") |> tryCatch(error = conditionMessage)
#> [1] "evaluated now"
# return() inside a braced magrittr step has no function to return from.
early <- function(flag) {
flag %>% {
if (.) { # nolint: object_usage_linter.
return("yes")
}
"no"
}
}
tryCatch(early(TRUE), error = conditionMessage)
#> [1] "no function to return from, jumping to top level"
# nolint end
-
The call stack contains the
%>%call, and the step is called asstack(.)with the dot, so a traceback has an extra frame and the error call isfails(.). With the native pipe the stack contains onlystack(1), and the error call isfails(books). The magrittr 2.0.0 NEWS says the rewrite of the pipe in C was done for minimal cost and minimal impact on backtraces, which is why the stack is short; the native pipe has no impact at all. -
The function body keeps
d %>% head(2L)for the package pipe and showshead(d, 2L)for the native one. A static tool sees a call to a function named%>%in the first case, which is why theobject_usage_linterof lintr reported the dot in the braced example as a variable with no visible binding (the script carries a# nolintfor it); it sees the plain call in the second. -
Before magrittr 2.0.0 the pipe evaluated its left-hand sides eagerly. Since 2.0.0 it evaluates lazily, as the native pipe does, so an unused left-hand side is never evaluated and a trailing
tryCatch()ortry()handles an error raised upstream. The 2.0.0 NEWS lists, as side effects of that change, a trailinginvisible()hiding the whole result andreturn()inside a braced step: it now fails with the message shown at the end of the script (no function to return from, jumping to top level), where the native pipe, which does not accept braces on the right at all, never reaches that situation. -
Speed. The native pipe costs nothing at run time because nothing is left to run. The package pipe adds one function call, implemented in C since 2.0.0. In a private micro-benchmark on the machine used for this page, three trivial nested steps were indistinguishable from the native pipe and about three times slower with the package pipe, well under a microsecond per step. That is irrelevant for a data-analysis pipeline and relevant only for a pipe inside a loop that runs many millions of times. No timing is printed here because it depends on the machine.
| Feature | Native |> (base R, 4.1.0) | magrittr %>% (package) |
|---|---|---|
Kind |
Syntax, rewritten by the parser |
Ordinary function, evaluated at run time |
Right-hand side |
A call: |
A call, a bare name, a parenthesised function, or braces |
Placeholder |
|
|
First-argument rule |
None: the placeholder replaces it |
If the dot appears only in nested calls the left-hand side is also inserted first |
Anonymous function |
|
|
Functional sequence |
None |
|
Operators as callee |
Not allowed |
Allowed, plus aliases such as |
Other pipes |
None |
|
Evaluation |
Lazy |
Lazy since 2.0.0 (eager in 1.5) |
Error call and stack |
Shows the nested call, no extra frame |
Shows |
Needs a package |
No, R 4.1.0 or later |
Yes (and attached, or |
When the package pipe is still used
The tidyverse style guide, a community document, recommends the base pipe over magrittr’s and says that as of R 4.3.0 the base pipe provides all the features of magrittr that it recommends using (tidyverse style guide, community: Pipes). The package pipe is still found and still reasonable in these cases:
-
Code that has to run on R before 4.1.0: a package that supports older versions of R cannot use
|>. -
Older code bases written with
%>%throughout, where converting is churn and a file should not mix both pipes (thepipe_consistency_linterbelow enforces that). -
Pipelines that really use the dot several times or in nested calls, braces as lambdas, functional sequences or the three extra pipes.
-
Code that other people write with the same habit, since
%>%is re-exported by dplyr, which the tidyverse attaches (see the older-tutorials section).
Everywhere else, prefer |>: it has no dependency, no run-time cost, a simpler stack and a stricter set of
rules that are checked at parse time.
Readable pipes
The tidyverse style guide (a community document, not part of R) has a short chapter on pipes (tidyverse style guide, community: Pipes). Its rules, applied by the script below, are:
-
Use
|>to emphasise a sequence of actions rather than the object that the actions are performed on. Do not use it when several objects are manipulated at once, or when a meaningful intermediate object deserves an informative name. -
A space before
|>, and usually a new line after it; after the first step, each line is indented by two spaces. This makes it easier to add or rearrange steps and harder to overlook one. -
If the arguments of a function do not all fit on one line, put each argument on its own line and indent.
-
A short pipe may be written on one line, but short pipes tend to grow, so the guide generally recommends one function per line. A pipe of a single step is just a call: prefer the plain call unless the arguments span several lines, when the guide recommends the pipe even for a single step.
-
Assignment may be written as
result <-on its own line,result <- data |>on the same line, or at the end of the pipe with->. The guide lists the third form as acceptable but notes that it makes reading harder, because the name that acts as a heading comes last; this page uses the first two.
source("scenario.R")
# One step per line, the pipe at the end of the line, two spaces of indentation.
recent <- books |>
subset(!is.na(year) & year > 2000, select = c(title, year)) |>
sort_by(~ year) |>
head(3L)
recent
#> title year
#> 11 Paper Moons 2005
#> 6 A Map of Salt 2008
#> 9 Rivers of Glass 2012
# A short pipe may stay on one line, but it tends to grow.
books |> subset(year > 2015) |> nrow()
#> [1] 8
# A pipe of one step is just a call.
head(books, 2L)
#> title author isbn year genre
#> 1 The Dispossessed Ursula K. Le Guin 978-0-306-40615-7 1974 Science fiction
#> 2 Pride and Prejudice Jane Austen 978-0-14-143951-8 1813 Classic
#> pages
#> 1 387
#> 2 432
# Put the long argument lists on their own lines.
loans |>
transform(
member = tolower(trimws(member)),
days_out = as.integer(return_date - loan_date)
) |>
subset(!is.na(days_out) & days_out > loan_days, select = loan_id) |>
head(3L)
#> loan_id
#> 4 L004
#> 5 L005
#> 7 L007
# Name an intermediate object when it deserves a name.
returned <- loans |> subset(!is.na(return_date))
nrow(returned)
#> [1] 53
# lintr checks the continuation and consistency rules (lintr 3.4.0).
bad_continuation <- "books |> subset(year > 2000) |>\n head(2L)"
mixed_pipes <- "books |> head(2L) %>% nrow()"
lintr::lint(
text = bad_continuation,
linters = lintr::pipe_continuation_linter()
)
#> <text>:1:30: style: [pipe_continuation_linter] Put a space before `|>` and a new line after it, unless the full pipeline fits on one line.
#> books |> subset(year > 2000) |>
#> ^~
lintr::lint(
text = mixed_pipes,
linters = lintr::pipe_consistency_linter()
)
#> <text>:1:19: style: [pipe_consistency_linter] Use the |> pipe operator instead of the %>% pipe operator.
#> books |> head(2L) %>% nrow()
#> ^~~
The lintr package (3.4.0, also a community package) checks two of these rules by default.
pipe_continuation_linter() reports a step that does not follow the pipe on a new line, unless the entire pipeline fits
on one line; its message, shown at the end of the script, says to put a space before the pipe and a new line after it
(lintr package documentation: pipe_continuation_linter).
pipe_consistency_linter() reports the use of the %>% pipe when |> is the configured one, which is the default;
with pipe = "auto" it only enforces that one file uses one pipe
(lintr package documentation: pipe_consistency_linter).
The default assignment_linter objects to the right-pointing arrow and to the assignment pipe
(lintr package documentation: assignment_linter), which is why
the older-tutorials script carries a # nolint on its -> line. The page on
Style, Linting and Static Analysis shows how to run
and configure the linters.
Where older tutorials differ
Tutorials written before R 4.1.0 teach %>% because there was no alternative, and later ones often copy them. A reader who meets these habits should know why they are being replaced.
| Older tutorial | Current practice | Why | Version |
|---|---|---|---|
|
|
Base R has had a pipe since 4.1.0; it needs no package, is rewritten by the parser, and its errors are found at parse time |
R 4.1.0 |
The dot |
|
The native pipe has no implicit first-argument rule and no multi-use of the lhs. A lambda or a temporary variable is explicit |
R 4.2.0 for |
|
|
The right-hand side of the native pipe must be a call; a bare name is a parse error |
R 4.1.0 |
|
|
|
R 4.1.0 |
|
|
Extraction chains on the placeholder arrived in 4.3.0 and are described as experimental in the NEWS |
R 4.3.0 |
|
|
The native pipe has no extra operators; the replacements are plain assignment and |
always |
Nested calls, |
A pipe, or named intermediate objects |
Reading order follows execution order; a name is better than |
always |
Assignment at the end of the chain with |
|
The name acts as a heading, and the default |
always |
A pipe at the start of the next line |
The pipe at the end of the line |
A line that starts with |
always |
The last script contrasts the old spellings with the current one and checks that they give the same answer.
source("scenario.R")
# Nested calls read from the inside out.
nested <- nrow(head(subset(books, year > 2000), 3L))
# Overwriting one temporary variable at every step.
tmp <- subset(books, year > 2000)
tmp <- head(tmp, 3L)
temporary <- nrow(tmp)
# The current spelling: one step per line, read from top to bottom.
piped <- books |>
subset(year > 2000) |>
head(3L) |>
nrow()
# Assignment at the end of the chain with the right-pointing arrow.
books |>
subset(year > 2000) |>
head(3L) |>
nrow() -> arrow_result # nolint: assignment_linter.
c(nested, temporary, piped, arrow_result)
#> [1] 3 3 3 3
# dplyr re-exports magrittr's %>%, so old tutorials get the pipe by attaching
# dplyr or the tidyverse without ever loading magrittr.
identical(dplyr::`%>%`, magrittr::`%>%`)
#> [1] TRUE
See also
-
Functions for function calls, arguments and
\(x)in general. -
Lazy Evaluation and Promises for why an unused left-hand side is never evaluated.
-
Language Objects and Quoting for
quote(), calls and the rewritten expression. -
The apply Family and Functionals for
Reduce()and friends. -
The tidyverse and dplyr for pipelines of data-frame verbs, which are written with the native pipe too.
-
Style, Linting and Static Analysis for lintr and styler.
-
Syntax and Lexical Structure for parse errors and the operator tokens.