The tidyverse and dplyr
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. The reference material consulted while preparing these pages is listed in the section bibliography. |
Checked against R 4.6.1 (2026-06-24) on 2026-10-20; R 4.6.2 is scheduled for 2026-10-29. The tidyverse is a collection of add-on packages that share one design: tables are tidy data frames, functions are small verbs whose first argument is the data and whose result is the same kind of table, and the verbs are chained with a pipe. dplyr is its package for working on a table, the one every other tidyverse page starts from, and this page is about it. It covers what the tidyverse is and what library(tidyverse) attaches, how a tibble differs from a base data frame, the little tidy evaluation that the verbs need (data masking, .data, .env and {{ }}), the verbs themselves (filter(), select(), mutate(), arrange(), summarise(), group_by() and .by, across(), rowwise(), window functions, joins, case_when()), the shelf analyses rewritten with them, a table of the base R equivalents, the real error messages, and the functions that newer releases have superseded. The packages used were checked at these versions: tidyverse 2.0.0 (the package that installs and attaches the others), dplyr 1.2.1 and tibble 3.3.1; the examples also call functions of rlang 1.3.0 (rlang::cnd_message()), and use tidyselect 1.2.1, pillar 1.11.1 and vctrs 0.7.3, which dplyr and tibble load for themselves. One example attaches the whole tidyverse, which attaches ggplot2 4.0.3, forcats 1.0.1, lubridate 1.9.5, purrr 1.2.2, readr 2.2.0, stringr 1.6.0 and tidyr 1.3.2 along with dplyr and tibble; only their startup message is shown, and their own pages are ggplot2 and the Grammar of Graphics and tidyr, purrr and stringr. magrittr 2.0.5 supplies the %>% pipe that dplyr re-exports (one example runs it), and conflicted 1.2.0 is named in prose and not run.
Every example is a script that was run with Rscript on R 4.6.1 with warnings printed as they happen (options(warn = 1)), and the lines that start with #> are what it printed, with the messages that R writes to the standard error stream (such as the startup message of a package) placed where they appeared. The scripts also pass the default lintr 3.4.0 linters; the one # nolint: comment marks the line that deliberately uses the %>% pipe. Most scripts load only dplyr, with library(dplyr, warn.conflicts = FALSE): the section on attaching the tidyverse explains the masking message that warn.conflicts = FALSE hides, and repeating it in every block would only add noise. The characters ×, ℹ and ✖ in the output are the ones R prints in a UTF-8 console, and the quotation marks around the strings of a tibble are the tibble’s own. Errors and warnings that a script shows on purpose are caught, and printed either with conditionMessage() or, where the header matters, in the form the console shows. The examples use the library-shelf scenario of the section: the data frames books and loans, the constants shelf_today and loan_days and the helper normalise_isbn() come from the scenario.R file described on the section index, and a script that needs them starts with source("scenario.R"), which reads books.csv with read.csv("books.csv") and loans.csv with read.csv("loans.csv", colClasses = "character"). books has 16 rows and loans has 60, both deliberately untidy; Data Frames cleans them with base R, and this page repeats the same analyses with dplyr so that the numbers can be compared (they agree).
This page is the first of the three data-ecosystem pages. The data frame that dplyr works on is the one of Data Frames, the data-mask idea is the one of Evaluation and Non-Standard Evaluation, and the pipe is the one of Pipes and the Native Pipe. Reshaping with tidyr, iteration with purrr and strings with stringr are in tidyr, purrr and stringr, and the same operations in data.table’s syntax are in data.table.
What the tidyverse is
The tidyverse web site calls it an opinionated collection of packages for data science whose members share one design philosophy, one grammar and common data structures (tidyverse.org). It is not a language extension and it is not part of R: every package in it is an ordinary add-on package, installed from CRAN with install.packages("tidyverse"), which installs the whole collection. What makes it recognisable is a small set of design principles, written down by its creator as the tidy tools manifesto, which describes itself as aspirational and likely to change. Its four principles are to reuse existing data structures, to compose simple functions with the pipe, to embrace functional programming, and to design for humans. Three of them are the ones that show on every line of tidyverse code.
-
Tidy data. Tidy data is a way of laying out a table so that every tool can rely on it: each variable is a column, each observation is a row and each value is a cell (the tidyr article on tidy data calls this Codd’s third normal form in statistical language). A tidy table is a plain data frame, so the first principle, reuse existing data structures, is met by the tibble, a data frame with stricter behaviour.
-
Consistent APIs. The functions of dplyr are verbs. The first argument is the table, the other arguments describe what to do in terms of the table’s columns, and the result is a table of the same kind: a tibble in, a tibble out; a grouped tibble in, a grouped tibble out. A verb either computes with the columns (
mutate(),filter(),summarise()) or selects some of them (select(),rename(),relocate()), and the helpers that select columns are the same everywhere. -
Pipes. Because every verb takes a table and returns a table, a step can be chained to the next one with a pipe, so that code reads in the order in which the work is done. The tidyverse used magrittr’s
%>%for years; since R 4.1.0 the language has its own pipe,|>, and the documentation of dplyr uses it throughout since version 1.2.0 (the NEWS). This page uses|>; the pipes page explains the pipe itself.
The principles describe the design; what is installed is a set of packages with a division of work. The core of the tidyverse is the set of nine packages that library(tidyverse) attaches. lubridate joined the core in tidyverse 2.0.0, and stringr and forcats had been added in 1.2.0 (the tidyverse NEWS).
| Package | What it provides | Where it is covered |
|---|---|---|
dplyr |
verbs for data frames: filter, select, mutate, arrange, summarise, group, join |
this page |
tibble |
the tibble, a data frame with stricter and friendlier behaviour |
this page |
tidyr |
tidying: pivot between long and wide, separate columns, nest and unnest |
|
purrr |
functional programming: |
|
stringr |
consistent functions for strings |
|
forcats |
tools for factors |
|
lubridate |
dates and times |
|
readr |
reading and writing delimited text into tibbles |
|
ggplot2 |
the grammar of graphics |
Source: tidyverse.org: Package overview
Beyond the core, the tidyverse package lists many more packages that it installs but does not attach (jsonlite, broom, httr and others; tidyverse_packages() prints the full list), and the tidyverse design is also followed by packages outside the collection. The R4DS book "R for Data Science" by Hadley Wickham and others is the community’s long introduction to this style; it is community material and is named here only as further reading.
Tidy data in practice
A table is tidy or untidy relative to the question. The shelf loans table is tidy in the sense above (one loan per row, one variable per column), but a summary of it can be laid out in either way. The script counts the loans of every member per year twice. table() returns a matrix whose column headings are the values of a variable (2025, 2026): the variable year is hidden in the layout and the cell is the value of the variable n. count() returns one row per combination, with member, year and n as columns, which is the form that filter(), group_by() and ggplot2 expect. table() leaves a missing value out unless it is asked not to, where count() keeps it as a group.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
few <- as_tibble(loans) |>
filter(member %in% c("Farid Karimi", "Ben Okafor"))
# Untidy: the years are column headings, and the cells hold a count.
table(few$member, format(few$loan_date, "%Y"))
#>
#> 2025 2026
#> Ben Okafor 5 0
#> Farid Karimi 1 1
# Tidy: one row per member and year, one column per variable.
few |> count(member, year = format(loan_date, "%Y"))
#> # A tibble: 3 × 3
#> member year n
#> <chr> <chr> <int>
#> 1 Ben Okafor 2025 5
#> 2 Farid Karimi 2025 1
#> 3 Farid Karimi 2026 1
Turning one layout into the other, wide to long and back, is what pivot_longer() and pivot_wider() of tidyr do, and they are the subject of the next page.
Attaching the tidyverse
library(tidyverse) attaches the nine core packages in one call. The startup message lists them with their versions (the header’s "tidyverse 2.0.0" is the version of the meta-package), and then reports the conflicts: names that one of the newly attached packages defines and that were already defined by a package attached earlier. A conflict is not an error, but from that moment the name refers to the tidyverse function, and the older one is hidden ("masked") and has to be called as package::name().
library(tidyverse)
#> ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
#> ✔ dplyr 1.2.1 ✔ readr 2.2.0
#> ✔ forcats 1.0.1 ✔ stringr 1.6.0
#> ✔ ggplot2 4.0.3 ✔ tibble 3.3.1
#> ✔ lubridate 1.9.5 ✔ tidyr 1.3.2
#> ✔ purrr 1.2.2
#> ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
#> ✖ dplyr::filter() masks stats::filter()
#> ✖ dplyr::lag() masks stats::lag()
#> ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
# The same report on demand, and the order of the search path.
tidyverse_conflicts()
#> ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
#> ✖ dplyr::filter() masks stats::filter()
#> ✖ dplyr::lag() masks stats::lag()
#> ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
search()[2:11]
#> [1] "package:lubridate" "package:forcats" "package:stringr"
#> [4] "package:dplyr" "package:purrr" "package:readr"
#> [7] "package:tidyr" "package:tibble" "package:ggplot2"
#> [10] "package:tidyverse"
length(tidyverse_packages())
#> [1] 31
The report names the two conflicts of an ordinary session: dplyr::filter() masks stats::filter() and dplyr::lag() masks stats::lag(). The help page adds that four conflicts are deliberately left out: dplyr’s intersect(), union(), setequal() and setdiff(), which make the base functions generic and should not harm existing code. tidyverse_conflicts() prints the same report on demand, options(tidyverse.quiet = TRUE) silences the startup message (the tidyverse 1.2.0 NEWS), and loading the conflicted package first (named here, not run) turns every conflict into an error that asks for an explicit package::name(); tidyverse 2.0.0 advertises it in the message for that reason, and omits its own report when conflicted is loaded. search() shows the order the packages were attached in: the last one attached, lubridate, comes first, and the earliest, package:stats, comes after all of them. For the base-R view of that mechanism see Scoping, Environments and Closures.
Attaching dplyr alone prints the longer message that warn.conflicts = FALSE hides: the objects filter and lag are masked from package:stats, and intersect, setdiff, setequal and union from package:base. The masking has an effect. filter(1:5, rep(1 / 3, 3)) is the moving-average filter of the stats package, and with dplyr attached it fails with no applicable method for 'filter' applied to an object of class "c('integer', 'numeric')", because dplyr’s filter() is a generic for data frames (the last line of the script in the section on errors shows it). stats::filter() and stats::lag() are still there under their qualified names.
Two habits follow. A script that uses only dplyr should attach only dplyr (library(dplyr)), so that the reader and the next maintainer see what the code depends on. And code in a package should not depend on tidyverse at all: the tidyverse package exists to install and attach the others (its few functions, such as tidyverse_conflicts(), are about the collection itself), so a package lists only the packages it calls in Imports and writes dplyr::filter() or imports the names it needs (see Packages, Libraries and Namespaces).
Tibbles
A tibble is a data frame with a different print method and a few stricter rules. Its class is c("tbl_df", "tbl", "data.frame"), so every function that accepts a data frame accepts a tibble, and base R functions such as nrow(), lm() and aggregate() work on it. A tibble is created with tibble(), converted from a data frame (or a list, a matrix) with as_tibble(), and written row by row with tribble(); all three come from the tibble package, which dplyr re-exports. read.csv() returns a plain data frame, so the scripts convert the shelf data with as_tibble(): readr’s read_csv() returns a tibble directly (see tidyr, purrr and stringr).
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
# read.csv() returns a data frame; as_tibble() converts it.
books_tbl <- as_tibble(books)
class(books_tbl)
#> [1] "tbl_df" "tbl" "data.frame"
# More than 20 rows: only the first 10 are shown. Text with blanks is quoted.
as_tibble(loans)
#> # A tibble: 60 × 5
#> loan_id isbn member loan_date return_date
#> <chr> <chr> <chr> <date> <date>
#> 1 L001 9780141439518 "Hugo Silva" 2025-01-14 2025-02-04
#> 2 L002 9780306406157 "Ben Okafor" 2025-01-16 2025-01-26
#> 3 L003 9780002000291 "Dev Patel" 2025-01-21 2025-01-22
#> 4 L004 9780005000571 "Eva Lindqvist" 2025-01-26 2025-02-21
#> 5 L005 978030640615 "Ben Okafor" 2025-02-19 2025-03-17
#> 6 L006 9780004000404 "Hugo Silva" 2025-02-28 2025-03-17
#> 7 L007 9780005000571 "Grace Liu" 2025-03-11 2025-04-04
#> 8 L008 9780001000100 "ana ruiz " 2025-03-27 2025-04-06
#> 9 L009 9780001000100 "Hugo Silva" 2025-03-30 2025-04-30
#> 10 L010 9780306406157 "Hugo Silva" 2025-04-06 2025-04-27
#> # ℹ 50 more rows
The print method shows the dimensions in the first line (# A tibble: 60 × 5), the type of every column in angle brackets under its name (<chr> character, <date>, <int>, <dbl> double, <lgl>, <fct> factor, <list>), and only as many rows as fit the options: a table with more than 20 rows prints its first 10, and a footer says how many rows (and columns, if they do not fit the width) were left out. A character column that holds a value with leading or trailing blanks, or an empty value, is printed in quotes, as member is here (it holds "ana ruiz "), which makes the padding visible. NA in a character column prints as <NA>, and in other types as NA. The print is for reading: the data are unchanged, and a tibble is converted back with as.data.frame().
How a tibble differs from a data frame
The differences are the reason for tibbles, and they are the traps of data frames that Data Frames lists, closed. The tibble package states them as invariants, a small set of rules for [[, $ and [ that always hold (tibble package documentation: Invariants).
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books_tbl <- as_tibble(books)
# [ with one column: a vector for a data frame, a tibble for a tibble.
class(books[, "year"])
#> [1] "integer"
class(books_tbl[, "year"])
#> [1] "tbl_df" "tbl" "data.frame"
# $ matches a partial name on a data frame, never on a tibble.
books$ye[1:3]
#> [1] 1974 1813 2019
tryCatch(books_tbl$ye, warning = conditionMessage)
#> [1] "Unknown or uninitialised column: `ye`."
# Only length-one values are recycled; a column may use an earlier one.
tryCatch(tibble(a = 1:3, b = 1:2), error = conditionMessage)
#> [1] "Tibble columns must have compatible sizes.\n• Size 3: Existing data.\n• Size 2: Column `b`.\nℹ Only values of size one are recycled."
tibble(x = 1:3, y = x * 2)
#> # A tibble: 3 × 2
#> x y
#> <int> <dbl>
#> 1 1 2
#> 2 2 4
#> 3 3 6
tryCatch(data.frame(x = 1:3, y = x * 2), error = conditionMessage)
#> [1] "object 'x' not found"
# Row names are dropped on conversion; ask for them as a column.
named <- data.frame(pages = c(387, 432), row.names = c("dispossessed", "pride"))
as_tibble(named, rownames = "key")
#> # A tibble: 2 × 2
#> key pages
#> <chr> <dbl>
#> 1 dispossessed 387
#> 2 pride 432
# Names are taken as given, with no check.names mangling.
names(data.frame(`my col` = 1))
#> [1] "my.col"
names(tibble(`my col` = 1))
#> [1] "my col"
| Behaviour | Base data frame | Tibble |
|---|---|---|
|
a vector (drops to the simplest type) unless |
always a tibble; |
|
partial match: returns the |
no partial matching: |
|
error, unless the lengths divide |
error: only values of size one are recycled |
|
error: |
works: columns are created in order and a later one may use an earlier one |
row names |
kept; |
removed by |
names such as |
mangled by |
kept as given; a duplicated name is an error unless |
text columns |
|
the same: tibbles never create factors on their own |
list columns |
possible, awkward to build and print |
first-class: printed with the type and length of each element |
printing |
the whole table, up to |
dimensions, column types and a window of rows |
The partial matching of $ deserves emphasis because it is silent in a data frame: books$ye returns year today and NULL, with no warning, as soon as a second column starting with ye is added. A tibble warns, and the warning text is the one in the table. Setting options(warnPartialMatchDollar = TRUE) makes base R warn too. Subsetting rows with [ follows the same pattern: a tibble keeps the type, and drop = FALSE is never needed.
Looking inside a tibble
glimpse() is the tibble’s answer to str(): one line per column, transposed so that wide tables fit, with the type and the first values. print() takes n (rows) and width, and the pillar package, which draws the tables, reads the options pillar.print_max, pillar.print_min, pillar.width and pillar.sigfig. tribble() writes a small table in rows, with the column names as formulas (~from_date), and is how the lookup tables of the examples below are written.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books_tbl <- as_tibble(books)
# glimpse(): one line per column, so wide tables fit.
glimpse(books_tbl)
#> Rows: 16
#> Columns: 6
#> $ title <chr> "The Dispossessed", "Pride and Prejudice", " the quiet streets…
#> $ author <chr> "Ursula K. Le Guin", "Jane Austen", "Mara Quill", "Mara Quill",…
#> $ isbn <chr> "978-0-306-40615-7", "978-0-14-143951-8", "9780001000100", "978…
#> $ year <int> 1974, 1813, 2019, 2021, 2016, 2008, NA, 1999, 2012, 2023, 2005,…
#> $ genre <chr> "Science fiction", "Classic", "Fiction", "fiction", "Mystery", …
#> $ pages <int> 387, 432, 214, 198, 301, NA, 96, 280, 352, 256, 88, 410, 175, 3…
# print() takes n and width; the footer says what was left out.
print(books_tbl, n = 3)
#> # A tibble: 16 × 6
#> title author isbn year genre pages
#> <chr> <chr> <chr> <int> <chr> <int>
#> 1 "The Dispossessed" Ursula K. Le Guin 978-0-306-40615-7 1974 Scienc… 387
#> 2 "Pride and Prejudice" Jane Austen 978-0-14-143951-8 1813 Classic 432
#> 3 " the quiet streets " Mara Quill 9780001000100 2019 Fiction 214
#> # ℹ 13 more rows
Tidy evaluation in brief
Inside a dplyr verb, filter(books, year > 2020) refers to the column year as if it were a variable. That is data masking: the arguments are not evaluated in the calling environment but in a mask built from the columns of the table, with the calling environment behind it. It is the same idea as with() and subset() of base R, described in Evaluation and Non-Standard Evaluation, extended by the rlang package into tidy evaluation with a few tools for the cases where the mask is in the way. A second mechanism, tidy selection, is used by the arguments that choose columns (select(), rename(), relocate(), across(), pull()): there title:isbn, starts_with("t") and where(is.numeric) are not ordinary R expressions but a small selection language of the tidyselect package.
The tools, in the order in which they are needed:
-
.dataand.env. A column and a variable in the workspace with the same name collide, and the column wins: with a variableyearin the global environment,filter(books, year > year)compares the column with itself..data$yearis the column and.env$yearis the variable, whatever else exists..data[[col]]takes the column from a stringcol, which is how a function receives a column name as text. -
{{ }}("embrace"). In a function that passes its argument on to a verb,{{ var }}says: this argument holds a column name that the caller wrote unquoted, evaluate it inside the mask. It lets a function be called ascount_by(books, genre). -
all_of()andany_of(). A character vector of column names inselect()is selected withall_of(x), which fails if one is missing, orany_of(x), which ignores the missing ones. A bare character vector is ambiguous (is it a column or a variable?) and tidyselect warns that this use is deprecated.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
books$genre <- tolower(trimws(books$genre))
# A variable with the name of a column is hidden by the column.
year <- 2020
books |> filter(year > year) |> nrow()
#> [1] 0
books |> filter(.data$year > .env$year) |> nrow()
#> [1] 3
# {{ }} passes a column name through a function argument.
count_by <- function(data, var) {
data |> count({{ var }}, name = "books")
}
books |> count_by(genre) |> head(2)
#> # A tibble: 2 × 2
#> genre books
#> <chr> <int>
#> 1 classic 2
#> 2 fiction 4
# .data[[ ]] takes the column name as a string.
mean_of <- function(data, col) {
data |> summarise(mean = mean(.data[[col]], na.rm = TRUE))
}
mean_of(books, "pages")
#> # A tibble: 1 × 1
#> mean
#> <dbl>
#> 1 268.
# all_of() and any_of() select the columns named in a character vector.
wanted <- c("title", "no_such_column")
books |> select(any_of(wanted)) |> names()
#> [1] "title"
tryCatch(select(books, all_of(wanted)), error = conditionMessage)
#> [1] "ℹ In argument: `all_of(wanted)`.\nCaused by error in `all_of()`:\n! Can't subset elements that don't exist.\n✖ Element `no_such_column` doesn't exist."
The first filter() returns no rows because year > year is the column compared with itself, and the second returns the three books published after 2020. count_by() works for any column; mean_of() takes a string. The rlang pages for the .data pronoun and the embrace operator describe the details, and the data-masking topic page rlang package documentation: What is data-masking and why do I need curly-curly? gives the reasoning. A function that takes several columns passes them with ..., which needs no embrace.
The dplyr verbs
dplyr’s functions are grouped by what they work on: rows (filter(), arrange(), distinct(), slice(), count()), columns (select(), rename(), relocate(), mutate(), pull()), groups (group_by(), summarise(), reframe(), .by), and pairs of tables (the joins, bind_rows() and the set operations). The reference index and the introductory article (dplyr package documentation: Introduction to dplyr) list them all. The sections below take them in that order, using the shelf data.
filter(): choose rows
filter() keeps the rows for which the conditions are TRUE. Several conditions separated by commas are combined with &, so a comma is the idiomatic spelling of "and" (filter(a > 1, b == "x")). A row where the condition is NA is dropped, like a FALSE, which is the difference to base R’s df[df$year > 2000, ] that returns a row of NA. dplyr 1.2.0 added the companion filter_out(), which names the rows to drop and keeps the NA ones, and when_any() and when_all(), the element-wise | and & of any number of conditions; if_any() and if_all() build a condition over several columns. between(), %in% and is.na() are the usual helpers.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
books$genre <- tolower(trimws(books$genre))
# Conditions separated by commas are combined with &.
books |> filter(genre == "fiction", year >= 2018) |> select(title, year)
#> # A tibble: 4 × 2
#> title year
#> <chr> <int>
#> 1 " the quiet streets " 2019
#> 2 "Loud Roofs" 2021
#> 3 "THE LONG TABLE" 2023
#> 4 "Ten Winters" 2022
# NA is dropped by filter(); filter_out() names the rows to drop instead and
# keeps the NA rows (dplyr 1.2.0).
books |> filter(year > 2000) |> nrow()
#> [1] 12
books |> filter_out(year > 2000) |> nrow()
#> [1] 4
# when_any() combines conditions with | (dplyr 1.2.0).
books |> filter(when_any(year < 1900, pages < 100)) |> pull(title)
#> [1] "Pride and Prejudice" "Garden of Small Fires" "Paper Moons"
# if_any() and if_all() build one condition from several columns.
books |> filter(if_any(c(year, pages), is.na)) |> pull(title)
#> [1] "A Map of Salt" "Garden of Small Fires" "Ten Winters"
filter(year > 2000) keeps 12 of the 16 books and filter_out(year > 2000) leaves 4: the three older books and the one whose year is missing, so the two add up to all 16. across() inside filter() is an error in the current release; if_any() and if_all() replace it (the example at the end of the superseded functions below shows the message). slice() chooses rows by position; the other slice_*() functions are in the next section.
arrange(), distinct(), count() and slice_*()
arrange() sorts rows by one or more columns; desc() reverses one column, and missing values always come last. Character columns are sorted in the C locale, not in the locale of the session: capital letters before small letters, not the order of a dictionary. This changed in dplyr 1.1.0 (the NEWS: arrange() and group_by() now use the C locale, for speed and reproducibility), and .locale gives the order of a language when it matters (arrange(x, .locale = "en"); it needs the stringi package, which the tidyverse installs). Base R’s order() uses the locale of the session (Data Frames has the section on sorting).
distinct() keeps the first row of each combination of the columns named (all columns when none are named; .keep_all = TRUE also keeps the other columns). count() is group_by() plus summarise(n = n()) plus ungroup() in one call, with sort = TRUE to put the largest group first, wt = to sum a column instead of counting, and name = to rename the count; add_count() adds the count as a column and keeps every row. The slice_*() functions choose rows by position (slice_head(), slice_tail()), by value (slice_max(), slice_min(), which keep ties unless with_ties = FALSE) and at random (slice_sample()); all take by =.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
books$genre <- tolower(trimws(books$genre))
# arrange(): desc() reverses a column; NA always comes last.
books |> arrange(genre, desc(year)) |> select(genre, year, title) |> head(3)
#> # A tibble: 3 × 3
#> genre year title
#> <chr> <int> <chr>
#> 1 classic 1999 The Lamp at Noon
#> 2 classic 1813 Pride and Prejudice
#> 3 fiction 2023 THE LONG TABLE
# Text is sorted in the C locale (capitals first); .locale asks for a language.
tibble(x = c("b", "A", "a", "B")) |> arrange(x) |> pull(x)
#> [1] "A" "B" "a" "b"
tibble(x = c("b", "A", "a", "B")) |> arrange(x, .locale = "en") |> pull(x)
#> [1] "a" "A" "b" "B"
# distinct() and count() (with sort); slice_max() with by = for the longest
# book of each genre.
books |> distinct(genre) |> nrow()
#> [1] 6
books |> count(genre, sort = TRUE) |> head(3)
#> # A tibble: 3 × 2
#> genre n
#> <chr> <int>
#> 1 fiction 4
#> 2 non-fiction 3
#> 3 science fiction 3
books |> slice_max(pages, n = 1, by = genre) |> select(genre, title, pages)
#> # A tibble: 6 × 3
#> genre title pages
#> <chr> <chr> <int>
#> 1 science fiction Notes on Gravity 410
#> 2 classic Pride and Prejudice 432
#> 3 fiction THE LONG TABLE 256
#> 4 mystery Midnight Ledger 301
#> 5 non-fiction Rivers of Glass 352
#> 6 poetry Garden of Small Fires 96
select(), rename(), relocate() and pull(): choose and order columns
select() keeps the columns chosen with the tidyselect language: names, positions, ranges (title:isbn), exclusion with -, boolean combinations with &, | and !, and the helpers everything(), last_col(), starts_with(), ends_with(), contains(), matches(), num_range(), all_of(), any_of() and where(), which keeps the columns for which a function returns TRUE (where(is.character)). Writing new = old inside select() renames and keeps only the selection; rename() renames and keeps all; rename_with() applies a function to the names. relocate() moves columns, to the front by default and with .before or .after otherwise. pull() returns one column as a vector, by name, position (negative counts from the end) or with name = as a named vector.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
# select() takes names, ranges, exclusions and helpers.
books |> select(title:isbn) |> names()
#> [1] "title" "author" "isbn"
books |> select(-isbn, -author) |> names()
#> [1] "title" "year" "genre" "pages"
books |> select(where(is.numeric)) |> names()
#> [1] "year" "pages"
books |> select(starts_with("t") | ends_with("s")) |> names()
#> [1] "title" "pages"
# A new name inside select() renames; rename() keeps the other columns.
books |> select(book = title, year) |> names()
#> [1] "book" "year"
books |> rename(n_pages = pages) |> names()
#> [1] "title" "author" "isbn" "year" "genre" "n_pages"
books |> rename_with(toupper, .cols = c(title, author)) |> names()
#> [1] "TITLE" "AUTHOR" "isbn" "year" "genre" "pages"
# relocate() moves columns: to the front by default.
books |> relocate(genre) |> names()
#> [1] "genre" "title" "author" "isbn" "year" "pages"
books |> relocate(pages, .before = isbn) |> names()
#> [1] "title" "author" "pages" "isbn" "year" "genre"
# pull() extracts one column as a vector, optionally named.
books |> pull(pages, name = title) |> head(2)
#> The Dispossessed Pride and Prejudice
#> 387 432
mutate(): create and change columns
mutate() adds columns, replaces columns that exist under the same name and removes a column that is set to NULL. The expressions are evaluated one after the other, so a later one can use an earlier one, and a result of length one is recycled to every row (a result of any other wrong length is an error). The arguments .before and .after place new columns, and .keep says which columns survive: "all" (the default), "used" (the inputs and the results), "unused" (the columns not used) or "none" (only the results, like transmute(), which is superseded). .before, .after and .keep were experimental in dplyr 1.0.0 and became stable in 1.1.0. On a grouped table, or with .by, an expression is evaluated once per group, so mean(pages) is the group mean next to every row.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
books$genre <- tolower(trimws(books$genre))
# New columns go at the end; a later one can use an earlier one.
books |>
mutate(age = 2026L - year, decade = year %/% 10L * 10L) |>
select(title, year, age, decade) |>
head(2)
#> # A tibble: 2 × 4
#> title year age decade
#> <chr> <int> <int> <int>
#> 1 The Dispossessed 1974 52 1970
#> 2 Pride and Prejudice 1813 213 1810
# .before and .after place a column, .keep chooses the columns that survive.
books |> mutate(pages_k = pages / 1000, .after = pages) |> names()
#> [1] "title" "author" "isbn" "year" "genre" "pages" "pages_k"
books |> mutate(age = 2026L - year, .keep = "used") |> names()
#> [1] "year" "age"
books |> mutate(age = 2026L - year, .keep = "none") |> names()
#> [1] "age"
# NULL drops a column. A summary is recycled next to every row of its group.
books |>
mutate(isbn = NULL, vs_mean = pages - mean(pages, na.rm = TRUE),
.by = genre) |>
select(title, genre, pages, vs_mean) |>
head(3)
#> # A tibble: 3 × 4
#> title genre pages vs_mean
#> <chr> <chr> <int> <dbl>
#> 1 "The Dispossessed" science fiction 387 14
#> 2 "Pride and Prejudice" classic 432 76
#> 3 " the quiet streets " fiction 214 -8.67
# A result of the wrong length is an error.
tryCatch(mutate(books, z = 1:3), error = conditionMessage)
#> [1] "ℹ In argument: `z = 1:3`.\nCaused by error:\n! `z` must be size 16 or 1, not 3."
case_when() and if_else()
if_else(condition, true, false, missing = ) is the vectorised if: both branches are checked for type, so mixing an integer and a string is an error, where base ifelse() coerces silently. case_when() takes a sequence of condition ~ value formulas, tries them in order and gives, for each row, the value of the first condition that is TRUE. A row that matches no condition gets .default (NA if none is given) and so does a row whose conditions are all NA. dplyr 1.2.0 added .unmatched = "error", which turns a forgotten case into an error instead of an NA, and recode_values(), replace_values() and replace_when() for lookups by value; they supersede case_match() (deprecated in 1.2.0) and recode().
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
# if_else() has two branches and checks their types.
books |>
mutate(size = if_else(pages >= 300, "long", "short", missing = "unknown")) |>
count(size)
#> # A tibble: 3 × 2
#> size n
#> <chr> <int>
#> 1 long 6
#> 2 short 8
#> 3 unknown 2
tryCatch(if_else(c(TRUE, FALSE), 1L, "a"), error = conditionMessage)
#> [1] "Can't combine `true` <integer> and `false` <character>."
# case_when() takes the first condition that is TRUE; NA conditions do not
# match and fall to .default.
books |>
mutate(
era = case_when(
year < 1900 ~ "19th century or earlier",
year < 2000 ~ "20th century",
year >= 2000 ~ "21st century",
.default = "unknown"
)
) |>
count(era)
#> # A tibble: 4 × 2
#> era n
#> <chr> <int>
#> 1 19th century or earlier 1
#> 2 20th century 2
#> 3 21st century 12
#> 4 unknown 1
# .unmatched = "error" makes a forgotten case an error (dplyr 1.2.0).
tryCatch(
mutate(books, era = case_when(year < 1900 ~ "old", .unmatched = "error")),
error = \(e) conditionMessage(e$parent)
)
#> [1] "Each location must be matched.\n✖ Locations 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16 are unmatched."
Groups: group_by(), summarise() and .by
group_by() does not change the data: it records which columns define the groups, and the later verbs work group by group. summarise() collapses each group to one row, with one column for the grouping columns and one for every expression, and ungroup() removes the grouping. A grouped tibble prints a Groups: line with the number of groups, group_vars() returns the grouping columns and n_groups() their number. n() (the size of the group) and n_distinct() (the number of different values, which counts NA as one) are the two summaries that need no column. The result of a grouped summarise() is sorted by the grouping columns, with NA as the last group.
When summarise() groups by several columns it drops only the last one, so the result is still grouped by the others, and a message says so. The message is one of the most seen in tidyverse code, and its wording changed: dplyr 1.2.0 reworded it to be clearer (the NEWS), and a tutorial may show the older one-line form that begins with `summarise()` has grouped output by. The .groups argument chooses the grouping of the result ("drop_last", "drop", "keep", "rowwise") and silences the message.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
loans <- as_tibble(loans) |>
mutate(
member = na_if(tools::toTitleCase(trimws(member)), ""),
held = as.integer(coalesce(return_date, shelf_today) - loan_date),
overdue = held > loan_days
)
# One row per group, sorted by the key; NA is a group of its own.
by_member <- loans |> group_by(member)
group_vars(by_member)
#> [1] "member"
by_member |>
summarise(loans = n(), books = n_distinct(isbn)) |>
tail(2)
#> # A tibble: 2 × 3
#> member loans books
#> <chr> <int> <int>
#> 1 Hugo Silva 10 8
#> 2 <NA> 1 1
# With two grouping columns the result stays grouped by the first, and a
# message says so; .groups = "drop" returns an ungrouped table.
by_year <- loans |>
mutate(year = as.integer(format(loan_date, "%Y"))) |>
group_by(year, overdue)
by_year |> summarise(loans = n())
#> `summarise()` has regrouped the output.
#> ℹ Summaries were computed grouped by year and overdue.
#> ℹ Output is grouped by year.
#> ℹ Use `summarise(.groups = "drop_last")` to silence this message.
#> ℹ Use `summarise(.by = c(year, overdue))` for per-operation grouping
#> (`?dplyr::dplyr_by`) instead.
#> # A tibble: 5 × 3
#> # Groups: year [3]
#> year overdue loans
#> <int> <lgl> <int>
#> 1 2025 FALSE 22
#> 2 2025 TRUE 20
#> 3 2026 FALSE 5
#> 4 2026 TRUE 12
#> 5 NA NA 1
by_year |> summarise(loans = n(), .groups = "drop") |> class()
#> [1] "tbl_df" "tbl" "data.frame"
The .by argument
A grouping that sticks to the table is a source of bugs: an ungroup() that was forgotten changes the meaning of the next verb. dplyr 1.1.0 added .by (and by in the slice_*() functions), which groups for one operation only: the result is ungrouped, and the argument works in mutate(), summarise(), reframe(), filter() and slice_*(). A second difference is the order: group_by() sorts the groups by their keys, and .by keeps the order in which the groups first appear. .by was experimental when it appeared and the 1.2.0 NEWS lists it as stable. Giving .by to a table that is already grouped is an error.
reframe() (new in 1.1.0, stable in 1.2.0) is summarise() for the case where each group yields any number of rows; since dplyr 1.2.0 a summarise() expression that returns more or fewer than one row is an error that points to reframe(), and reframe() always returns an ungrouped table.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
loans <- as_tibble(loans) |>
mutate(
member = na_if(tools::toTitleCase(trimws(member)), ""),
held = as.integer(coalesce(return_date, shelf_today) - loan_date),
overdue = held > loan_days
) |>
filter(!is.na(member), !is.na(overdue))
# .by groups for this operation only: the result is not grouped, and the
# groups keep the order of first appearance.
loans |>
summarise(loans = n(), mean_held = mean(held), .by = member) |>
head(3)
#> # A tibble: 3 × 3
#> member loans mean_held
#> <chr> <int> <dbl>
#> 1 Hugo Silva 10 22.8
#> 2 Ben Okafor 5 23.2
#> 3 Dev Patel 6 33.7
# Mixing .by with group_by() is an error.
tryCatch(
loans |> group_by(member) |> summarise(n = n(), .by = overdue),
error = conditionMessage
)
#> [1] "Can't supply `.by` when `.data` is a grouped data frame."
# reframe() returns any number of rows per group.
loans |> reframe(q = quantile(held, c(0.25, 0.75)), .by = overdue)
#> # A tibble: 4 × 2
#> overdue q
#> <lgl> <dbl>
#> 1 FALSE 10
#> 2 FALSE 18.5
#> 3 TRUE 25
#> 4 TRUE 36
across(), pick(), if_any() and if_all(): the same thing on many columns
across(.cols, .fns) applies a function (or a named list of functions) to a tidyselect selection of columns, inside mutate() or summarise(). The result columns keep the input name when one function is given and get name_fn when a list is given; the .names argument is a glue pattern ({.col} by default for one function, {.col}_{.fn} for several, or your own such as "median_{.col}") that controls the names. across() was added in dplyr 1.0.0 (the 1.0.0 NEWS), and the family of scoped verbs with the suffixes _if, _at and _all is superseded by across() and pick(). Two details of the current release matter. Extra arguments for .fns must be put in an anonymous function, \(x) mean(x, na.rm = TRUE): passing them through the ... of across() was deprecated in dplyr 1.1.0, and the script shows the warning. And across() is not for filter(): if_any() and if_all() combine a predicate over several columns with | and &. pick() (new in 1.1.0) selects columns and returns them as a data frame, for functions that take a whole data frame; it replaces cur_data() and across() without .fns.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
books$genre <- tolower(trimws(books$genre))
# across(.cols, .fns): the same function on several columns, chosen here by
# type, with the new names built from a pattern.
books |>
summarise(
across(
where(is.numeric),
\(x) median(x, na.rm = TRUE),
.names = "median_{.col}"
),
.by = genre
) |>
head(2)
#> # A tibble: 2 × 3
#> genre median_year median_pages
#> <chr> <dbl> <dbl>
#> 1 science fiction 2014 387
#> 2 classic 1906 356
# In mutate(), across() transforms columns in place.
books |> mutate(across(c(title, author), trimws)) |> pull(title) |> head(2)
#> [1] "The Dispossessed" "Pride and Prejudice"
# Extra arguments in the dots of across() are deprecated since dplyr 1.1.0.
tryCatch(
summarise(books, across(c(year, pages), mean, na.rm = TRUE)),
warning = \(w) cat(conditionMessage(w), "\n")
)
#> There was 1 warning in `summarise()`.
#> ℹ In argument: `across(c(year, pages), mean, na.rm = TRUE)`.
#> Caused by warning:
#> ! The `...` argument of `across()` is deprecated as of dplyr 1.1.0.
#> Supply arguments directly to `.fns` through an anonymous function instead.
#>
#> # Previously
#> across(a:b, mean, na.rm = TRUE)
#>
#> # Now
#> across(a:b, \(x) mean(x, na.rm = TRUE))
# pick() passes selected columns on as a data frame.
books |>
mutate(n_missing = rowSums(is.na(pick(year, pages)))) |>
filter(n_missing > 0) |>
select(title, n_missing)
#> # A tibble: 3 × 2
#> title n_missing
#> <chr> <dbl>
#> 1 A Map of Salt 1
#> 2 Garden of Small Fires 1
#> 3 Ten Winters 1
The same ideas are described in the column-wise operations article.
rowwise(): one row at a time
Most dplyr and R functions are vectorised, so a calculation across columns is written as arithmetic (back - start) or with pmin() and pmax(). A function that collapses its input, such as max() or median(), sees the whole column and returns one number. rowwise() treats every row as a group of one, so the function is called once per row; c_across() gathers the columns of the current row into a vector. A rowwise table prints as Rowwise:, and ungroup() removes the mode. For sums and means, rowSums() and rowMeans() on pick() are faster than a rowwise computation, and the column-wise across() is faster than both when the operation is the same for every column. The row-wise operations article is the full reference.
library(dplyr, warn.conflicts = FALSE)
# The day each loan started, was due and came back.
days <- tibble(start = c(1, 3, 7), due = c(22, 24, 28), back = c(21, 30, 8))
# max() collapses its input, so it needs rowwise(); c_across() collects the
# columns of one row. For sums, rowSums() on pick() is much faster.
days |>
rowwise() |>
mutate(
latest = max(start, due, back),
span = diff(range(c_across(start:back)))
) |>
ungroup() |>
mutate(total = rowSums(pick(start:back)))
#> # A tibble: 3 × 6
#> start due back latest span total
#> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 1 22 21 22 21 44
#> 2 3 24 30 30 27 57
#> 3 7 28 8 28 21 43
Window functions
A window function takes a vector and returns a vector of the same length, so it can be used in mutate() and filter() next to the rows, on the whole table or within groups. Three families are used most. The offsets lag() and lead() look at the previous and next row (the first or last is the default, NA); dplyr’s lag() masks stats::lag(), the time-series function with the same name, and both are for a different job. The cumulative functions are cumsum(), cummean(), cummax() and friends. The ranking functions differ in how they treat ties: row_number() breaks them by position, min_rank() gives tied values the lowest rank and leaves gaps (1, 1, 3), dense_rank() leaves no gaps (1, 1, 2), percent_rank() rescales the minimum rank to the range 0 to 1 and ntile(x, n) cuts the ordered values into n buckets of nearly equal size. The ranks are ascending: min_rank(desc(x)) ranks the largest first. Since mutate() and filter() apply the function per group when the data are grouped or .by is given, "the longest loan of each member" is filter(held == max(held), .by = member). The window functions article is the long version.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
ana <- as_tibble(loans) |>
mutate(
member = tools::toTitleCase(trimws(member)),
held = as.integer(coalesce(return_date, shelf_today) - loan_date)
) |>
filter(member == "Ana Ruiz", !is.na(loan_date)) |>
arrange(loan_date)
# lag() and lead() look at the previous and the next row; cumsum() and
# row_number() give running values.
ana |>
mutate(
since_previous = as.integer(loan_date - lag(loan_date)),
next_held = lead(held),
running_days = cumsum(held),
n_so_far = row_number()
) |>
select(loan_id, since_previous, held, next_held, running_days, n_so_far) |>
head(3)
#> # A tibble: 3 × 6
#> loan_id since_previous held next_held running_days n_so_far
#> <chr> <int> <int> <int> <int> <int>
#> 1 L008 NA 10 4 10 1
#> 2 L014 41 4 37 14 2
#> 3 L015 28 37 23 51 3
# The ranking functions treat ties differently.
tibble(held = c(30, 10, 10, 20)) |>
mutate(
row_number = row_number(held),
min_rank = min_rank(held),
dense_rank = dense_rank(held),
ntile2 = ntile(held, 2)
)
#> # A tibble: 4 × 5
#> held row_number min_rank dense_rank ntile2
#> <dbl> <int> <int> <int> <int>
#> 1 30 4 4 3 2
#> 2 10 1 1 1 1
#> 3 10 2 1 1 1
#> 4 20 3 3 2 2
Joins
A join combines the columns of two tables by matching rows on a key. In x |> left_join(y) the table x is on the left. Mutating joins add the columns of y to x: left_join() keeps every row of x, right_join() every row of y, inner_join() only the rows that match in both, and full_join() all of them; a row without a match gets NA in the columns that came from the other table. Filtering joins use y only to select the rows of x: semi_join() keeps the rows that have a match, anti_join() the rows that have none (the base-R idiom x[x$k %in% y$k, ] and x[!x$k %in% y$k, ]). cross_join() returns every combination.
The key is given by by: a character vector (by = "isbn"), or join_by(), the specification of dplyr 1.1.0, which takes unquoted names and operators (join_by(isbn == book_isbn) for different names). Without by, the join uses the columns the two tables have in common and prints Joining with `by = join_by(isbn)`; a script should name the key. Columns with the same name that are not keys get the suffix (.x and .y by default). dplyr preserves the order of the rows of x, where merge() sorts on the key. The join compares the key columns exactly, so a key of two different types is an error (isbn as text in loans and a number in another table) and a key written differently in two tables silently matches nothing: the shelf ISBNs have to be normalised with normalise_isbn() first, as in Data Frames.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books) |>
mutate(isbn = normalise_isbn(isbn), title = trimws(title))
loans <- as_tibble(loans)
slim <- select(books, isbn, title, genre)
# The four mutating joins differ only in which rows they keep.
c(
inner = nrow(inner_join(loans, slim, by = "isbn")),
left = nrow(left_join(loans, slim, by = "isbn")),
right = nrow(right_join(loans, slim, by = "isbn")),
full = nrow(full_join(loans, slim, by = "isbn"))
)
#> inner left right full
#> 59 60 60 61
# Without by, the common columns are used and dplyr says so.
left_join(loans, slim) |> select(loan_id, title) |> head(2)
#> Joining with `by = join_by(isbn)`
#> # A tibble: 2 × 2
#> loan_id title
#> <chr> <chr>
#> 1 L001 Pride and Prejudice
#> 2 L002 The Dispossessed
# Filtering joins add no columns: the loan with no book, the book never lent.
anti_join(loans, slim, by = "isbn") |> select(loan_id, isbn)
#> # A tibble: 1 × 2
#> loan_id isbn
#> <chr> <chr>
#> 1 L005 978030640615
anti_join(slim, loans, by = "isbn") |> pull(title)
#> [1] "Rivers of Glass"
# bind_rows() stacks tables by column name; .id records the source.
bind_rows(
first = slice_head(loans, n = 1),
last = slice_tail(loans, n = 1),
.id = "which"
) |>
select(which, loan_id)
#> # A tibble: 2 × 2
#> which loan_id
#> <chr> <chr>
#> 1 first L001
#> 2 last L060
# Keys with different names are joined with join_by(); unmatched = "error"
# turns a lost row into an error.
catalogue <- rename(slim, book_isbn = isbn)
loans |>
left_join(catalogue, by = join_by(isbn == book_isbn)) |>
select(loan_id, title) |>
head(2)
#> # A tibble: 2 × 2
#> loan_id title
#> <chr> <chr>
#> 1 L001 Pride and Prejudice
#> 2 L002 The Dispossessed
tryCatch(
inner_join(loans, slim, by = "isbn", unmatched = "error"),
error = conditionMessage
)
#> [1] "Each row of `x` must have a match in `y`.\nℹ Row 5 of `x` does not have a match."
The four row counts reproduce the ones of merge() in the base-R page (59, 60, 60, 61): the loan with the short ISBN has no book, and the book without an ISBN has no loan. anti_join() finds both without a join result to look through, and unmatched = "error" makes a lost row an error that names the first offending row.
Duplicate keys: relationship
When a key is not unique in y, or in x, a row is matched more than once and the result has more rows than x. dplyr 1.1.1 added relationship, which states the expected cardinality ("one-to-one", "one-to-many", "many-to-one", "many-to-many") and signals an error when the data do not satisfy it; without it, an equality join checks only for the rarer and almost always unintended many-to-many case and warns. The 1.1.1 NEWS records the history: dplyr 1.1.0 warned about both one-to-many and many-to-many matches, which was too often, so 1.1.1 limited the warning to many-to-many (which can multiply the rows beyond recognition) and added relationship; relationship = "many-to-one" is the replacement for multiple = "error", and the values "error" and "warning" of multiple were deprecated in 1.1.1 and are now defunct. The remedy for a many-to-many warning is to find the duplicate key (count(y, key) |> filter(n > 1)), then either make the key unique, join a different table, or declare the relationship.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
# Two copies of one book: the key is not unique in y.
copies <- tibble(
isbn = c("9780141439518", "9780141439518", "9780306406157"),
copy = c("A", "B", "A")
)
two_books <- as_tibble(loans) |>
filter(isbn %in% copies$isbn) |>
select(loan_id, isbn)
nrow(two_books)
#> [1] 10
# Several loans per book in x and two copies in y: many-to-many. The join
# multiplies the rows (10 become 13) and warns.
tryCatch(left_join(two_books, copies, by = "isbn"), warning = conditionMessage)
#> [1] "Detected an unexpected many-to-many relationship between `x` and `y`.\nℹ Row 1 of `x` matches multiple rows in `y`.\nℹ Row 3 of `y` matches multiple rows in `x`.\nℹ If a many-to-many relationship is expected, set `relationship =\n \"many-to-many\"` to silence this warning."
# relationship = states the expected cardinality and checks it.
tryCatch(
left_join(two_books, copies, by = "isbn", relationship = "many-to-one"),
error = conditionMessage
)
#> [1] "Each row in `x` must match at most 1 row in `y`.\nℹ Row 1 of `x` matches multiple rows in `y`."
two_books |>
left_join(copies, by = "isbn", relationship = "many-to-many") |>
nrow()
#> [1] 13
# With a unique key in y the row count is kept.
two_books |>
left_join(
distinct(copies, isbn, .keep_all = TRUE),
by = "isbn",
relationship = "many-to-one"
) |>
nrow()
#> [1] 10
join_by(): inequality, rolling and overlap joins
join_by() is not only an equality. It accepts <, <=, > and >= for inequality joins, closest() for rolling joins (the nearest match), and between(), within() and overlaps() for ranges. The condition has to be written in terms of the columns of the two tables; an arbitrary expression such as due - 40 >= date is not allowed, so compute the column with mutate() first. Inequality joins return all the matches by default, and keep defaults to keeping both key columns for them. This is the dplyr form of the SQL range join and of data.table’s on = with a non-equi condition. See join_by and the two-table verbs article.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
# An inequality join: each loan takes the rate in force on its due date,
# that is, the closest start date that is not after it (a rolling join).
rates <- tribble(
~from_date, ~fine_per_day,
as.Date("2025-01-01"), 0.10,
as.Date("2025-09-01"), 0.15,
as.Date("2026-03-01"), 0.20
)
as_tibble(loans) |>
mutate(due = loan_date + loan_days) |>
left_join(rates, by = join_by(closest(due >= from_date))) |>
select(loan_id, due, fine_per_day) |>
slice(c(1, 40, 60))
#> # A tibble: 3 × 3
#> loan_id due fine_per_day
#> <chr> <date> <dbl>
#> 1 L001 2025-02-04 0.1
#> 2 L040 2025-12-09 0.15
#> 3 L060 2026-06-18 0.2
# between() matches a value to a range; an unmatched row gets NA.
eras <- tribble(
~era, ~from, ~to,
"before 1900", 1000, 1899,
"20th century", 1900, 1999,
"21st century", 2000, 2099
)
as_tibble(books) |>
left_join(eras, by = join_by(between(year, from, to))) |>
count(era)
#> # A tibble: 4 × 2
#> era n
#> <chr> <int>
#> 1 20th century 2
#> 2 21st century 12
#> 3 before 1900 1
#> 4 <NA> 1
The rolling join gives each loan the rate in force on its due date: the latest start date that is not after it. The between() join puts each book in the era whose range holds its year, and the book without a year matches none.
Binding tables and set operations
bind_rows() stacks tables by column name (a column missing from one table becomes NA, and .id records the table each row came from) and bind_cols() places them side by side (the row counts must match, or be one). union(), intersect(), setdiff(), union_all() and symdiff() treat the rows of two tables as sets. coalesce() takes the first non-missing value of several vectors and na_if() turns a value into NA; the pair cleans the shelf data below.
The shelf analyses with dplyr
The analyses of Data Frames (clean the data, flag the overdue loans, count by month, summarise by genre) take about the same number of lines with dplyr, and read in the order they run. Each script below repeats the setup it needs, so that it runs on its own.
Cleaning books and loans
mutate() is the cleaning verb. The titles lose their blanks with trimws(), the authors have repeated blanks collapsed, and the two spellings of Le Guin are unified with replace_values() of dplyr 1.2.0, a lookup by value (it takes old ~ new formulas, so no condition is repeated, and is how case_match() is replaced). The genre is lower-cased, the ISBN is made bare with the scenario’s normalise_isbn(), and an empty ISBN becomes NA with na_if(). The result of the genre count is the base page’s: six genres, with 4 fiction books.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
# books: trim blanks, one spelling per author and genre, a bare ISBN-13
# (NA when the field was empty).
books <- as_tibble(books) |>
mutate(
title = trimws(title),
author = gsub("\\s+", " ", trimws(author)),
author = replace_values(author, "Le Guin, Ursula K." ~ "Ursula K. Le Guin"),
genre = tolower(trimws(genre)),
isbn = na_if(normalise_isbn(isbn), "")
)
books |> count(genre)
#> # A tibble: 6 × 2
#> genre n
#> <chr> <int>
#> 1 classic 2
#> 2 fiction 4
#> 3 mystery 2
#> 4 non-fiction 3
#> 5 poetry 2
#> 6 science fiction 3
books |> filter(author == "Ursula K. Le Guin") |> pull(title)
#> [1] "The Dispossessed" "The Lamp at Noon"
books |> filter(is.na(isbn)) |> select(title, isbn)
#> # A tibble: 1 × 2
#> title isbn
#> <chr> <chr>
#> 1 Rivers of Glass <NA>
# loans: tidy the member names; an empty name is NA.
loans <- as_tibble(loans) |>
mutate(member = na_if(tools::toTitleCase(trimws(member)), ""))
loans |> distinct(member) |> arrange(member) |> pull(member)
#> [1] "Ana Ruiz" "Ben Okafor" "Chloe Martin" "Dev Patel"
#> [5] "Eva Lindqvist" "Farid Karimi" "Grace Liu" "Hugo Silva"
#> [9] NA
Overdue loans
A loan is held until its return date, or until shelf_today when the book is still out (coalesce()), and is overdue when it was held for more than loan_days (21) days. The counts are the ones of the base-R page: of 60 loans 32 are overdue, 27 are not and one (the impossible date) is NA. The filter() on held < 0 | is.na(held) lists the two rows that need a decision, and the last query lists the overdue loans that have not come back, longest first (seven in all; the script shows the first three).
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
# Days held: until the return date, or until shelf_today when the book is
# still out. A loan is overdue when it was held for more than loan_days.
loans <- as_tibble(loans) |>
mutate(
member = na_if(tools::toTitleCase(trimws(member)), ""),
held = as.integer(coalesce(return_date, shelf_today) - loan_date),
overdue = held > loan_days
)
loans |> count(overdue)
#> # A tibble: 3 × 2
#> overdue n
#> <lgl> <int>
#> 1 FALSE 27
#> 2 TRUE 32
#> 3 NA 1
# Rows that need a look: a return before the loan, an impossible date.
loans |>
filter(held < 0 | is.na(held)) |>
select(loan_id, loan_date, return_date, held)
#> # A tibble: 2 × 4
#> loan_id loan_date return_date held
#> <chr> <date> <date> <int>
#> 1 L012 2025-04-23 2025-04-20 -3
#> 2 L020 NA 2025-07-15 NA
# The overdue loans that have not come back, longest first.
loans |>
filter(overdue, is.na(return_date)) |>
arrange(desc(held)) |>
select(loan_id, member, held) |>
head(3)
#> # A tibble: 3 × 3
#> loan_id member held
#> <chr> <chr> <int>
#> 1 L054 Chloe Martin 84
#> 2 L055 Eva Lindqvist 81
#> 3 L056 Dev Patel 71
Loans per month
The month is a text key ("2025-01"), and count() counts the loans in it. The 59 loans with a valid date fall in 17 distinct months, the same 17 months that cut(loans$loan_date, "month") gave in Dates and Times, and each month has a loan. When a month has none, count() has no row for it; filling the gaps is tidyr::complete() or a join to a table of all months. slice_max() keeps ties, so three busiest months would be six rows without with_ties = FALSE.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
# Loans per month; the loan on 2025-02-30 has no date and is left out.
per_month <- as_tibble(loans) |>
filter(!is.na(loan_date)) |>
mutate(month = format(loan_date, "%Y-%m")) |>
count(month, name = "loans")
per_month |> head(3)
#> # A tibble: 3 × 2
#> month loans
#> <chr> <int>
#> 1 2025-01 4
#> 2 2025-02 2
#> 3 2025-03 3
c(months = nrow(per_month), loans = sum(per_month$loans))
#> months loans
#> 17 59
per_month |> slice_max(loans, n = 3, with_ties = FALSE)
#> # A tibble: 3 × 2
#> month loans
#> <chr> <int>
#> 1 2025-07 7
#> 2 2025-10 7
#> 3 2025-01 4
Overdue rate and loan duration by genre
The loans are joined to the genre of their book. The overdue share per genre drops the loan with no overdue flag and computes the count and the share in one summarise(), with .by. The result is the table of the base-R page: poetry has the highest share (4 of 6), science fiction the lowest (7 of 15), and fiction is the biggest genre (23 loans, 12 overdue). The columns are named n_overdue and share, not overdue, for the reason shown in the pitfall below. The mean duration is computed for the loans that came back with a plausible duration (not negative, not NA); the means are those that the ggplot2 page uses for its error bars: 21.4 days for classic, 19.3 for fiction, 20.8 for mystery, 21.2 for non-fiction, 20.0 for poetry and 20.5 for science fiction.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books) |>
mutate(genre = tolower(trimws(genre)), isbn = normalise_isbn(isbn))
loans <- as_tibble(loans) |>
mutate(
held = as.integer(coalesce(return_date, shelf_today) - loan_date),
overdue = held > loan_days
) |>
inner_join(select(books, isbn, genre), by = "isbn")
# Overdue rate by genre; the loan with no overdue flag is dropped.
loans |>
filter(!is.na(overdue)) |>
summarise(
loans = n(),
n_overdue = sum(overdue),
share = round(n_overdue / loans, 2),
.by = genre
) |>
arrange(desc(share), genre)
#> # A tibble: 6 × 4
#> genre loans n_overdue share
#> <chr> <int> <int> <dbl>
#> 1 poetry 6 4 0.67
#> 2 classic 5 3 0.6
#> 3 non-fiction 5 3 0.6
#> 4 fiction 23 12 0.52
#> 5 mystery 4 2 0.5
#> 6 science fiction 15 7 0.47
# Mean duration by genre, for returned loans with a plausible duration.
loans |>
mutate(duration = as.integer(return_date - loan_date)) |>
filter(!is.na(duration), duration >= 0) |>
summarise(n = n(), mean_days = mean(duration), .by = genre) |>
arrange(genre)
#> # A tibble: 6 × 3
#> genre n mean_days
#> <chr> <int> <dbl>
#> 1 classic 5 21.4
#> 2 fiction 18 19.3
#> 3 mystery 4 20.8
#> 4 non-fiction 5 21.2
#> 5 poetry 5 20
#> 6 science fiction 13 20.5
A pitfall: a summary hides its own input
In summarise() (and mutate()) the expressions are evaluated in order, and a name created by one is visible to the next ones. If a summary is named like the column it summarises, the expressions after it see the summary instead of the column: mean(overdue) after overdue = sum(overdue) is the mean of a single number, 32. There is no error and no warning.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
loans <- as_tibble(loans) |>
mutate(
held = as.integer(coalesce(return_date, shelf_today) - loan_date),
overdue = held > loan_days
) |>
filter(!is.na(overdue))
# A name created in summarise() hides the column for the expressions after
# it: mean(overdue) is the mean of the number just made.
loans |> summarise(overdue = sum(overdue), share = mean(overdue))
#> # A tibble: 1 × 2
#> overdue share
#> <int> <dbl>
#> 1 32 32
# Give the summary a new name, or put the expression that needs the column
# first.
loans |> summarise(share = mean(overdue), n_overdue = sum(overdue))
#> # A tibble: 1 × 2
#> share n_overdue
#> <dbl> <int>
#> 1 0.542 32
Give the summary a new name, or put the expression that needs the column first.
Putting it together
The last script computes the base-R aggregate of Data Frames and the dplyr one side by side, and compares them: the two agree on every number.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
# Base R, as on the data frames page.
books$isbn <- normalise_isbn(books$isbn)
books$genre <- tolower(trimws(books$genre))
end <- loans$return_date
end[is.na(end)] <- shelf_today
loans$held <- as.integer(end - loans$loan_date)
loans$overdue <- loans$held > loan_days
joined <- merge(loans, books[c("isbn", "genre")], by = "isbn")
joined$n <- 1L
joined$late <- as.integer(joined$overdue)
by_genre <- aggregate(cbind(loans = n, overdue = late) ~ genre, joined, sum)
# dplyr: join once, drop the loans with no overdue flag, summarise by genre.
by_genre_tbl <- as_tibble(loans) |>
inner_join(select(as_tibble(books), isbn, genre), by = "isbn") |>
filter(!is.na(overdue)) |>
summarise(loans = n(), overdue = sum(overdue), .by = genre) |>
arrange(genre)
# The two agree.
by_genre_tbl
#> # A tibble: 6 × 3
#> genre loans overdue
#> <chr> <int> <int>
#> 1 classic 5 3
#> 2 fiction 23 12
#> 3 mystery 4 2
#> 4 non-fiction 5 3
#> 5 poetry 6 4
#> 6 science fiction 15 7
all.equal(as.data.frame(by_genre_tbl), by_genre)
#> [1] TRUE
dplyr and base R side by side
The table is the dplyr article on dplyr and base R lined up with the base-R operations of Data Frames. A dplyr verb takes and returns a data frame; the base-R forms work on vectors and are written with [, $ and a function for each job, which is the main reason the dplyr code is shorter and more regular. The script after the table checks six of the pairs: it computes the dplyr result and the base result and compares them (ignoring the class and the row names, the two things a tibble does differently).
| dplyr | Base R | Differences |
|---|---|---|
|
|
dplyr sorts text in the C locale unless |
|
|
|
|
|
|
|
|
names the rows to drop; keeps the |
|
|
|
|
|
tidyselect helpers ( |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ties are kept by default in |
|
|
|
|
|
|
|
|
the formula |
|
|
a group statistic as a column next to every row. |
|
|
|
|
|
the base form assigns the columns back; |
|
|
|
|
nested |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
with several key columns the base form is much harder. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
see the table of tibble invariants. |
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
books$genre <- tolower(trimws(books$genre))
bk <- as.data.frame(books)
# same() ignores the class and the row names, which a tibble and a data
# frame treat differently.
same <- function(x, y) {
x <- as.data.frame(x)
y <- as.data.frame(y)
rownames(x) <- rownames(y) <- NULL
isTRUE(all.equal(x, y))
}
same(arrange(books, year, title), bk[order(bk$year, bk$title), ])
#> [1] TRUE
same(filter(books, year > 2000), subset(bk, year > 2000))
#> [1] TRUE
same(mutate(books, age = 2026L - year), transform(bk, age = 2026L - year))
#> [1] TRUE
same(select(books, title, year), bk[c("title", "year")])
#> [1] TRUE
dp <- books |>
summarise(pages = mean(pages, na.rm = TRUE), .by = genre) |>
arrange(genre)
same(dp, aggregate(pages ~ genre, data = bk, FUN = mean))
#> [1] TRUE
identical(count(books, genre)$n, as.vector(table(books$genre)))
#> [1] TRUE
The vignette makes the same point from the dplyr side: base R solutions use [ in many different ways where dplyr has a single-purpose verb for each job, and per-group operations take many forms in base R and one form in dplyr. For how the base functions behave in detail see Data Frames (merge(), aggregate(), split(), order()) and The apply Family and Functionals. dplyr is not always the faster choice: for tables with millions of rows or for repeated grouped work, data.table is the usual alternative, and Performance, Profiling and Benchmarking shows how to measure rather than guess.
Common errors and warnings
dplyr’s messages are written with the cli package: a headline that starts with !, bullets starting with ℹ (information) or ✖ (the problem), and, for an error inside an argument, an In argument: bullet with the expression that failed and Caused by error: with the underlying message. conditionMessage() returns that text without the Error in `verb()`: header; the script prints each condition with rlang::cnd_message(e, prefix = TRUE), which is how the console shows it. Warnings raised inside mutate() and its variants are collected, and last_dplyr_warnings() (since dplyr 1.1.0) lists the ones of the last top-level call.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
loans <- as_tibble(loans)
# shown() prints an error or warning the way the console does, with the
# "Error in `verb()`:" header that conditionMessage() leaves out.
shown <- function(expr) {
tryCatch(
expr,
error = \(e) cat(rlang::cnd_message(e, prefix = TRUE), "\n"),
warning = \(w) cat(rlang::cnd_message(w, prefix = TRUE), "\n")
)
}
shown(filter(books, nope > 1))
#> Error in `filter()`:
#> ℹ In argument: `nope > 1`.
#> Caused by error:
#> ! object 'nope' not found
shown(select(books, nope))
#> Error in `select()`:
#> ! Can't select columns that don't exist.
#> ✖ Column `nope` doesn't exist.
shown(filter(books, genre = "fiction"))
#> Error in `filter()`:
#> ! We detected a named input.
#> ℹ This usually means that you've used `=` instead of `==`.
#> ℹ Did you mean `genre == "fiction"`?
shown(summarise(books, p = range(pages, na.rm = TRUE)))
#> Error in `summarise()`:
#> ℹ In argument: `p = range(pages, na.rm = TRUE)`.
#> Caused by error:
#> ! `p` must be size 1, not 2.
#> ℹ To return more or less than 1 row per group, use `reframe()`.
shown(left_join(loans, mutate(books, isbn = 1L), by = "isbn"))
#> Error in `left_join()`:
#> ! Can't join `x$isbn` with `y$isbn` due to incompatible types.
#> ℹ `x$isbn` is a <character>.
#> ℹ `y$isbn` is a <integer>.
# With dplyr attached, filter() is not stats::filter().
shown(filter(1:5, rep(1 / 3, 3)))
#> Error in `UseMethod()`:
#> ! no applicable method for 'filter' applied to an object of class "c('integer', 'numeric')"
| Message | Raised by | Cause and fix |
|---|---|---|
|
|
The name is neither a column nor a variable that can be found. Check it with |
|
|
A name or position that is not in the table. Tutorials show the same mistake as |
|
|
A name in the vector is missing. |
|
|
|
|
|
The new column has the wrong length; only length one is recycled. |
|
|
An expression returned a vector. Use |
|
|
The function was called outside a verb. |
|
the joins |
A |
|
the joins |
The key has different types in the two tables (character and integer here). Convert one side. |
|
the joins, |
A row has no match; the message names the first one. |
|
the mutating joins |
Both tables have repeated keys. Find the duplicates, make a key unique, or set |
|
the joins, |
|
|
|
|
|
|
The result is still grouped by all but the last column. Use |
|
|
Extra arguments were passed through the dots. Use an anonymous function. |
|
|
Use |
|
|
The two branches have different types. |
|
|
Some rows match no condition. |
|
|
No partial matching; the column does not exist. |
|
|
A column has a length that is neither 1 nor the common length. |
|
|
The call meant |
Pitfalls
-
A summary hides its own input. In
summarise(overdue = sum(overdue), share = mean(overdue)), the second expression sees the newoverdue. Name results differently. -
A grouping that was forgotten.
group_by()leaves the table grouped for every following verb, and a latermutate(),filter()orslice()works per group. Finish withungroup(), give.groups = "drop", or use.by, which never leaves a grouping behind. -
NAin conditions.filter()drops theNArows,case_when()does not match them, andmean()returnsNAunlessna.rm = TRUE.count()andgroup_by()keepNAas a group;table()drops it. -
The order of text.
arrange()andgroup_by()use the C locale: capitals first, no accents folded. Pass.localetoarrange()when a language’s order is wanted, and expect a different order from baseorder()andsort()in a non-C locale. -
Joins are exact. Differently formatted keys match nothing (the hyphens in the shelf ISBNs), duplicate keys multiply rows, and a key of two types is an error. Name
by, normalise the key first, and staterelationship. -
The masked names. With the tidyverse attached
filter()andlag()are dplyr’s, not stats'..data$xand.env$xsettle a clash between a column and a variable; a bare character vector in a selection needsall_of(). -
Partial matching is gone.
tb$yeisNULLwith a warning on a tibble and a silentyearon a data frame, and code that works on one may not work on the other. -
rowwise()is slow. It calls the expression once per row. Prefer vectorised arithmetic,pmin(),pmax(),rowSums()oracross(). -
slice_max()andslice_min()keep ties. Setwith_ties = FALSEfor exactlynrows. -
Printing is not the data. A tibble print rounds to a few significant digits (
pillar.sigfig) and cuts strings;pull()oras.data.frame()shows the stored values.
Superseded, deprecated and defunct
dplyr uses lifecycle stages. A superseded function still works, is not going to be removed, and has a better replacement; there is no warning. A deprecated function warns, and a defunct one is an error. The dplyr 1.2.0 NEWS lists many functions deprecated since 0.7.0 (2017), 1.0.0 (2020) and 1.1.0 (2023) that became defunct in that release; code from old tutorials hits them. The script runs examples of each stage.
library(dplyr, warn.conflicts = FALSE)
source("scenario.R")
books <- as_tibble(books)
# Superseded: still works, no warning, and has a shorter replacement.
old <- mutate_at(books, c("title", "author"), trimws)
new <- mutate(books, across(c(title, author), trimws))
identical(old, new)
#> [1] TRUE
# Deprecated: it still runs, with a warning.
tryCatch(
case_match(books$genre, "Classic" ~ "classic"),
warning = conditionMessage
)
#> [1] "`case_match()` was deprecated in dplyr 1.2.0.\nℹ Please use `recode_values()` instead."
# Defunct: an error.
tryCatch(summarise_each(books, mean), error = conditionMessage)
#> [1] "`summarise_each()` was deprecated in dplyr 0.7.0 and is now defunct.\nℹ Please use `across()` instead."
tryCatch(
filter(books, across(c(year, pages), \(x) x > 1)),
error = \(e) cat(conditionMessage(e), "\n")
)
#> ℹ In argument: `across(c(year, pages), function(x) x > 1)`.
#> Caused by error:
#> ! `..1` must be a logical vector, not a <tbl_df/tbl/data.frame> object.
#> ℹ If you used `across()` to generate this data frame, please use `if_any()` or
#> `if_all()` instead.
# %>% and |> give the same result; a comma and & in filter() as well.
a <- books %>% # nolint: pipe_consistency_linter.
filter(year > 2000, pages > 200)
b <- books |> filter(year > 2000 & pages > 200)
identical(a, b)
#> [1] TRUE
| Older code | Current code | Stage in dplyr 1.2.1 |
|---|---|---|
|
|
superseded (the reference pages carry the superseded badge) |
|
|
|
underscored verbs: |
the verb with tidy evaluation: |
defunct (deprecated in 0.7.0) |
|
|
defunct (deprecated in 1.0.0) |
|
|
superseded |
|
|
superseded |
|
|
superseded |
|
|
superseded |
|
|
superseded |
|
|
deprecated in 1.2.0 (soft: it warns when called from the console or a script) |
|
|
superseded |
|
|
both are current; |
|
|
defunct in 1.2.0 (deprecated in 1.1.0) |
|
|
defunct (deprecated in 1.0.8) |
|
|
extra arguments in |
|
|
soft-deprecated in 1.1.0; warns once per session in 1.2.0 |
|
|
deprecated in 1.1.0 |
|
|
defunct in 1.2.0 (deprecated in 1.1.1) |
|
|
deprecated in 1.1.0 |
|
|
soft-deprecated in 1.2.0 |
|
|
not deprecated: magrittr’s pipe is still re-exported; the native pipe is the style of the current documentation |
|
|
the same result; the comma is the convention |
Version notes
| Release | Change that affects this page |
|---|---|
dplyr 1.0.0 (2020-05-29) |
|
dplyr 1.1.0 (2023-01-29) |
|
tidyverse 2.0.0 (2023-02-22) |
lubridate joins the core packages that |
dplyr 1.1.1 (2023-03-22) |
|
dplyr 1.2.0 (2026-02-03) |
|
dplyr 1.2.1 (2026-04-03) |
compliance with R’s C API; no change to the functions on this page. |
R 4.1.0 |
the native pipe |
The release dates and the entries are those of the dplyr and tidyverse NEWS pages (dplyr NEWS, tidyverse NEWS); the R side is the R 4.1.0 entry of the R NEWS.
Where older tutorials differ
Tutorials from before 2023 show a tidyverse that has changed in several places. When one of them disagrees with this page, the table says why.
| Older tutorial | Current practice | Why | Version |
|---|---|---|---|
|
|
the language has its own pipe, and the dplyr documentation uses it; |
R 4.1.0, dplyr 1.2.0 |
|
|
lubridate is a core package |
tidyverse 2.0.0 |
|
|
per-operation grouping cannot leave a grouping behind |
dplyr 1.1.0 |
|
|
|
dplyr 1.0.0, 1.2.0 |
|
|
the dots of |
dplyr 1.1.0 |
|
|
|
dplyr 1.0.8, 1.2.0 |
|
|
the new helpers are explicit about ties and ordering |
dplyr 1.0.0 |
|
|
|
dplyr 1.1.0 |
|
|
|
dplyr 1.1.1, 1.2.0 |
|
|
|
dplyr 1.1.0, 1.2.0 |
|
the message of 1.2.1 (see the table of errors), or no message with |
the message was reworded |
dplyr 1.2.0 |
|
|
|
dplyr 1.1.0 |
|
|
|
dplyr 1.2.0 |
|
|
the embrace and the |
rlang |
|
|
a bare vector is deprecated in tidyselect |
tidyselect 1.1.0 |
|
|
the old names are defunct |
dplyr 1.2.0 |
|
|
reproducible and faster |
dplyr 1.1.0 |
For the full list of changes by release see What’s New and Migrating Older Code.
Where to go next
-
tidyr, purrr and stringr for
pivot_longer()andpivot_wider(), nesting,map(), the stringr functions, forcats, lubridate and readr, which complete the tidyverse pipeline. -
data.table for the other large data-manipulation ecosystem, with in-place modification and a terse
DT[i, j, by]syntax. -
Data Frames for the base-R versions of every operation here, and Subsetting and Indexing for the rules of
[,[[and$that tibbles tighten. -
Evaluation and Non-Standard Evaluation and Language Objects and Quoting for the mechanism behind data masking.
-
Pipes and the Native Pipe for
|>and%>%, and The apply Family and Functionals for the base alternatives toacross()androwwise(). -
ggplot2 and the Grammar of Graphics to plot the summaries, and Databases with DBI for running dplyr verbs on a database table through dbplyr.
-
Creating Packages and Packages, Libraries and Namespaces for using dplyr inside a package.
References
-
tidyverse.org and Package overview; The tidy tools manifesto, tidyverse_conflicts, tidyverse_packages and the tidyverse NEWS
-
Introduction to dplyr, Grouped data, Column-wise operations, Row-wise operations, Window functions, Two-table verbs, Programming with dplyr, Recoding columns and replacing values, dplyr <-> base R and the dplyr NEWS
-
filter, arrange, distinct, count, slice, select, rename, relocate, pull, mutate, if_else, case_when, replace_values, coalesce and na_if
-
summarise, reframe, group_by, .by, across, pick, rowwise, lead and lag, glimpse
-
tibble, as_tibble and Invariants: Comparing behavior with data frames
-
rlang package documentation: What is data-masking and why do I need curly-curly?, .data and the embrace operator