Missing and Special Values
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. The reference material consulted while preparing these pages is listed in the section bibliography. |
Checked against R 4.6.1 (2026-06-24) on 2026-10-20; R 4.6.2 is scheduled for 2026-10-29. Everything on this page is base R (the base, stats and utils packages that ship with R), so no add-on package version applies; the lintr 3.4.0 default linters were used only to check the scripts, and the add-on functions that people reach for next (tidyr::drop_na(), tidyr::replace_na(), dplyr::coalesce()) are named in prose only and are not run.
Statistics is the field where R was born, and a statistical language has to answer a question that most languages avoid: what do you store when a value exists in the world but you do not know it? R’s answer is a family of special values. NA is the marker for a value that is not available, NaN is the result of arithmetic that has no sensible answer, Inf and -Inf are the infinities of IEEE 754 arithmetic, and NULL is the absence of an object altogether. They look alike on the console and they are tested in different ways, and most of the errors that beginners meet (missing value where TRUE/FALSE needed, argument is of length zero, a mean that comes out as NA) are one of these four being used as if it were another. This page separates the four, shows the rule by which NA spreads through a computation, lists the tools that remove, count and fill missing values, and ends with the zero-length vectors that NULL has as relatives and the checks that keep a condition from receiving one.
Every example below is a script that was run with Rscript on R 4.6.1 with warnings printed as they happen (options(warn = 1)), and the lines starting with #> are what it printed. The scripts pass the default lintr linters; where an idiom being shown is exactly what a default linter dislikes (x == NA and x %in% NA) the line carries a # nolint: comment. A script is shown in several consecutive blocks, one per topic, and a later block uses the variables of the earlier ones. Errors and warnings that a script shows on purpose are caught with tryCatch() and printed as their conditionMessage(), so the text is exactly what R says; a message that contains a line break shows it as \n. The examples use the library scenario of the section: books, loans, shelf_today, loan_days and the book class come from the scenario.R file described on the section index, and the scripts that use them start with source("scenario.R"). The shelf has real gaps on purpose: books$year has one NA, books$pages has two, loans$loan_date has one (the impossible date 2025-02-30), loans$return_date has seven (the loans still out), and books$isbn has one empty string "" that is not an NA.
Four things that are called "missing"
The manuals keep the four notions apart, and so must you. An Introduction to R introduces NA in the section on missing values: when a value is not available in the statistical sense, a place in the vector is reserved for it with the special value NA; any operation on an NA becomes an NA, because if the specification of an operation is incomplete the result cannot be known; and is.na(x) is the function that finds them (An Introduction to R: Missing values). The R Language Definition adds that this is the statistical sense of "missing" and must not be confused with the missing property of a function argument that was not supplied (The R Language Definition: NA handling); that other meaning is on Functions. NULL has its own subsection in the Language Definition: a special object used whenever it is necessary to indicate that an object is absent, and it should not be confused with a vector or list of zero length (The R Language Definition: NULL).
| Value | Type | Length | Meaning | Test |
|---|---|---|---|---|
|
logical; typed variants |
1 |
a value that exists but is not known; a marker, not a number |
|
|
double (and the parts of a complex) |
1 |
the result of a numerical computation that is undefined, such as |
|
|
double |
1 |
a number beyond the largest double, or a limit; an ordinary extreme value |
|
|
|
0 |
the absence of an object: no value, not a value that is unknown |
|
The first script puts the four side by side on the shelf. The page counts of "A Map of Salt" and "Ten Winters" are unknown, so their cells are NA; the vector still has 16 elements and each unknown keeps its place. NULL, on the other hand, has no place at all: c(10, NULL, 30) is a vector of length 2, because NULL contributes no elements. The last lines show the consequence for testing: is.na() of a NULL is an empty logical vector, not FALSE, so it cannot be used as a test for NULL (the condition of an if would be a zero-length condition; see Zero-length vectors).
source("scenario.R")
# Four different things that are all loosely "missing"
typeof(NA)
#> [1] "logical"
typeof(NaN)
#> [1] "double"
typeof(Inf)
#> [1] "double"
typeof(NULL)
#> [1] "NULL"
length(NA)
#> [1] 1
length(NaN)
#> [1] 1
length(Inf)
#> [1] 1
length(NULL)
#> [1] 0
# The shelf has real gaps: one year and two page counts are unknown
books$year
#> [1] 1974 1813 2019 2021 2016 2008 NA 1999 2012 2023 2005 2017 2020 2014 2022
#> [16] 2018
sum(is.na(books$year))
#> [1] 1
books$pages
#> [1] 387 432 214 198 301 NA 96 280 352 256 88 410 175 322 NA 244
sum(is.na(books$pages))
#> [1] 2
# A missing value is a marker, not a value
x <- c(10, NA, 30)
x[2L]
#> [1] NA
length(x)
#> [1] 3
x
#> [1] 10 NA 30
c(10, NULL, 30)
#> [1] 10 30
length(c(10, NULL, 30))
#> [1] 2
Source: The R Language Definition: NULL
# All four, side by side
kinds <- list(na = NA, nan = NaN, inf = Inf, null = NULL)
vapply(kinds, length, integer(1L))
#> na nan inf null
#> 1 1 1 0
lapply(kinds, is.na)
#> $na
#> [1] TRUE
#>
#> $nan
#> [1] TRUE
#>
#> $inf
#> [1] FALSE
#>
#> $null
#> logical(0)
#>
vapply(kinds, is.null, logical(1L))
#> na nan inf null
#> FALSE FALSE FALSE TRUE
Source: R documentation: NA
NA and its typed variants
NA is a reserved word, a logical constant of length 1 that contains a missing-value indicator, and it can be coerced to any other vector type except raw. There are also constants for the other atomic types that support missing values: NA_integer_, NA_real_, NA_complex_ and NA_character_. The help page documents that all five are reserved words in the language (R documentation: NA). The reason there are several is that an atomic vector has one type, so the missing value inside it must have that type too. The Language Definition points out the one case in which the difference matters to a user: the default NA is logical, so a bare NA used as an index triggers logical rather than numeric indexing (see Where missing values come from).
The first block shows the five constants and how a vector keeps its type when an NA joins it: c(1L, NA) is still integer, c("a", NA) is still character, because the logical NA is the lowest type in the coercion order and is promoted to the type of the other elements. The second block is the most common confusion: the character NA is not the string "NA". A vector c("NA", NA) holds a string with two letters and a missing value; is.na() finds only the second, nchar() reports 2 for the first and NA for the second, and printing without quotes shows the missing one as <NA> to tell them apart (the Introduction to R mentions this printing). Real data brings the string case from files: read.csv() treats the text NA as a missing value in every column, including text columns (its na.strings argument defaults to "NA"), which is how the country code of Namibia becomes a missing value; na.strings = character(0) keeps it as text. The block after the empty year of the shelf shows both reads (the script builds the data with a text string, so no file is needed).
# The default NA is logical; it takes the type of the vector it joins
typeof(NA)
#> [1] "logical"
typeof(NA_integer_)
#> [1] "integer"
typeof(NA_real_)
#> [1] "double"
typeof(NA_character_)
#> [1] "character"
typeof(NA_complex_)
#> [1] "complex"
Source: R documentation: NA
# The typed constants exist so that a column keeps its type
typeof(c(1L, NA))
#> [1] "integer"
typeof(c(1.5, NA))
#> [1] "double"
typeof(c("a", NA))
#> [1] "character"
typeof(c(TRUE, NA))
#> [1] "logical"
typeof(c(NA, 1L))
#> [1] "integer"
# NA of character type is not the string "NA"
x <- c("NA", NA)
x
#> [1] "NA" NA
is.na(x)
#> [1] FALSE TRUE
nchar(x)
#> [1] 2 NA
x == "NA"
#> [1] TRUE NA
print(x, quote = FALSE)
#> [1] NA <NA>
Source: R documentation: NA
# The row of the shelf with an empty year is an integer NA
year <- read.csv("books.csv")$year
typeof(year)
#> [1] "integer"
year[7L]
#> [1] NA
typeof(year[7L])
#> [1] "integer"
Source: R documentation: typeof
# read.csv() reads the string NA as missing in every column, even text
codes <- "code,country\nNA,Namibia\nUS,United States"
read.csv(text = codes)$code
#> [1] NA "US"
read.csv(text = codes, na.strings = character(0))$code
#> [1] "NA" "US"
Source: R documentation: read.table
The type of an NA is visible to identical(), which compares the type as well as the value, and to the is.*() functions: identical(NA, NA_integer_) is FALSE even though both print as NA, and is.numeric(NA) is FALSE because the plain NA is logical. That is why code that must return a typed missing value spells it out. The book() helper of the shelf scenario has author = NA_character_ and year = NA_integer_ as defaults, which is what lets validate_book() and new_book() check types with is.character() and is.integer(); and vapply() needs a template of the right type, as in vapply(x, f, NA_real_), where the NA_real_ is only a way of writing a length-one double (see The apply Family and Functionals).
# The type of an NA matters for a function that dispatches on the type
identical(NA, NA_integer_)
#> [1] FALSE
identical(NA_integer_, NA_integer_)
#> [1] TRUE
is.integer(NA)
#> [1] FALSE
is.logical(NA)
#> [1] TRUE
is.numeric(NA_integer_)
#> [1] TRUE
is.numeric(NA)
#> [1] FALSE
Source: R documentation: identical
Coercion keeps the missingness and changes the type: as.integer(NA), as.character(NA) and as.numeric(NA_character_) are all missing values of the new type, and as.character(NA) is NA_character_, never the string "NA". rep(NA_real_, 3L) is the idiom for a double vector that is to be filled later; character(2L) is two empty strings, not two missing values, and numeric(3L) is three zeros, which silently look like data. The last block shows the one type with no missing value: as.raw(NA) is 00 with a warning, because a byte cannot be unknown (the Language Definition says there is no NA for raw vectors).
# Coercing an NA gives an NA of the new type, never the string "NA"
as.integer(NA)
#> [1] NA
as.character(NA)
#> [1] NA
as.numeric(NA_character_)
#> [1] NA
as.logical(NA_integer_)
#> [1] NA
is.na(as.character(NA))
#> [1] TRUE
Source: R documentation: NA
# Making a typed vector of unknowns
rep(NA_real_, 3L)
#> [1] NA NA NA
rep(NA, 3L)
#> [1] NA NA NA
vector("character", 2L)
#> [1] "" ""
character(2L)
#> [1] "" ""
rep(NA_character_, 2L)
#> [1] NA NA
Source: R documentation: vector
# There is no NA for raw vectors: NA becomes 00 with a warning
tryCatch(as.raw(NA), warning = \(w) conditionMessage(w))
#> [1] "out-of-range values treated as 0 in coercion to raw"
suppressWarnings(as.raw(c(1L, NA)))
#> [1] 01 00
is.na(as.raw(0:2))
#> [1] FALSE FALSE FALSE
Source: R documentation: raw
Finally, the help page recommends is.na<- as the safer way to set elements to missing, because it behaves sensibly for classes such as factors (is.na() versus == NA shows it). See Atomic Vectors and Types for the six atomic types and the coercion hierarchy.
NaN, Inf and -Inf
NaN ("not a number") and the infinities come from the floating-point arithmetic of the computer, not from R, and R follows IEEE 754 for doubles. Dividing a nonzero double by zero gives Inf or -Inf; log(0) is -Inf; a computation that overflows, such as exp(1000), is Inf; and an operation with no defined result (0/0, Inf - Inf, Inf * 0) is NaN. The Language Definition says that NaN exists only in the double type and in the real or imaginary components of the complex type (The R Language Definition: NA handling). An integer has no way to store either: integer division by zero is NA (5L %/% 0L), while the double 5 %/% 0 is Inf and 5 %% 0 is NaN, as described on Numbers and Arithmetic.
A function called outside its domain returns NaN and says so with the warning NaNs produced, which is the same warning for sqrt(-1) and log(-1). It is a warning because the result is a legitimate NaN, so the computation continues; the script catches it to print its text.
# Where NaN and Inf come from: arithmetic that has no sensible answer
0 / 0
#> [1] NaN
Inf - Inf
#> [1] NaN
Inf * 0
#> [1] NaN
1 / 0
#> [1] Inf
-1 / 0
#> [1] -Inf
5 %% 0
#> [1] NaN
5 %/% 0
#> [1] Inf
5L %/% 0L
#> [1] NA
5L %% 0L
#> [1] NA
log(0)
#> [1] -Inf
exp(1000)
#> [1] Inf
.Machine$double.xmax * 10
#> [1] Inf
Source: R documentation: Arithmetic
# Warnings: a function outside its domain gives NaN and says so
tryCatch(sqrt(-1), warning = \(w) conditionMessage(w))
#> [1] "NaNs produced"
tryCatch(log(-1), warning = \(w) conditionMessage(w))
#> [1] "NaNs produced"
suppressWarnings(sqrt(c(4, -1, 9)))
#> [1] 2 NaN 3
Source: R documentation: MathFun
The four predicates are summarised by the next block. They answer different questions: is.na() is TRUE for NA and for NaN, is.nan() is TRUE only for NaN, is.finite() is TRUE only for ordinary numbers (it is FALSE for NA, NaN and both infinities), and is.infinite() is TRUE only for the infinities. Be careful with the direction of the relation: every NaN is NA as far as is.na() is concerned, but an NA is not a NaN, so is.nan(NA) is FALSE. If you need to tell the cases apart, test is.nan() first, or use is.na(x) & !is.nan(x) for “a true NA”.
# The four tests, on the same six values
v <- c(1, NA, NaN, Inf, -Inf, 0)
v
#> [1] 1 NA NaN Inf -Inf 0
rbind(
is_na = is.na(v),
is_nan = is.nan(v),
is_finite = is.finite(v),
is_infinite = is.infinite(v)
)
#> [,1] [,2] [,3] [,4] [,5] [,6]
#> is_na FALSE TRUE TRUE FALSE FALSE FALSE
#> is_nan FALSE FALSE TRUE FALSE FALSE FALSE
#> is_finite TRUE FALSE FALSE FALSE FALSE TRUE
#> is_infinite FALSE FALSE FALSE TRUE TRUE FALSE
Source: R documentation: is.finite
# NaN is a kind of NA to is.na(), but NA is not a NaN
is.na(NaN)
#> [1] TRUE
is.nan(NA)
#> [1] FALSE
is.nan(NA_real_)
#> [1] FALSE
is.nan(c(1, NA, NaN))
#> [1] FALSE FALSE TRUE
Source: R documentation: is.finite
The help page for NA warns about one more difference between the two. A numerical computation that mixes an NA and a NaN (for example NA_real_ + NaN) may give either, depending on the platform and the compiler, and the help page says this is not guaranteed and may change on future CPUs, so this page does not print such a result and your code should not depend on it (R documentation: NA). Test for both with is.na() and do not rely on which of the two a mixed computation yields.
Comparisons follow the same rule as NA: NaN is not comparable, even to itself, so NaN == NaN is NA. The infinities, in contrast, are ordinary values that can be compared, so Inf == Inf is TRUE and Inf > 1e308 is TRUE; max() and range() return them without complaint, which is why a -Inf or Inf in a result is a clue that a maximum or minimum was taken over an empty vector (see Zero-length vectors). Three rules hold for every argument, NA and NaN included, because the answer does not depend on the unknown: NA^0 is 1, 1^NA is 1, and Inf^0 is 1, as documented on the help page of the arithmetic operators. NA * 0, in contrast, is NA, not 0: R applies no algebraic shortcut to an unknown, only the documented rules.
# Comparisons: NaN is incomparable, Inf is an ordinary extreme
NaN == NaN # nolint: equals_na_linter.
#> [1] NA
Inf == Inf
#> [1] TRUE
Inf > 1e308
#> [1] TRUE
-Inf < 0
#> [1] TRUE
Inf > NA
#> [1] NA
max(c(1, Inf, NA), na.rm = TRUE)
#> [1] Inf
range(c(2, Inf, -Inf, 7))
#> [1] -Inf Inf
Source: R documentation: Comparison
# Rules that hold for all arguments
NA^0
#> [1] 1
1^NA
#> [1] 1
NA * 0
#> [1] NA
NaN^0
#> [1] 1
Inf^0
#> [1] 1
Source: R documentation: Arithmetic
is.finite() and is.infinite() accept any vector: they return FALSE for strings (a string is not a finite number), TRUE for finite integers and logicals, and logical(0) for NULL. Converting loses the distinction: as.integer(NaN) is NA, and as.integer(Inf) is an NA with the warning NAs introduced by coercion to integer range, because an integer cannot hold it; as.character() keeps the spellings "NaN", "Inf" and "-Inf" but turns NA into NA_character_. is.nan() is defined for numbers; on a string or NULL it returns FALSE or logical(0), and on a list it is an error.
# is.finite() and is.infinite() on other types
is.finite(c(1L, NA, 3L))
#> [1] TRUE FALSE TRUE
is.finite(c("a", "b"))
#> [1] FALSE FALSE
is.finite(TRUE)
#> [1] TRUE
is.infinite(NA)
#> [1] FALSE
is.finite(NULL)
#> logical(0)
Source: R documentation: is.finite
# Only doubles (and the parts of a complex) can hold NaN or Inf
as.integer(NaN)
#> [1] NA
suppressWarnings(as.integer(c(1.9, NaN, Inf)))
#> [1] 1 NA NA
tryCatch(as.integer(Inf), warning = \(w) conditionMessage(w))
#> [1] "NAs introduced by coercion to integer range"
as.character(c(NA, NaN, Inf, -Inf))
#> [1] NA "NaN" "Inf" "-Inf"
as.logical(NaN)
#> [1] NA
is.nan(1L)
#> [1] FALSE
is.nan("a")
#> [1] FALSE
is.nan(NULL)
#> logical(0)
tryCatch(is.nan(list(1)), error = \(e) conditionMessage(e))
#> [1] "default method not implemented for type 'list'"
Source: R documentation: integer
NULL
NULL is the one object of the type NULL. It has length 0, no attributes and no modifiable properties, and there is only one NULL in R, to which every reference points. It is what R returns when there is nothing to return: a function whose body is empty or that ends with an if without a taken branch, a for loop, invisible(NULL), a missing list component. It is also how you ask for something to be removed (x$a <- NULL, rownames(m) <- NULL) and how you say that an argument has no value (f(data = NULL)). The first block shows the type and the family resemblance to the empty vector and the empty list: they all have length 0 but they are three different objects, identical(NULL, list()) and identical(NULL, logical(0)) are FALSE, and NULL is the only one for which is.null() is TRUE.
R 4.4.0 changed one of these relationships: is.atomic(NULL) is now FALSE, because NULL is not an atomic vector. The NEWS entry notes that strict backward compatibility for old code means replacing is.atomic(foo) by is.null(foo) || is.atomic(foo), which should happen sparingly (R NEWS: Changes in R 4.4.0). In the same release NCOL(NULL) became 0 instead of 1, for consistency with cbind() (NROW and NCOL below).
# NULL is its own type, with length 0 and no attributes
typeof(NULL)
#> [1] "NULL"
class(NULL)
#> [1] "NULL"
mode(NULL)
#> [1] "NULL"
length(NULL)
#> [1] 0
is.null(NULL)
#> [1] TRUE
is.atomic(NULL)
#> [1] FALSE
is.vector(NULL)
#> [1] FALSE
is.list(NULL)
#> [1] FALSE
identical(NULL, list())
#> [1] FALSE
identical(NULL, logical(0))
#> [1] FALSE
identical(NULL, c())
#> [1] TRUE
Source: R documentation: NULL
# It is what a function returns when there is nothing to return
f <- function() {}
f()
#> NULL
is.null(f())
#> [1] TRUE
print(invisible(NULL))
#> NULL
is.null(if (FALSE) 1)
#> [1] TRUE
is.null(for (i in 1:2) i)
#> [1] TRUE
is.null(list(a = 1)$b)
#> [1] TRUE
is.null(c(a = 1)["b"])
#> [1] FALSE
Source: R documentation: NULL
NULL vanishes when it is combined: c(1, NULL, 3) is a vector of two elements, c(NULL, NULL) is NULL, and unlist() drops the NULL components of a list. R 4.1.0 made c() remove NULL arguments before dispatching to methods, which simplified the writing of c() methods, while keeping the NULL when it is the first argument for backward compatibility (R NEWS: Changes in R 4.1.0). That behaviour is useful for building a result by appending (out <- NULL followed by out <- c(out, x)), though growing a vector one element at a time is slow for large inputs (see Vectorisation and Iteration).
Extracting from NULL never fails: NULL[1L], NULL[[1L]], NULL$a and NULL[["a"]] are all NULL. That is why x$misspelled returns NULL instead of an error and a typo in a component name surfaces later, as a zero-length condition (Conditions that are not one TRUE or FALSE). The one exception, from R 4.3.0, is the empty subscript: NULL[[]] is an error, like x[[]] (R NEWS: Changes in R 4.3.0). You cannot set an attribute on NULL: attr(nothing, "unit") <- "pages" and names(nothing) <- "a" stop with attempt to set an attribute on NULL, and the same assignment written directly on the constant fails earlier with invalid (NULL) left side of assignment. Arithmetic and comparison on NULL give a zero-length vector of the type of the operation (NULL + 1 is numeric(0), NULL == 1 is logical(0)), which is one more way to end up with a condition of length zero.
# NULL vanishes from c(), and unlist() drops it
c(1, NULL, 3)
#> [1] 1 3
c(NULL, NULL)
#> NULL
c(list(1), NULL, list(3))
#> [[1]]
#> [1] 1
#>
#> [[2]]
#> [1] 3
#>
unlist(list(1, NULL, 3))
#> [1] 1 3
unlist(list(NULL, NULL))
#> NULL
length(list(1, NULL, 3))
#> [1] 3
length(c(list(1), NULL, list(3)))
#> [1] 2
Source: R documentation: c
# Indexing NULL gives NULL, not an error
NULL[1L]
#> NULL
NULL[[1L]]
#> NULL
NULL$a
#> NULL
NULL[["a"]]
#> NULL
tryCatch(NULL[[]], error = \(e) conditionMessage(e))
#> [1] "missing subscript"
Source: R documentation: Extract
# No attributes can be set on it
nothing <- NULL
tryCatch(
attr(nothing, "unit") <- "pages",
error = \(e) conditionMessage(e)
)
#> [1] "attempt to set an attribute on NULL"
tryCatch(
names(nothing) <- "a",
error = \(e) conditionMessage(e)
)
#> [1] "attempt to set an attribute on NULL"
tryCatch(
attr(NULL, "unit") <- "pages",
error = \(e) conditionMessage(e)
)
#> [1] "invalid (NULL) left side of assignment"
attributes(NULL)
#> NULL
names(NULL)
#> NULL
Source: The R Language Definition: NULL
# Operations on NULL give zero-length results of the operation's type
NULL + 1
#> numeric(0)
NULL == 1
#> logical(0)
NULL > 1
#> logical(0)
paste("a", NULL)
#> [1] "a "
nchar(NULL)
#> integer(0)
toupper(NULL)
#> character(0)
sum(NULL)
#> [1] 0
is.na(NULL)
#> logical(0)
rev(NULL)
#> NULL
sort(NULL)
#> NULL
Source: R documentation: NULL
is.na() versus == NA
The Introduction to R states the rule in one sentence: x == NA is quite different from is.na(x), because NA is not a value but a marker for a quantity that is not available, so the comparison asks whether an unknown is equal to another unknown, which is undecidable and therefore NA (An Introduction to R: Missing values). The result of x == NA is a vector of the length of x made only of NA, whatever x holds. Using it as an index returns a vector of NA values, summing it gives NA, and using it in an if is an error. The default lintr linters flag x == NA (equals_na_linter) for this reason, which is why the first block has # nolint comments: it is showing the mistake on purpose.
source("scenario.R")
# x == NA is never TRUE and never FALSE: every element is NA
books$year == NA # nolint: equals_na_linter.
#> [1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
books$year[books$year == NA] # nolint: equals_na_linter.
#> [1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
sum(books$year == NA) # nolint: equals_na_linter.
#> [1] NA
is.na() is the test. It returns a logical vector of the same length (copying the names, dim and dimnames of its argument), so which(is.na(x)) gives the positions, x[is.na(x)] the missing elements, sum(is.na(x)) the number of them and mean(is.na(x)) the proportion, because TRUE counts as 1. The next block finds the one book without a year (row 7, "Garden of Small Fires") and shows the proportion of unknown page counts, 2 of 16. When the result must be a single TRUE or FALSE for an if, remember that a comparison involving an NA is itself NA: if (first_year < 2000L) stops with missing value where TRUE/FALSE needed when the year is unknown, and if (NA) stops with the same message. Decide what an unknown should do and say it: if (is.na(first_year)) handles it first, or isTRUE(first_year < 2000L) folds the unknown into FALSE.
# is.na() is the test
is.na(books$year)
#> [1] FALSE FALSE FALSE FALSE FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE
#> [13] FALSE FALSE FALSE FALSE
which(is.na(books$year))
#> [1] 7
books$title[is.na(books$year)]
#> [1] "Garden of Small Fires"
sum(is.na(books$pages))
#> [1] 2
mean(is.na(books$pages))
#> [1] 0.125
Source: R documentation: NA
# if (NA) is an error, and so is a condition that is NA
tryCatch(if (NA) "yes", error = \(e) conditionMessage(e))
#> [1] "missing value where TRUE/FALSE needed"
first_year <- books$year[[7L]]
tryCatch(
if (first_year < 2000L) "old" else "new",
error = \(e) conditionMessage(e)
)
#> [1] "missing value where TRUE/FALSE needed"
Source: R documentation: Control
# Ask the question that has an answer
if (is.na(first_year)) "year unknown" else "year known"
#> [1] "year unknown"
if (isTRUE(first_year < 2000L)) "old" else "old or unknown"
#> [1] "old or unknown"
Source: R documentation: Logic
There is one place where NA is compared with NA and found equal on purpose, and it is the set of lookup functions. The Language Definition notes that an NA value will match another NA in match(), and NaN matches only NaN (The R Language Definition: NA handling). So NA %in% books$year is TRUE and match(NA, books$year) is 7, and unique(), duplicated() and table(useNA = "ifany") all treat NA as a value that can repeat. The linter flags x %in% NA as well, since anyNA(x) or is.na(x) say it better. A data frame has an is.na() method that returns a logical matrix with the dimensions and dimnames of the frame, so colSums(is.na(books)) is the count of unknowns by column (0 for the titles, 1 for the year, 2 for the pages), and which(is.na(books), arr.ind = TRUE) gives the row and column of every one of them. complete.cases() and anyNA() give whole-row and whole-object answers.
# NA is found by match() and %in%, because they compare NA with NA
NA %in% books$year
#> [1] TRUE
NA %in% books$title
#> [1] FALSE
match(NA, books$year)
#> [1] 7
match(c(1974L, NA), books$year)
#> [1] 1 7
which(books$year %in% NA) # nolint: equals_na_linter.
#> [1] 7
Source: R documentation: match
# A whole table at once: is.na() on a data frame is a logical matrix
colSums(is.na(books))
#> title author isbn year genre pages
#> 0 0 0 1 0 2
colSums(is.na(loans))
#> loan_id isbn member loan_date return_date
#> 0 0 0 1 7
dim(is.na(books))
#> [1] 16 6
which(is.na(books), arr.ind = TRUE)
#> row col
#> [1,] 7 4
#> [2,] 6 6
#> [3,] 15 6
Source: R documentation: colSums
# Rows with at least one NA, and the cells that are NA
rows_with_na <- !complete.cases(books)
which(rows_with_na)
#> [1] 6 7 15
books$title[rows_with_na]
#> [1] "A Map of Salt" "Garden of Small Fires" "Ten Winters"
anyNA(books)
#> [1] TRUE
anyNA(books$title)
#> [1] FALSE
Source: R documentation: complete.cases
Absence is not the same as an empty value. The ISBN of "Rivers of Glass" is the empty string "", which is a perfectly good string of length zero: is.na("") is FALSE, sum(is.na(books$isbn)) is 0, and the missing ISBN is found by books$isbn == "" or !nzchar(books$isbn). Whether the empty string should count as missing is a decision about the data; Filling and recoding shows how to turn it into a real NA when reading or afterwards. The last block sets a value to missing: x[2L] <- NA assigns, and the replacement function is.na<- takes an index and sets those elements to the NA of the vector’s type, which for a character vector is NA_character_; the help page adds that it behaves differently for factors (R documentation: NA).
# An empty string is not NA
books$isbn[[9L]]
#> [1] ""
is.na(books$isbn[[9L]])
#> [1] FALSE
sum(books$isbn == "")
#> [1] 1
sum(!nzchar(books$isbn))
#> [1] 1
sum(is.na(books$isbn))
#> [1] 0
Source: R documentation: nchar
# Marking a value as missing: assign NA, or is.na<-
x <- c(10, 20, 30)
x[2L] <- NA
x
#> [1] 10 NA 30
is.na(x) <- 3L
x
#> [1] 10 NA NA
y <- c("a", "b", "c")
is.na(y) <- y == "b"
y
#> [1] "a" NA "c"
Source: R documentation: NA
The questions that are asked most often have exactly one correct spelling:
| Question | Wrong | Right |
|---|---|---|
Which elements are missing? |
|
|
Is any element missing? |
|
|
How many are missing? |
|
|
Is this single value missing, with a safe |
|
|
Is this single condition |
|
|
Which rows have no missing value? |
|
|
Is the value |
|
|
Is the object absent? |
|
|
Source: R documentation: NA
Where missing values come from
A missing value enters a computation from a short list of places, and knowing the list helps to find the one that put it there. Data entry is the obvious source: an empty field in a CSV file becomes NA in numeric, logical and date columns (the year of "Garden of Small Fires", the page counts of the two books with a blank, the unreturned loans), and a text that cannot be parsed becomes NA with a warning. The rest are R’s own rules, and the script below shows them in turn:
-
An index that selects an element that does not exist gives
NAfor an atomic vector (x[5L],x["z"]) andNULLfor a list (lst[["z"]]), while the double-bracket forms on an atomic vector stop withsubscript out of bounds. AnNAin the index itself gives anNAelement (x[c(1L, NA)]), with the names of an atomic vector turning into<NA>. -
The type of the
NAused as an index matters. A bareNAis logical, soy[NA]is a logical index, which is recycled to the length ofyand selects every element as unknown: the result is fourNAvalues.y[NA_integer_]is an integer index and selects one element, the result being oneNA. The Language Definition points at this case in its NA handling section (The R Language Definition: Indexing by vectors). -
A logical index that contains
NAgives anNAfor eachNAin the index. In a data frame this means a whole row ofNAvalues with the row nameNA: with the two unknown page counts,books[books$pages > 300L, ]has 8 rows although only 6 books have more than 300 pages, and the 4th and 5th rows of the printed excerpt show one of the phantom rows.which()drops theNAvalues,subset()does so too, and!is.na(big) & bigsays it explicitly. This is the most common way for missing values to damage a data frame silently. -
Assignment with an
NAindex is restricted:z[c(1L, NA)] <- c(0L, 1L)is an error,NAs are not allowed in subscripted assignments, because R cannot know which of the two values goes where; it is allowed with a single replacement value, and theNApositions are skipped. -
Making a vector longer with
length<-, and joins that keep unmatched rows (merge(all = TRUE),merge(all.x = TRUE)), pad withNA. -
Numbers: integer overflow gives
NAwith the warningNAs produced by integer overflow, a failed parse withas.integer("seven")givesNAand the warningNAs introduced by coercion, and a conversion outside the integer range givesNAandNAs introduced by coercion to integer range(see Numbers and Arithmetic).
source("scenario.R")
# Indexing: an out-of-range position, a name that is absent, an NA index
x <- c(a = 10, b = 20, c = 30)
x[5L]
#> <NA>
#> NA
x["z"]
#> <NA>
#> NA
x[NA_integer_]
#> <NA>
#> NA
x[NA]
#> <NA> <NA> <NA>
#> NA NA NA
x[c(1L, NA)]
#> a <NA>
#> 10 NA
x[c(TRUE, NA, FALSE)]
#> a <NA>
#> 10 NA
tryCatch(x[["z"]], error = \(e) conditionMessage(e))
#> [1] "subscript out of bounds"
tryCatch(x[[5L]], error = \(e) conditionMessage(e))
#> [1] "subscript out of bounds"
tryCatch(x[[NA_integer_]], error = \(e) conditionMessage(e))
#> [1] "subscript out of bounds"
lst <- list(a = 1, b = 2)
lst[5L]
#> $<NA>
#> NULL
#>
lst[["z"]]
#> NULL
lst[[NA_character_]]
#> NULL
lst[NA]
#> $<NA>
#> NULL
#>
#> $<NA>
#> NULL
#>
# A logical NA index recycles; a typed NA index selects one element
y <- 1:4
y[NA]
#> [1] NA NA NA NA
y[NA_integer_]
#> [1] NA
typeof(NA)
#> [1] "logical"
# Logical subscripts that contain NA keep an NA row in a data frame
big <- books$pages > 300L
sum(big, na.rm = TRUE)
#> [1] 6
nrow(books[big, ])
#> [1] 8
books[big, c("title", "pages")][4:5, ]
#> title pages
#> NA <NA> NA
#> 9 Rivers of Glass 352
nrow(books[which(big), ])
#> [1] 6
nrow(subset(books, pages > 300L))
#> [1] 6
nrow(books[!is.na(big) & big, ])
#> [1] 6
Source: R documentation: Extract
# Assigning with an NA subscript: allowed only with a single value
z <- 1:5
tryCatch(
z[c(1L, NA)] <- c(0L, 1L),
error = \(e) conditionMessage(e)
)
#> [1] "NAs are not allowed in subscripted assignments"
z[c(1L, NA)] <- 0L
z
#> [1] 0 2 3 4 5
z[c(TRUE, NA, FALSE, FALSE, FALSE)] <- 9L
z
#> [1] 9 2 3 4 5
z[NA] <- 7L
z
#> [1] 9 2 3 4 5
Source: R documentation: Extract
# length<- and merges pad with NA
v <- 1:3
length(v) <- 5L
v
#> [1] 1 2 3 NA NA
left <- data.frame(isbn = c("a", "b", "c"), n = 1:3)
right <- data.frame(isbn = c("a", "c", "d"), m = 4:6)
merge(left, right, all = TRUE)
#> isbn n m
#> 1 a 1 4
#> 2 b 2 NA
#> 3 c 3 5
#> 4 d NA 6
merge(left, right, all.x = TRUE)
#> isbn n m
#> 1 a 1 4
#> 2 b 2 NA
#> 3 c 3 5
merge(left, right)
#> isbn n m
#> 1 a 1 4
#> 2 c 3 5
Source: R documentation: merge
# Numbers: overflow, failed parse, 0/0
tryCatch(.Machine$integer.max + 1L, warning = \(w) conditionMessage(w))
#> [1] "NAs produced by integer overflow"
tryCatch(as.integer("seven"), warning = \(w) conditionMessage(w))
#> [1] "NAs introduced by coercion"
tryCatch(as.integer(3e9), warning = \(w) conditionMessage(w))
#> [1] "NAs introduced by coercion to integer range"
0 / 0
#> [1] NaN
is.na(0 / 0)
#> [1] TRUE
Source: R documentation: integer
The subsetting rules are the subject of Subsetting and Indexing; the rows of a data frame are on Data Frames.
How NA propagates
The rule of the manuals is that numeric and logical calculations with NA generally return NA, because the result would depend on a value nobody knows. The exceptions are the cases where the answer would be the same whatever the unknown was, and R does implement those: FALSE & NA is FALSE and TRUE | NA is TRUE (The R Language Definition: NA handling). It is Kleene’s three-valued logic, with NA as the third value "unknown". The first block shows arithmetic and comparison (every one is NA), and the second the logic operators, which are worth learning as a table:
x |
y |
x & y |
x | y |
!x |
|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Source: R documentation: Logic
The same logic extends to the summaries any() and all(): any(c(NA, TRUE)) is TRUE because one TRUE is enough whatever the unknown is, while any(c(NA, FALSE)) is NA; all(c(NA, FALSE)) is FALSE and all(c(NA, TRUE)) is NA. The scalar operators && and || evaluate left to right and stop as soon as the answer is known, so FALSE && NA is FALSE and TRUE || NA is TRUE, but NA && TRUE is NA; a guard of the form isTRUE(x) && ... or !is.na(x) && ... keeps the unknown out of the condition (see Operators and Precedence for the length-one rule and its errors).
source("scenario.R")
# Arithmetic and comparison: an unknown input gives an unknown output
NA + 1
#> [1] NA
NA * 0
#> [1] NA
NA - NA
#> [1] NA
NA > 1
#> [1] NA
NA == NA # nolint: equals_na_linter.
#> [1] NA
-NA
#> [1] NA
c(1, NA, 3) * 2
#> [1] 2 NA 6
c(1, NA, 3) > 2
#> [1] FALSE NA TRUE
Source: R documentation: Arithmetic
# Three-valued logic: NA only when the answer depends on the unknown
NA & TRUE
#> [1] NA
NA & FALSE
#> [1] FALSE
NA | TRUE
#> [1] TRUE
NA | FALSE
#> [1] NA
!NA
#> [1] NA
xor(NA, TRUE)
#> [1] NA
c(TRUE, NA, FALSE) & c(NA, NA, NA)
#> [1] NA NA FALSE
c(TRUE, NA, FALSE) | c(NA, NA, NA)
#> [1] TRUE NA NA
any(c(NA, TRUE))
#> [1] TRUE
any(c(NA, FALSE))
#> [1] NA
all(c(NA, FALSE))
#> [1] FALSE
all(c(NA, TRUE))
#> [1] NA
Source: R documentation: Logic
# && and || stop early, so the unknown can be skipped
FALSE && NA
#> [1] FALSE
TRUE || NA
#> [1] TRUE
NA && FALSE
#> [1] FALSE
NA || TRUE
#> [1] TRUE
NA && TRUE
#> [1] NA
isTRUE(NA && TRUE)
#> [1] FALSE
Source: R documentation: Logic
Summaries and na.rm
A summary of a vector that contains an NA is NA, because the sum or the maximum of a set that includes an unknown is unknown. Most summary functions take the argument na.rm, which, when TRUE, removes the missing values (the NA and NaN values) before the computation. sum(books$pages) is NA and sum(books$pages, na.rm = TRUE) is 3755, the sum of the 14 known page counts; the mean is 268.2143 (rounded by the printing), the median 268, the range 88 to 432. na.rm = TRUE answers the question "of the values I know", and it changes the size of the sample behind the statistic, so for a report you also want the count of what was dropped (sum(is.na(x))).
The na.rm argument must be a single TRUE or FALSE. Since R 4.4.0, a call that names it twice, for instance sum(1, na.rm = TRUE, na.rm = FALSE), is an error: formal argument "na.rm" matched by multiple actual arguments; before, the summary group silently produced either of two answers (R NEWS: Changes in R 4.4.0). An NA as the value of na.rm is also an error for sum(): NA in coercion to boolean.
# Summaries: one NA makes the result NA, na.rm = TRUE skips it
sum(books$pages)
#> [1] NA
sum(books$pages, na.rm = TRUE)
#> [1] 3755
mean(books$pages)
#> [1] NA
mean(books$pages, na.rm = TRUE)
#> [1] 268.2143
max(books$year)
#> [1] NA
max(books$year, na.rm = TRUE)
#> [1] 2023
range(books$pages, na.rm = TRUE)
#> [1] 88 432
median(books$pages)
#> [1] NA
median(books$pages, na.rm = TRUE)
#> [1] 268
prod(c(2, NA, 3), na.rm = TRUE)
#> [1] 6
sd(books$pages, na.rm = TRUE)
#> [1] 107.9866
var(c(1, 2, NA, 4))
#> [1] NA
var(c(1, 2, NA, 4), na.rm = TRUE)
#> [1] 2.333333
Source: R documentation: sum
# na.rm must be a single TRUE or FALSE
tryCatch(
sum(1, na.rm = TRUE, na.rm = FALSE),
error = \(e) conditionMessage(e)
)
#> [1] "formal argument \"na.rm\" matched by multiple actual arguments"
tryCatch(sum(1:3, na.rm = NA), error = \(e) conditionMessage(e))
#> [1] "NA in coercion to boolean"
Source: R documentation: sum
The cumulative functions (cumsum(), cumprod(), cummax(), cummin()) have no na.rm: once an NA is met, every later element is NA. That is the right answer for a running total, and the way to continue past it is to replace the NA values first (see Filling and recoding). Counting is just arithmetic on logicals: sum(books$pages > 300L, na.rm = TRUE) is 6 and sum(books$pages > 300L) is NA, the latter because the two unknown page counts could be either side of the threshold. which() ignores the NA values and gives the positions of the TRUE ones, and which.max() gives the position of the largest known value, while a logical index keeps an NA for each NA, as Where missing values come from showed for rows.
# Cumulative functions have no na.rm: the NA spreads to the end
cumsum(c(1, 2, NA, 4))
#> [1] 1 3 NA NA
cummax(c(1, 3, NA, 2))
#> [1] 1 3 NA NA
cumprod(c(2, NA, 3))
#> [1] 2 NA NA
Source: R documentation: cumsum
# Counting NA by hand
n_total <- length(books$pages)
n_known <- sum(!is.na(books$pages))
n_total - n_known
#> [1] 2
Source: R documentation: Extremes
# TRUE counts as 1, so counts and proportions of NA are sums and means
sum(is.na(books$pages))
#> [1] 2
mean(is.na(books$pages))
#> [1] 0.125
sum(books$pages > 300L, na.rm = TRUE)
#> [1] 6
sum(books$pages > 300L)
#> [1] NA
Source: R documentation: which
# which() and which.max() ignore NA; logical subscripts do not
which(books$pages > 300L)
#> [1] 1 2 5 9 12 14
which.max(books$pages)
#> [1] 2
books$title[books$pages > 300L]
#> [1] "The Dispossessed" "Pride and Prejudice" "Midnight Ledger"
#> [4] NA "Rivers of Glass" "Notes on Gravity"
#> [7] "The Slow Comet" NA
books$title[which(books$pages > 300L)]
#> [1] "The Dispossessed" "Pride and Prejudice" "Midnight Ledger"
#> [4] "Rivers of Glass" "Notes on Gravity" "The Slow Comet"
Source: R documentation: which
The functions of the table below are the ones with a missing-value argument, and the argument differs by family. The statistics from stats that work on two or more variables (cor(), cov(), var() with several columns) take a use argument instead of na.rm, with the choices "everything" (the default: NA propagates), "all.obs" (an NA is an error), "complete.obs" (drop every row with an NA in any variable), "na.or.complete" and "pairwise.complete.obs" (drop per pair).
| Function | Argument | Default behaviour with an NA |
|---|---|---|
|
|
result is |
|
|
result is |
|
|
error: |
|
|
|
|
|
|
|
none |
|
|
none (they ignore |
position of the extreme of the known values |
|
|
removes the |
|
|
puts the |
|
|
ranks them last; |
|
|
leaves |
|
|
|
Source: R documentation: sum
Strings, sorting and tables
Strings have their own missing value, NA_character_, and the string functions disagree on what to do with it. Most keep it (toupper(), substr(), trimws(), nchar() return NA), but paste() and paste0() convert it to the text "NA" (paste("pages:", c(214L, NA)) is "pages: NA"), sprintf() does the same, and format() pads it to the common width. So a result that goes through paste() can no longer be tested with is.na(); test the inputs first. nchar() has a keepNA argument: the default returns NA for the character types (type = "chars" and "bytes") and 2 for type = "width", and keepNA = FALSE always returns 2, the number of characters in the printed NA. nzchar(NA) is TRUE by default, which means that nzchar() is not a missing-value test either: use !is.na(x) && nzchar(x) for "a non-empty string" (nzchar(x, keepNA = TRUE) returns NA for an NA).
Failing coercions produce NA and a warning, and the shape of the warning is the same in all of them: NAs introduced by coercion. That is a signal, not a nuisance, and the way to deal with it is to look at the strings that failed (x[is.na(as.numeric(x)) & !is.na(x)], with the warning suppressed) rather than to suppress the warning on the first line.
source("scenario.R")
# Strings: NA_character_ is kept by most functions, but not by paste()
paste("pages:", c(214L, NA))
#> [1] "pages: 214" "pages: NA"
paste0("year ", books$year[6:8])
#> [1] "year 2008" "year NA" "year 1999"
paste(c("a", NA), collapse = "-")
#> [1] "a-NA"
nchar(c("abc", NA))
#> [1] 3 NA
nchar(NA)
#> [1] NA
nchar(NA_character_)
#> [1] NA
nchar(NA, keepNA = FALSE)
#> [1] 2
nchar(NA, type = "width")
#> [1] 2
nzchar(c("abc", "", NA))
#> [1] TRUE FALSE TRUE
nzchar(c("abc", "", NA), keepNA = TRUE)
#> [1] TRUE FALSE NA
toupper(c("abc", NA))
#> [1] "ABC" NA
substr(c("abc", NA), 1L, 2L)
#> [1] "ab" NA
trimws(c(" abc ", NA))
#> [1] "abc" NA
is.na(paste(NA))
#> [1] FALSE
paste(NA_character_)
#> [1] "NA"
sprintf("%d pages", c(214L, NA))
#> [1] "214 pages" "NA pages"
sprintf("%s", NA)
#> [1] "NA"
format(c(1.5, NA, 3))
#> [1] "1.5" " NA" "3.0"
format(NA)
#> [1] "NA"
format(NA_character_)
#> [1] "NA"
as.character(c(1.5, NA, 3))
#> [1] "1.5" NA "3"
Source: R documentation: paste
# Coercion that fails gives NA with a warning
tryCatch(as.numeric("12 pages"), warning = \(w) conditionMessage(w))
#> [1] "NAs introduced by coercion"
tryCatch(as.integer("x"), warning = \(w) conditionMessage(w))
#> [1] "NAs introduced by coercion"
suppressWarnings(as.numeric(c("12", "x", "3.5")))
#> [1] 12.0 NA 3.5
as.numeric(c("12", NA))
#> [1] 12 NA
as.logical(c("TRUE", "yes", "T", NA))
#> [1] TRUE NA TRUE NA
tryCatch(.Machine$integer.max + 1L, warning = \(w) conditionMessage(w))
#> [1] "NAs produced by integer overflow"
Source: R documentation: as.numeric
Sorting must put the missing values somewhere. sort() removes them by default (na.last = NA) and order() puts them last (na.last = TRUE), so the two do not agree on a vector with an NA: sort(x) has 3 elements and x[order(x)] has 4. Both take na.last = TRUE, FALSE or NA (for order(), NA removes the indices; for sort() it removes the values), and rank() has the further choice na.last = "keep", which returns an NA rank for an NA value instead of ranking it. The consequence for a data frame: books[order(books$pages), ] puts the two books with an unknown page count at the end, while order(books$pages, na.last = NA) leaves them out; the head of books[order(books$pages, decreasing = TRUE), ] shows the largest known books, still with the unknown ones last in the full result.
table() leaves NA out unless asked: useNA = "ifany" adds an <NA> count when there is at least one, "always" adds it even if it is zero, and "no" (the default) leaves it out. The default hides the missing values in exactly the place where a reader wants to see them, so a summary table of a column with unknowns is usually better with useNA = "ifany". A factor does not have NA among its levels (levels(factor(c("a", NA, "b"))) is "a" "b"), the missing value being stored as the integer code NA; addNA() makes it a level when you need that. unique() and duplicated() treat all NA values as the same value, and tabulate() ignores them. See Factors for the details of the factor case.
# Sorting: NA is dropped by sort(), but put last by order()
x <- c(3, NA, 1, 2)
sort(x)
#> [1] 1 2 3
sort(x, na.last = TRUE)
#> [1] 1 2 3 NA
sort(x, na.last = FALSE)
#> [1] NA 1 2 3
sort(x, decreasing = TRUE, na.last = TRUE)
#> [1] 3 2 1 NA
order(x)
#> [1] 3 4 1 2
order(x, na.last = FALSE)
#> [1] 2 3 4 1
order(x, na.last = NA)
#> [1] 3 4 1
x[order(x)]
#> [1] 1 2 3 NA
rank(x)
#> [1] 3 4 1 2
rank(x, na.last = "keep")
#> [1] 3 NA 1 2
rev(x)
#> [1] 2 1 NA 3
Source: R documentation: sort
# The books ordered by pages, with the unknown ones last or dropped
books$title[order(books$pages)]
#> [1] "Paper Moons" "Garden of Small Fires" "Orchard Logic"
#> [4] "Loud Roofs" " the quiet streets " "Salt and Static"
#> [7] "THE LONG TABLE" "The Lamp at Noon" "Midnight Ledger"
#> [10] "The Slow Comet" "Rivers of Glass" "The Dispossessed"
#> [13] "Notes on Gravity" "Pride and Prejudice" "A Map of Salt"
#> [16] "Ten Winters"
books$title[order(books$pages, na.last = NA)]
#> [1] "Paper Moons" "Garden of Small Fires" "Orchard Logic"
#> [4] "Loud Roofs" " the quiet streets " "Salt and Static"
#> [7] "THE LONG TABLE" "The Lamp at Noon" "Midnight Ledger"
#> [10] "The Slow Comet" "Rivers of Glass" "The Dispossessed"
#> [13] "Notes on Gravity" "Pride and Prejudice"
head(books[order(books$pages, decreasing = TRUE), c("title", "pages")], 4L)
#> title pages
#> 2 Pride and Prejudice 432
#> 12 Notes on Gravity 410
#> 1 The Dispossessed 387
#> 9 Rivers of Glass 352
Source: R documentation: order
# unique(), duplicated() and table() treat NA as a value that can repeat
unique(c(1, NA, 2, NA))
#> [1] 1 NA 2
duplicated(c(1, NA, 2, NA))
#> [1] FALSE FALSE FALSE TRUE
table(c(1, NA, 2, NA))
#>
#> 1 2
#> 1 1
table(c(1, NA, 2, NA), useNA = "ifany")
#>
#> 1 2 <NA>
#> 1 1 2
table(c(1, 2, 2), useNA = "ifany")
#>
#> 1 2
#> 1 2
table(c(1, 2, 2), useNA = "always")
#>
#> 1 2 <NA>
#> 1 2 0
table(books$pages > 300L)
#>
#> FALSE TRUE
#> 8 6
table(books$pages > 300L, useNA = "ifany")
#>
#> FALSE TRUE <NA>
#> 8 6 2
table(books$year %/% 10L * 10L, useNA = "ifany")
#>
#> 1810 1970 1990 2000 2010 2020 <NA>
#> 1 1 1 2 6 4 1
tabulate(c(1L, NA, 2L, 2L))
#> [1] 1 2
Source: R documentation: table
# cut() and factor() leave NA out of the levels
f <- factor(c("a", NA, "b", "a"))
levels(f)
#> [1] "a" "b"
f
#> [1] a <NA> b a
#> Levels: a b
table(f)
#> f
#> a b
#> 2 1
table(f, useNA = "ifany")
#> f
#> a b <NA>
#> 2 1 1
nlevels(addNA(f))
#> [1] 3
levels(addNA(f))
#> [1] "a" "b" NA
Source: R documentation: factor
Removing missing values: na.omit(), complete.cases() and na.action
na.rm is an argument of one function. When the missing values have to be removed from the data itself, R has two families of tools. na.omit() returns the object without the missing observations, and records what it removed in an attribute called na.action, of class omit, that holds the positions (or, for a data frame, the row names) of the dropped elements. The attribute is the reason the printed vector from na.omit() shows an attr(,"na.action") section: the object is still a numeric vector that every function can use, and as.vector() or c() strips the attribute when it is in the way. complete.cases() returns the logical vector that says which rows are complete, for any mix of vectors, matrices and data frames of the same length, and is the better choice when you want to keep control of the subsetting: books[complete.cases(books), ] is what na.omit(books) returns, without the attribute. If the arguments differ in length it stops with not all arguments have the same length.
source("scenario.R")
# na.omit() drops the NA and records what it dropped
pages <- na.omit(books$pages)
pages
#> [1] 387 432 214 198 301 96 280 352 256 88 410 175 322 244
#> attr(,"na.action")
#> [1] 6 15
#> attr(,"class")
#> [1] "omit"
length(pages)
#> [1] 14
attr(pages, "na.action")
#> [1] 6 15
#> attr(,"class")
#> [1] "omit"
attributes(pages)
#> $na.action
#> [1] 6 15
#> attr(,"class")
#> [1] "omit"
#>
class(attr(pages, "na.action"))
#> [1] "omit"
as.vector(pages)
#> [1] 387 432 214 198 301 96 280 352 256 88 410 175 322 244
Source: R documentation: na.fail
For one vector, mean(x, na.rm = TRUE) and mean(na.omit(x)) give the same number. For several variables they are different operations, and the difference is the reason to know both. na.rm = TRUE is applied variable by variable, so each statistic uses all the values that variable has; na.omit() on a data frame performs listwise deletion, which drops a row when any of its variables is missing, so every statistic is computed on the same rows. On the shelf, colMeans(books[c("year", "pages")], na.rm = TRUE) uses 15 years and 14 page counts, while colMeans(na.omit(...)) uses the 13 rows that have both, and the means differ (1998.73 and 268.21 against 1996.23 and 281.46). Neither is wrong; they answer different questions, and a correlation or a regression needs the second, which is what the use = "complete.obs" argument of cor() and the default of lm() do.
# mean(x, na.rm = TRUE) and mean(na.omit(x)) agree for a single vector
mean(books$pages, na.rm = TRUE)
#> [1] 268.2143
mean(na.omit(books$pages))
#> [1] 268.2143
identical(
mean(books$pages, na.rm = TRUE),
mean(na.omit(books$pages))
)
#> [1] TRUE
Source: R documentation: mean
# na.omit() on a data frame drops whole rows, which is not na.rm = TRUE
both <- books[c("year", "pages")]
nrow(both)
#> [1] 16
nrow(na.omit(both))
#> [1] 13
colMeans(both, na.rm = TRUE)
#> year pages
#> 1998.7333 268.2143
colMeans(na.omit(both))
#> year pages
#> 1996.2308 281.4615
rownames(na.omit(both))[1:3]
#> [1] "1" "2" "3"
attr(na.omit(both), "na.action")
#> 6 7 15
#> 6 7 15
#> attr(,"class")
#> [1] "omit"
Source: R documentation: na.fail
# complete.cases() gives the logical vector, for any set of columns
complete.cases(books)
#> [1] TRUE TRUE TRUE TRUE TRUE FALSE FALSE TRUE TRUE TRUE TRUE TRUE
#> [13] TRUE TRUE FALSE TRUE
sum(complete.cases(books))
#> [1] 13
books[!complete.cases(books), c("title", "year", "pages")]
#> title year pages
#> 6 A Map of Salt 2008 NA
#> 7 Garden of Small Fires NA 96
#> 15 Ten Winters 2022 NA
complete.cases(books$year, books$pages)
#> [1] TRUE TRUE TRUE TRUE TRUE FALSE FALSE TRUE TRUE TRUE TRUE TRUE
#> [13] TRUE TRUE FALSE TRUE
complete.cases(books[c("title", "year")])
#> [1] TRUE TRUE TRUE TRUE TRUE TRUE FALSE TRUE TRUE TRUE TRUE TRUE
#> [13] TRUE TRUE TRUE TRUE
books[complete.cases(books$pages), "title"][1:3]
#> [1] "The Dispossessed" "Pride and Prejudice" " the quiet streets "
complete.cases(c(1, NA), c("a", "b"))
#> [1] TRUE FALSE
complete.cases(data.frame(a = 1:3, b = c(NA, 1, NA)))
#> [1] FALSE TRUE FALSE
tryCatch(
complete.cases(1:3, 1:2),
error = \(e) conditionMessage(e)
)
#> [1] "not all arguments have the same length"
Source: R documentation: complete.cases
The na.action family
na.omit(), na.exclude(), na.fail() and na.pass() are four ways to decide what an analysis does with an NA, and they share one help page (R documentation: na.fail). They are called by the modelling functions (lm(), glm(), model.frame() and the other formula interfaces) through their na.action argument, whose default is the na.action option, which is "na.omit".
| Function | Behaviour | Typical use |
|---|---|---|
|
removes the observations with an |
the default; the model is fitted on the complete rows |
|
the same as |
when the output must line up with the input rows |
|
stops with the error |
a strict analysis that must not drop data silently |
|
returns the object unchanged |
when the called function deals with |
Source: R documentation: na.fail
The effect of the choice shows in a regression of the page count on the year. With the default the 13 complete rows are used (nobs() is 13 and there are 13 residuals); with na.action = na.exclude the same 13 rows are used but residuals() has 16 elements, with NA at rows 6, 7 and 15, so it can be added to the data frame as a column; with na.action = na.fail the call stops. The accessor na.action() returns the attribute of a fitted model, here the rows 6, 7 and 15 that were left out. The default can be changed for the session with options(na.action = "na.exclude"); the script saves the old value that options() returns and restores it, which is the pattern for any temporary option change (see Sessions, Workspaces and Startup). Modelling is the subject of Statistical Models.
# The other na.action functions
na.omit(c(1, NA, 3))
#> [1] 1 3
#> attr(,"na.action")
#> [1] 2
#> attr(,"class")
#> [1] "omit"
tryCatch(na.fail(c(1, NA, 3)), error = \(e) conditionMessage(e))
#> [1] "missing values in object"
na.fail(c(1, 2, 3))
#> [1] 1 2 3
na.pass(c(1, NA, 3))
#> [1] 1 NA 3
na.exclude(c(1, NA, 3))
#> [1] 1 3
#> attr(,"na.action")
#> [1] 2
#> attr(,"class")
#> [1] "exclude"
getOption("na.action")
#> [1] "na.omit"
Source: R documentation: na.fail
# Where na.action is used: modelling functions
fit_omit <- lm(pages ~ year, data = books)
fit_excl <- lm(pages ~ year, data = books, na.action = na.exclude)
nobs(fit_omit)
#> [1] 13
length(residuals(fit_omit))
#> [1] 13
length(residuals(fit_excl))
#> [1] 16
na.action(fit_omit)
#> 6 7 15
#> 6 7 15
#> attr(,"class")
#> [1] "omit"
names(residuals(fit_excl))[c(6L, 7L, 15L)]
#> [1] "6" "7" "15"
residuals(fit_excl)[c(6L, 7L, 15L)]
#> 6 7 15
#> NA NA NA
tryCatch(
lm(pages ~ year, data = books, na.action = na.fail),
error = \(e) conditionMessage(e)
)
#> [1] "missing values in object"
Source: R documentation: lm
# Changing the default for the session
old <- options(na.action = "na.exclude")
getOption("na.action")
#> [1] "na.exclude"
nobs(lm(pages ~ year, data = books))
#> [1] 13
length(residuals(lm(pages ~ year, data = books)))
#> [1] 16
options(old)
getOption("na.action")
#> [1] "na.omit"
Source: R documentation: options
Some statistics refuse to guess. quantile() has na.rm = FALSE and stops with an error rather than returning an NA (missing values and NaN's not allowed if 'na.rm' is FALSE), because quantiles need to sort the data. cor() returns NA for cor(books$year, books$pages) and takes the use argument shown in the table above; use = "complete.obs" and "pairwise.complete.obs" agree here because there are only two variables. summary() of a numeric vector adds an NAs column with the count, which is the cheapest way to see that missing values were present when a summary looks odd; descriptive statistics are on Descriptive Statistics and Tests.
# Statistics that refuse NA unless told
tryCatch(
quantile(books$pages),
error = \(e) conditionMessage(e)
)
#> [1] "missing values and NaN's not allowed if 'na.rm' is FALSE"
quantile(books$pages, na.rm = TRUE)
#> 0% 25% 50% 75% 100%
#> 88.0 202.0 268.0 344.5 432.0
cor(books$year, books$pages)
#> [1] NA
cor(books$year, books$pages, use = "complete.obs")
#> [1] -0.5069084
cor(books$year, books$pages, use = "pairwise.complete.obs")
#> [1] -0.5069084
summary(books$pages)
#> Min. 1st Qu. Median Mean 3rd Qu. Max. NAs
#> 88.0 202.0 268.0 268.2 344.5 432.0 2
Source: R documentation: quantile
Filling and recoding
Removing is not the only option and it is rarely a neutral one: dropping rows changes what the data represent. When a missing value has to be given a value, three tools do the job in base R, and all of them take a logical vector from is.na() to say where. ifelse(is.na(x), fill, x) is vectorised and returns a new vector; replace(x, is.na(x), fill) is the same without the wrapper (replace() is x[list] <- values written as a function); and the assignment x[is.na(x)] <- fill modifies in place. The fill can be a constant or a statistic of the known values (mean(x, na.rm = TRUE), median(x, na.rm = TRUE)). Which one is correct is a statistical question that R cannot answer: filling a page count with zero tells the next function that the book has no pages, and filling with the mean shrinks the variance. tidyr::replace_na() and dplyr::coalesce() do the same job for data frames and are named in prose only (see tidyr, purrr and stringr and The tidyverse and dplyr).
ifelse() returns NA where its test is NA, so a classification such as long or short based on a page count leaves the two unknown books unclassified, and the second form of the example says explicitly what to do with them. The type of the fill matters: replace(books$pages, is.na(books$pages), 0) turns an integer column into a double one, because 0 is a double, and filling the year with "unknown" turns the column into a character vector, and every number in it into a string, which is why the fill value for an integer column is written 0L and why a placeholder text does not belong inside a numeric column.
source("scenario.R")
# Filling with a constant: ifelse() and replace()
pages <- books$pages
ifelse(is.na(pages), 0L, pages)
#> [1] 387 432 214 198 301 0 96 280 352 256 88 410 175 322 0 244
replace(pages, is.na(pages), 0L)
#> [1] 387 432 214 198 301 0 96 280 352 256 88 410 175 322 0 244
pages[is.na(pages)] <- 0L
pages
#> [1] 387 432 214 198 301 0 96 280 352 256 88 410 175 322 0 244
sum(books$pages, na.rm = TRUE) == sum(pages)
#> [1] TRUE
Source: R documentation: ifelse
# Filling with a statistic of the known values
pages <- books$pages
pages[is.na(pages)] <- round(mean(pages, na.rm = TRUE))
pages
#> [1] 387 432 214 198 301 268 96 280 352 256 88 410 175 322 268 244
Source: R documentation: mean
# ifelse() keeps NA in the test as NA
ifelse(books$pages > 300L, "long", "short")
#> [1] "long" "long" "short" "short" "long" NA "short" "short" "long"
#> [10] "short" "short" "long" "short" "long" NA "short"
ifelse(is.na(books$pages), "unknown",
ifelse(books$pages > 300L, "long", "short")
)
#> [1] "long" "long" "short" "short" "long" "unknown" "short"
#> [8] "short" "long" "short" "short" "long" "short" "long"
#> [15] "unknown" "short"
Source: R documentation: ifelse
# The type of a fill value matters
typeof(replace(books$pages, is.na(books$pages), 0))
#> [1] "double"
typeof(replace(books$pages, is.na(books$pages), 0L))
#> [1] "integer"
typeof(replace(books$year, is.na(books$year), "unknown"))
#> [1] "character"
replace(books$year, is.na(books$year), "unknown")[5:8]
#> [1] "2016" "2008" "unknown" "1999"
Source: R documentation: replace
The opposite recoding is turning a placeholder into a real NA. The shelf has an empty ISBN "" for "Rivers of Glass"; replace(books$isbn, books$isbn == "", NA) makes it missing (the logical NA is coerced to NA_character_ because the vector is character), and is.na(x) <- !nzchar(x) does the same through the replacement function. The better moment is on reading: read.csv() and read.table() take na.strings, a character vector of the texts that mean missing, and na.strings = c("NA", "") reads the empty ISBN as NA. A code such as -99 for "not recorded", which old data files use, is converted in the same way by na.strings = "-99". Reading is covered by Reading and Writing Text Data.
# Turning a placeholder into NA: the empty ISBN of the shelf
books$isbn[9L]
#> [1] ""
isbn <- replace(books$isbn, books$isbn == "", NA)
isbn[8:10]
#> [1] "978-0-00-600062-4" NA "978-0-00-800085-1"
typeof(isbn)
#> [1] "character"
sum(is.na(isbn))
#> [1] 1
is.na(isbn) <- !nzchar(books$isbn)
isbn[9L]
#> [1] NA
Source: R documentation: read.table
# ... or do it while reading the file
books2 <- read.csv("books.csv", na.strings = c("NA", ""))
colSums(is.na(books2))
#> title author isbn year genre pages
#> 0 0 1 1 0 2
sum(books2$isbn == "", na.rm = TRUE)
#> [1] 0
csv_text <- "id,score\n1,10\n2,-99\n3,7"
read.csv(text = csv_text)$score
#> [1] 10 -99 7
read.csv(text = csv_text, na.strings = "-99")$score
#> [1] 10 NA 7
Source: R documentation: read.table
Dates carry their missing values like any other vector. The impossible loan date 2025-02-30 in loans.csv becomes NA when parsed with an explicit format (the scenario does that with as.Date(format = "%Y-%m-%d")), and without a format as.Date("2025-02-30") is an error, character string is not in a standard unambiguous format, because it tries the standard formats on the first non-missing element and none fits. as.Date(NA) is a missing Date, and a Date vector combined with NA stays a Date. The seven loans still out have an NA return date; subtracting dates keeps the NA (a difftime with 8 missing values, the seven unreturned loans and the one with the impossible loan date), and the overdue count, which compares the lengths with loan_days, must use na.rm = TRUE or which() as in How NA propagates: the result is the 25 returned loans that exceeded 21 days, while for the unreturned ones the days out are measured against shelf_today. Dates are on Dates and Times.
# Missing dates: the shelf's loans
sum(is.na(loans$loan_date))
#> [1] 1
which(is.na(loans$loan_date))
#> [1] 20
sum(is.na(loans$return_date))
#> [1] 7
still_out <- is.na(loans$return_date)
head(loans$loan_id[still_out], 3L)
#> [1] "L054" "L055" "L056"
class(loans$return_date)
#> [1] "Date"
loans$return_date[1:2]
#> [1] "2025-02-04" "2025-01-26"
loans$return_date[still_out][1:2]
#> [1] NA NA
as.Date("2025-02-30", format = "%Y-%m-%d")
#> [1] NA
is.na(as.Date("2025-02-30", format = "%Y-%m-%d"))
#> [1] TRUE
tryCatch(
as.Date("2025-02-30"),
error = \(e) conditionMessage(e)
)
#> [1] "character string is not in a standard unambiguous format"
as.Date(NA)
#> [1] NA
class(as.Date(NA))
#> [1] "Date"
c(shelf_today, NA)
#> [1] "2026-06-30" NA
days_out <- as.numeric(shelf_today - loans$loan_date[still_out])
days_out
#> [1] 84 81 71 65 58 44 33
sum(days_out > loan_days)
#> [1] 7
loan_length <- loans$return_date - loans$loan_date
class(loan_length)
#> [1] "difftime"
sum(is.na(loan_length))
#> [1] 8
sum(loan_length > loan_days, na.rm = TRUE)
#> [1] 25
Source: R documentation: as.Date
Base R has no function for "last observation carried forward", the fill that time-series data often needs, but the index trick of the last block does it: seq_along(x) * !is.na(x) is the position where the value is known and 0 where it is not, cummax() carries the last known position forward, and indexing with it repeats the known value. A leading NA gives the index 0, which [ drops, so a vector that starts with an unknown needs a guard. Packages offer ready-made versions of this and of interpolation.
# Last observation carried forward in base R
x <- c(5, NA, NA, 8, NA)
idx <- cummax(seq_along(x) * !is.na(x))
x[idx]
#> [1] 5 5 5 8 8
Source: R documentation: cumsum
NULL in lists
NULL is not a value that a vector can hold, but a list can hold it as a component, and that creates a trap that has its own entry in the R FAQ. The question is how to set a component of a list to NULL. The answer in R FAQ 7.1 is to use x[i] <- list(NULL), and not to set x[i] or x[[i]] to NULL, because that removes the component from the list (R FAQ 7.1: How can I set components of a list to NULL?). The same entry notes that for dropping the row names of a matrix rownames(x) <- NULL is the easier way. The script runs the experiment on a list with three named components: x[["b"]] <- NULL and x$b <- NULL remove b, so the list has two components; x["b"] <- NULL removes it as well, which is the case the FAQ warns about; and only x["b"] <- list(NULL) leaves a component named b whose value is NULL, so the list still has three components, x$b is NULL, is.null(x$b) is TRUE and "b" %in% names(x) is TRUE.
| Assignment | Result for x <- list(a = 1, b = "two", c = 3) |
|---|---|
|
removes the component |
|
removes the component |
|
keeps a component |
|
creates a list with a |
|
|
source("scenario.R")
# Three ways to "set a component to NULL", and what each one does
x <- list(a = 1, b = "two", c = 3)
x[["b"]] <- NULL
names(x)
#> [1] "a" "c"
length(x)
#> [1] 2
x <- list(a = 1, b = "two", c = 3)
x$b <- NULL
names(x)
#> [1] "a" "c"
x <- list(a = 1, b = "two", c = 3)
x["b"] <- NULL
names(x)
#> [1] "a" "c"
x <- list(a = 1, b = "two", c = 3)
x["b"] <- list(NULL)
names(x)
#> [1] "a" "b" "c"
length(x)
#> [1] 3
x$b
#> NULL
is.null(x$b)
#> [1] TRUE
"b" %in% names(x)
#> [1] TRUE
# The same with a position
y <- list(1, "two", 3)
y[[2L]] <- NULL
length(y)
#> [1] 2
y <- list(1, "two", 3)
y[2L] <- list(NULL)
length(y)
#> [1] 3
y
#> [[1]]
#> [1] 1
#>
#> [[2]]
#> NULL
#>
#> [[3]]
#> [1] 3
#>
# Creating a list with a NULL component
z <- list(a = 1, b = NULL, c = 3)
length(z)
#> [1] 3
names(z)
#> [1] "a" "b" "c"
str(z)
#> List of 3
#> $ a: num 1
#> $ b: NULL
#> $ c: num 3
Source: R documentation: list
The last row of the table is the second half of the trap: an absent component and a component that holds NULL look the same through $ and [[, both returning NULL, so is.null(x$b) cannot tell them apart. The test that can is on the names: "b" %in% names(x), or exists("b", where = x). The block also shows x[["b", exact = TRUE]], the form that never partial-matches: $ does partial matching on lists, so x$b can return a component named bar when no b exists (see Lists and Subsetting and Indexing).
# NULL as a missing component and as an absent one cannot be told apart by $
z2 <- list(a = 1)
z2$b
#> NULL
z2[["b"]]
#> NULL
is.null(z2$b)
#> [1] TRUE
z2[["b", exact = TRUE]]
#> NULL
"b" %in% names(z2)
#> [1] FALSE
exists("b", where = z2)
#> [1] FALSE
is.null(z2[["b"]])
#> [1] TRUE
z2$a <- NULL
length(z2)
#> [1] 0
Source: R documentation: Extract
Lists that hold NULL come from functions that sometimes return nothing. lapply() over a list returns a list that has a NULL where the function returned NULL; sapply() then cannot simplify the result and returns the list unchanged, which makes its return type depend on the data; and vapply() stops with an error that names the problem, values must be length 1,\n but FUN(X[[1]]) result is length 0. unlist() drops the NULL elements, which is sometimes what you want and sometimes hides a bug. To find or remove the empty components use vapply(x, is.null, logical(1L)) or lengths(x) == 0L (which also catches character(0) and list()), and Filter(Negate(is.null), x) or x[lengths(x) > 0L] to keep the others; the names are preserved either way.
# lapply() over a list returns NULL entries when the function does
lapply(list(1, 2, 3), \(v) if (v > 1) v)
#> [[1]]
#> NULL
#>
#> [[2]]
#> [1] 2
#>
#> [[3]]
#> [1] 3
#>
sapply(list(1, 2, 3), \(v) if (v > 1) v)
#> [[1]]
#> NULL
#>
#> [[2]]
#> [1] 2
#>
#> [[3]]
#> [1] 3
#>
vapply(list(1, 2, 3), \(v) if (v > 1) v else NA_real_, numeric(1L))
#> [1] NA 2 3
tryCatch(
vapply(list(1, 2, 3), \(v) if (v > 1) v, numeric(1L)),
error = \(e) conditionMessage(e)
)
#> [1] "values must be length 1,\n but FUN(X[[1]]) result is length 0"
unlist(lapply(list(1, 2, 3), \(v) if (v > 1) v))
#> [1] 2 3
Source: R documentation: lapply
# Finding and removing NULL components
parts <- list(a = 1, b = NULL, c = 3, d = NULL)
vapply(parts, is.null, logical(1L))
#> a b c d
#> FALSE TRUE FALSE TRUE
lengths(parts)
#> a b c d
#> 1 0 1 0
Filter(Negate(is.null), parts)
#> $a
#> [1] 1
#>
#> $c
#> [1] 3
#>
parts[!vapply(parts, is.null, logical(1L))]
#> $a
#> [1] 1
#>
#> $c
#> [1] 3
#>
parts[lengths(parts) > 0L]
#> $a
#> [1] 1
#>
#> $c
#> [1] 3
#>
names(parts)
#> [1] "a" "b" "c" "d"
Source: R documentation: Filter
Starting a result as NULL and growing it is a common idiom, and its behaviour was made consistent in R 4.0.0: res <- NULL; res$a <- 1 gives a list, and so does res[["a"]] <- 1 and res[[1L]] <- "x", since N <- NULL; N[[1]] <- val turns N into a list also when val has length one (R NEWS: Changes in R 4.0.0); out <- c(out, x) gives a vector. So start with list() or vector("list", n) when you want a list and with numeric(0) or character(0) when you want a vector, rather than relying on what NULL becomes.
# Growing from NULL
out <- NULL
for (i in 1:3) out <- c(out, i * 10)
out
#> [1] 10 20 30
res <- NULL
res$a <- 1
res
#> $a
#> [1] 1
#>
res2 <- NULL
res2[["a"]] <- 1
res2
#> $a
#> [1] 1
#>
res3 <- NULL
res3[["a"]] <- 1:2
res3
#> $a
#> [1] 1 2
#>
res4 <- NULL
res4[[1L]] <- "x"
res4
#> [[1]]
#> [1] "x"
#>
Source: R NEWS: Changes in R 4.0.0
The book objects of the scenario are lists of six fields of length 1 where a field that is unknown is NA, never NULL: new_book() and validate_book() rely on that, and b$pages is NA_integer_ with is.null(b$pages) being FALSE. Asking for a field that does not exist (b$notes) is NULL. The last lines show what happens when a NULL gets in through the back door: b$year <- NULL removes the field, and validate_book(b) then stops with missing value where TRUE/FALSE needed, which is a misleading message for a missing field. The reason is that is.na(NULL) is logical(0), !logical(0) is logical(0), and logical(0) && ... is NA (see Conditions that are not one TRUE or FALSE), so the if receives an NA. A validator should check the field names before the values.
In a data frame the same two operators do the two jobs: df$b <- NULL removes a column and df$c <- NA creates a column of logical NA, with the type logical, to be replaced by a typed vector later. df["a"] <- NULL on the last remaining column leaves a frame with 3 rows and no columns. See Data Frames.
# A book is a list of length-1 fields: a field cannot be NULL, it is NA
b <- book("Ten Winters", "Priya Raman", year = 2022)
b
#> <book> Ten Winters (Priya Raman, 2022)
str(unclass(b))
#> List of 6
#> $ title : chr "Ten Winters"
#> $ author: chr "Priya Raman"
#> $ isbn : chr NA
#> $ year : int 2022
#> $ genre : chr NA
#> $ pages : int NA
is.null(b$pages)
#> [1] FALSE
is.na(b$pages)
#> [1] TRUE
b$notes
#> NULL
is.null(b$notes)
#> [1] TRUE
b$year <- NULL
names(unclass(b))
#> [1] "title" "author" "isbn" "genre" "pages"
tryCatch(validate_book(b), error = \(e) conditionMessage(e))
#> [1] "missing value where TRUE/FALSE needed"
Source: R documentation: NULL
# Data frames: NULL removes a column, NA fills one
d <- data.frame(a = 1:3, b = c("x", "y", "z"))
d$b <- NULL
names(d)
#> [1] "a"
d$c <- NA
d
#> a c
#> 1 1 NA
#> 2 2 NA
#> 3 3 NA
vapply(d, typeof, "")
#> a c
#> "integer" "logical"
d$c <- NULL
d$e <- NULL
names(d)
#> [1] "a"
d["a"] <- NULL
dim(d)
#> [1] 3 0
Source: R documentation: data.frame
The null-coalescing operator %||%
Taking a default when something is NULL is so common (an optional argument, an absent list component, a missing option) that R 4.4.0 added an operator for it: L %||% R is the idiom for if (!is.null(L)) L else R (R NEWS: Changes in R 4.4.0; documented with the control-flow operators in R documentation: Control). The right operand is evaluated only when the left one is NULL, like ||. The operator tests for NULL and only for NULL: NA %||% "default" is NA, character(0) %||% "default" is character(0) and list() %||% "default" is list(), because none of them is NULL. For a missing value in a book field, b$genre %||% "unclassified" is NA, while b$notes %||% "no notes" is the default, and the safe test for the field is is.na(b$genre). Code that supports R older than 4.4.0 defines the helper itself (the function fallback() in the script is one possible definition) and loses nothing when it runs on a newer R, though a package that defines its own %||% masks the base one inside its namespace.
source("scenario.R")
# %||% returns the left side unless it is NULL (R 4.4.0)
NULL %||% "default"
#> [1] "default"
"value" %||% "default"
#> [1] "value"
1:3 %||% 0
#> [1] 1 2 3
character(0) %||% "default"
#> character(0)
NA %||% "default"
#> [1] NA
list() %||% "default"
#> list()
Source: R documentation: Control
# The same in plain R, which is what older code does
fallback <- function(l, r) if (is.null(l)) r else l
fallback(NULL, "default")
#> [1] "default"
Source: R documentation: Control
# Typical use: optional arguments and list components
opts <- list(sep = ";")
opts$sep %||% ","
#> [1] ";"
opts$header %||% TRUE
#> [1] TRUE
b <- book("Paper Moons", "Noor Haddad", year = 2005)
b$genre %||% "unclassified"
#> [1] NA
b$notes %||% "no notes"
#> [1] "no notes"
is.na(b$genre)
#> [1] TRUE
if (is.na(b$genre)) "unclassified" else b$genre
#> [1] "unclassified"
Source: R documentation: Control
# The right side is evaluated only when needed
NULL %||% {
cat("computing the default\n")
42
}
#> computing the default
#> [1] 42
"kept" %||% stop("not reached")
#> [1] "kept"
Source: R documentation: Control
# It is a function like any other operator
`%||%`(NULL, "x")
#> [1] "x"
Reduce(`%||%`, list(NULL, NULL, "third", "fourth"))
#> [1] "third"
args(`%||%`)
#> function (x, y)
#> NULL
Source: R documentation: Reduce
Zero-length vectors
A zero-length vector is a vector with no elements, and unlike NULL it has a type: character(0), integer(0), numeric(0), logical(0) and the empty list(). They are what a filter returns when nothing matches (x[x > 5], which(...), books$title[books$pages > 1000L & !is.na(books$pages)]), what an index of 0 selects (books$year[0L]), and what most functions return when given NULL. A data frame with no rows prints as <0 rows>, and a name that is not in a frame gives NULL. The Language Definition says that NULL is not to be confused with a vector or list of zero length (The R Language Definition: NULL), and the next table is the practical difference between the three neighbours.
| Question | NULL |
character(0) |
NA_character_ |
|---|---|---|---|
|
0 |
0 |
1 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Source: R documentation: NULL
source("scenario.R")
# Zero-length vectors of each type
character(0)
#> character(0)
integer(0)
#> integer(0)
numeric(0)
#> numeric(0)
logical(0)
#> logical(0)
length(character(0))
#> [1] 0
typeof(integer(0))
#> [1] "integer"
identical(character(0), vector("character", 0L))
#> [1] TRUE
c()
#> NULL
length(c())
#> [1] 0
x <- c(1, 2, 3)
x[x > 5]
#> numeric(0)
books$title[books$pages > 1000L & !is.na(books$pages)]
#> character(0)
which(books$pages > 1000L)
#> integer(0)
books$year[0L]
#> integer(0)
books[0L, c("title", "year")]
#> [1] title year
#> <0 rows> (or 0-length row.names)
nrow(books[books$year > 2100L & !is.na(books$year), ])
#> [1] 0
names(books)[0L]
#> character(0)
books$nope
#> NULL
books[["nope"]]
#> NULL
is.null(books$nope)
#> [1] TRUE
Source: R documentation: vector
Functions on an empty input follow the mathematics of the empty set, and a few of the results are surprising until the rule is seen. sum(integer(0)) is 0 and prod(numeric(0)) is 1, the identity elements of addition and multiplication; any(logical(0)) is FALSE and all(logical(0)) is TRUE (the "vacuous truth", for which no counter-example exists); mean(numeric(0)) is NaN, because it computes 0/0; median(numeric(0)) and var(numeric(0)) are NA. The ones that return empty things return a typed one: nchar(character(0)) is integer(0), toupper(character(0)) is character(0), cumsum(numeric(0)) is numeric(0). Two functions deserve a warning label. sapply() of an empty input returns list() instead of a typed empty vector, because it has no result to simplify, while vapply() returns the empty vector of its template type, so use vapply() whenever the input can be empty (see The apply Family and Functionals); and paste0("pages: ", integer(0)) is "pages: " and not character(0), because a zero-length argument is treated as the empty string, unless recycle0 = TRUE is given, an argument added in R 4.0.1 (R NEWS: Changes in R 4.0.1).
The loop variable of a for over an empty sequence is never set and the body runs zero times, provided that the sequence is a proper empty one: seq_along(NULL) and seq_len(0L) are integer(0), whereas 1:length(x) is 1:0, which counts down to 0 and runs the body twice (see Control Flow).
# Functions on zero-length input
sum(integer(0))
#> [1] 0
prod(numeric(0))
#> [1] 1
mean(numeric(0))
#> [1] NaN
median(numeric(0))
#> [1] NA
var(numeric(0))
#> [1] NA
length(unique(character(0)))
#> [1] 0
any(logical(0))
#> [1] FALSE
all(logical(0))
#> [1] TRUE
nchar(character(0))
#> integer(0)
paste0("a", character(0))
#> [1] "a"
paste("a", character(0))
#> [1] "a "
paste(character(0), collapse = "+")
#> [1] ""
paste0("pages: ", integer(0))
#> [1] "pages: "
paste0("pages: ", integer(0), recycle0 = TRUE)
#> character(0)
toupper(character(0))
#> character(0)
cumsum(numeric(0))
#> numeric(0)
rev(integer(0))
#> integer(0)
sort(integer(0))
#> integer(0)
which(logical(0))
#> integer(0)
seq_along(NULL)
#> integer(0)
seq_len(0L)
#> integer(0)
for (i in seq_along(NULL)) print(i)
vapply(NULL, nchar, integer(1L))
#> integer(0)
sapply(NULL, nchar)
#> list()
lapply(NULL, nchar)
#> list()
unlist(list())
#> NULL
integer(0) + 1L
#> integer(0)
integer(0) == 1L
#> logical(0)
sum(integer(0)) == 0L
#> [1] TRUE
Source: R documentation: sum
max() and min() of nothing
max() and min() of an empty vector have no mathematically sensible result, and R returns the identity of the operation, -Inf and Inf, with a warning: no non-missing arguments to max; returning -Inf and no non-missing arguments to min; returning Inf. The same warning appears when every value is NA and na.rm = TRUE removes them all, which is how a filter that matches nothing and a column that is all missing end up in the same message. max(character(0)) returns NA with a different warning, no non-missing arguments, returning NA. range() of an empty vector warns once for the minimum. which.max() and which.min() ignore both cases and return integer(0).
Two defences work. The first is to give max() a floor that is a legitimate answer for the data (max(x, 0), or max(numeric(0), -Inf), which is -Inf without the warning) and the second is to test first and choose the result explicitly, as the safe_max() function of the script does: if x is empty or all missing, return a default, else take the maximum of the known values. Suppressing the warning with suppressWarnings() leaves -Inf in the result, which then propagates as a number and shows up later as an impossible axis or a division by zero.
# max() and min() of nothing warn and return -Inf and Inf
tryCatch(max(numeric(0)), warning = \(w) conditionMessage(w))
#> [1] "no non-missing arguments to max; returning -Inf"
tryCatch(min(numeric(0)), warning = \(w) conditionMessage(w))
#> [1] "no non-missing arguments to min; returning Inf"
tryCatch(max(character(0)), warning = \(w) conditionMessage(w))
#> [1] "no non-missing arguments, returning NA"
tryCatch(
range(numeric(0)),
warning = \(w) conditionMessage(w)
)
#> [1] "no non-missing arguments to min; returning Inf"
suppressWarnings(max(numeric(0)))
#> [1] -Inf
suppressWarnings(max(c(NA_real_, NA_real_), na.rm = TRUE))
#> [1] -Inf
tryCatch(
max(c(NA_real_, NA_real_), na.rm = TRUE),
warning = \(w) conditionMessage(w)
)
#> [1] "no non-missing arguments to max; returning -Inf"
max(numeric(0), -Inf)
#> [1] -Inf
max(c(3, NA), 0, na.rm = TRUE)
#> [1] 3
which.max(numeric(0))
#> integer(0)
which.max(c(NA_real_, NA_real_))
#> integer(0)
tryCatch(
max(books$pages[books$year > 2100L], na.rm = TRUE),
warning = \(w) conditionMessage(w)
)
#> [1] "no non-missing arguments to max; returning -Inf"
Source: R documentation: Extremes
# A safe maximum: give a default
safe_max <- function(x, default = NA_real_) {
if (length(x) == 0L || all(is.na(x))) default else max(x, na.rm = TRUE)
}
safe_max(numeric(0))
#> [1] NA
safe_max(c(NA, NA), default = 0)
#> [1] 0
safe_max(books$pages)
#> [1] 432
Source: R documentation: Extremes
Conditions that are not one TRUE or FALSE
The condition of an if or a while must be a single TRUE or FALSE, and everything else is an error whose text depends on what the condition was instead. These are the messages, each captured from R 4.6.1 in the script that follows:
| Message | The condition was | Typical origin |
|---|---|---|
|
|
a typo in a column name ( |
|
|
a comparison with an unknown value, an |
|
a logical vector of length 2 or more |
a vectorised comparison used in an |
|
a string that is not |
|
|
the operand of |
|
|
|
|
The first two rows are the subject of this page. A NULL or an empty vector in a condition is the condition of length zero, and the cause is nearly always upstream: books$pagez (a typo) is NULL, NULL > 300L is logical(0), and if (logical(0)) stops with argument is of length zero; but filtering the right column on a title that does not exist gives the same message. Both messages mean that the value has no answer, and you should know before the if whether you expect exactly one answer. while reports the same two messages.
A zero-length operand of && or || is not an error in R 4.6.1: logical(0) && TRUE is NA, and logical(0) || TRUE is TRUE, as if the empty operand were an NA. So an if whose condition has a zero-length operand on the left of && reports missing value where TRUE/FALSE needed, not the length-zero message, which is the confusing error that validate_book() showed above. A NULL or a string is a different matter and is an error in && and ||, with the invalid 'x' type message.
source("scenario.R")
# The three messages of a condition that is not one TRUE or FALSE
tryCatch(if (logical(0)) "yes", error = \(e) conditionMessage(e))
#> [1] "argument is of length zero"
tryCatch(if (NULL) "yes", error = \(e) conditionMessage(e))
#> [1] "argument is of length zero"
tryCatch(if (NA) "yes", error = \(e) conditionMessage(e))
#> [1] "missing value where TRUE/FALSE needed"
tryCatch(if (NA_character_) "yes", error = \(e) conditionMessage(e))
#> [1] "argument is not interpretable as logical"
tryCatch(if ("a") "yes", error = \(e) conditionMessage(e))
#> [1] "argument is not interpretable as logical"
tryCatch(
if (c(TRUE, FALSE)) "yes",
error = \(e) conditionMessage(e)
)
#> [1] "the condition has length > 1"
tryCatch(while (NA) 1, error = \(e) conditionMessage(e))
#> [1] "missing value where TRUE/FALSE needed"
tryCatch(while (logical(0)) 1, error = \(e) conditionMessage(e))
#> [1] "argument is of length zero"
Source: R documentation: Control
# The same errors from a typo in a column name, or an empty filter
title <- "A Map of Salt"
tryCatch(
if (books$pagez[books$title == title] > 300L) "long",
error = \(e) conditionMessage(e)
)
#> [1] "argument is of length zero"
tryCatch(
if (books$pages[books$title == "No such title"] > 300L) "long",
error = \(e) conditionMessage(e)
)
#> [1] "argument is of length zero"
tryCatch(
if (books$pages[books$title == title] > 300L) "long",
error = \(e) conditionMessage(e)
)
#> [1] "missing value where TRUE/FALSE needed"
Source: R documentation: Control
# && and || with a zero-length operand: NA, not an error (R 4.3.0)
logical(0) && TRUE
#> [1] NA
logical(0) || TRUE
#> [1] TRUE
tryCatch(NULL && TRUE, error = \(e) conditionMessage(e))
#> [1] "invalid 'x' type in 'x && y'"
tryCatch(
c(TRUE, TRUE) && TRUE,
error = \(e) conditionMessage(e)
)
#> [1] "'length = 2' in coercion to 'logical(1)'"
tryCatch(
"a" && TRUE,
error = \(e) conditionMessage(e)
)
#> [1] "invalid 'x' type in 'x && y'"
Source: R documentation: Logic
Guarding a condition
Make the condition always have exactly one answer, and make the choice for the unknown explicit. The tools are few:
-
length(x) == 1L && !is.na(x) && x > 300Lspells out the three requirements in order, and relies on&&stopping at the firstFALSE. -
isTRUE(x > 300L)isTRUEonly when its argument is exactly the single valueTRUE;NA,FALSE,logical(0)and a vector of length 2 all giveFALSE.isFALSE()is its counterpart, true only for the singleFALSE. Both are in the help page of the logical operators, and they are the shortest way to fold "unknown" into one of the branches. The price is that a typo in the column name also folds intoFALSEsilently, so use them where unknown really means no. -
if (length(x))tests that a vector is not empty, using the rule that 0 isFALSEand any other number isTRUE;if (length(x) > 0L)says the same more plainly. -
any(x, na.rm = TRUE)andall(x, na.rm = TRUE)reduce a vector to one answer, withna.rm = TRUEdeciding to ignore the unknowns. For an empty vectorany()isFALSEandall()isTRUE, soall(x > 0)on an empty filter is true: checklength(x)first when "all of nothing" is not a sensible answer, as thevacuousline shows. -
nzchar(x)is not a test for a missing string (it isTRUEforNAby default), so use!is.na(x) && nzchar(x).
stopifnot() is the tool for conditions that must hold. It treats NA as a failure: stopifnot(books$pages > 0L) stops with books$pages > 0L are not all TRUE because of the two unknown page counts, and stopifnot(NA) says NA is not TRUE. An empty logical vector passes (stopifnot(logical(0)) is silent), as it is "all TRUE". A named argument supplies the message, as in stopifnot("pages must be known" = !anyNA(books$pages)), which is the form to use for input checks (see Defensive Programming and Input Validation and Conditions and Error Handling).
# Guard the condition so that it always has exactly one answer
pages <- books$pages[books$title == "A Map of Salt"]
if (length(pages) == 1L && !is.na(pages) && pages > 300L) "long" else "other"
#> [1] "other"
if (isTRUE(pages > 300L)) "long" else "other"
#> [1] "other"
isTRUE(logical(0))
#> [1] FALSE
isTRUE(NA)
#> [1] FALSE
isTRUE(c(TRUE, TRUE))
#> [1] FALSE
isFALSE(NA)
#> [1] FALSE
isFALSE(logical(0))
#> [1] FALSE
isFALSE(FALSE)
#> [1] TRUE
if (length(pages)) "has a value" else "empty"
#> [1] "has a value"
if (length(character(0))) "has a value" else "empty"
#> [1] "empty"
if (nzchar(NA_character_)) "has text" else "empty"
#> [1] "has text"
if (!is.na(books$isbn[[9L]]) && nzchar(books$isbn[[9L]])) "isbn" else "no isbn"
#> [1] "no isbn"
Source: R documentation: Logic
# any() and all() answer for a whole vector, but see the empty case
if (any(books$pages > 400L, na.rm = TRUE)) "some long" else "none long"
#> [1] "some long"
if (all(books$pages > 50L, na.rm = TRUE)) "all above 50" else "some short"
#> [1] "all above 50"
if (all(books$pages[books$year > 2100L] > 0L, na.rm = TRUE)) "vacuous"
#> [1] "vacuous"
all(logical(0))
#> [1] TRUE
any(logical(0))
#> [1] FALSE
Source: R documentation: all
# stopifnot() on NA and on nothing
tryCatch(
stopifnot(books$pages > 0L),
error = \(e) conditionMessage(e)
)
#> [1] "books$pages > 0L are not all TRUE"
tryCatch(
stopifnot(NA),
error = \(e) conditionMessage(e)
)
#> [1] "NA is not TRUE"
stopifnot(logical(0))
stopifnot(all(books$pages > 0L, na.rm = TRUE), !anyNA(books$title))
tryCatch(
stopifnot("pages must be known" = !anyNA(books$pages)),
error = \(e) conditionMessage(e)
)
#> [1] "pages must be known"
Source: R documentation: stopifnot
identical(), switch() and sizes
identical() and NA
identical() compares two objects for exact equality, including their types, and returns a single TRUE or FALSE that is never NA. That makes it the safe comparison for NA and NULL, but the exactness catches people: identical(NA, NA_integer_) is FALSE because the types differ, identical(NA_real_, NaN) is FALSE, identical(NULL, character(0)) is FALSE, and identical(books$year[7L], NA) is FALSE, because the year is an integer NA and the constant is a logical one. The comparison with the right type, identical(books$year[7L], NA_integer_), is TRUE, and is.na() avoids the question of type altogether. The second pitfall is the vector: identical(x, NA) for x <- c(NA, NA) is FALSE because the lengths differ, so it is not a test for "all missing"; use all(is.na(x)). The same exactness applies to isTRUE(all.equal(...)) on NULL against character(0).
source("scenario.R")
# identical(): exact, including the type of the NA
identical(NA, NA)
#> [1] TRUE
identical(NA, NA_integer_)
#> [1] FALSE
identical(NA_real_, NA_real_)
#> [1] TRUE
identical(NA_real_, NaN)
#> [1] FALSE
identical(NaN, NaN)
#> [1] TRUE
identical(c(1, NA), c(1, NA))
#> [1] TRUE
identical(books$year[7L], NA)
#> [1] FALSE
identical(books$year[7L], NA_integer_)
#> [1] TRUE
identical(is.na(books$year[7L]), TRUE)
#> [1] TRUE
is.na(books$year[7L])
#> [1] TRUE
identical(NULL, NULL)
#> [1] TRUE
identical(NULL, character(0))
#> [1] FALSE
identical(character(0), character(0))
#> [1] TRUE
identical(character(0), integer(0))
#> [1] FALSE
identical(list(), NULL)
#> [1] FALSE
isTRUE(all.equal(NULL, character(0)))
#> [1] FALSE
identical(logical(0), NULL)
#> [1] FALSE
Source: R documentation: identical
# identical() with a vector: identical(x, NA) is not "is x all NA"
x <- c(NA, NA)
identical(x, NA)
#> [1] FALSE
all(is.na(x))
#> [1] TRUE
identical(x, c(NA, NA))
#> [1] TRUE
identical(NA_integer_, c(NA_integer_))
#> [1] TRUE
identical(x[1L], NA)
#> [1] TRUE
Source: R documentation: identical
switch() and NA
switch() selects by a string or by a number, and its help page does not describe what it does with a missing value, so the next block reports what R 4.6.1 does rather than a documented guarantee. With a string, an NA_character_ is not an error: it takes the default alternative, and if an alternative is named NA (written "NA" = ...) it matches that one. With a number, an integer or logical NA, or a number outside the range, returns an invisible NULL. With a NULL, a zero-length vector or a vector of length two or more, it stops with EXPR must be a length 1 vector. So a switch() on data that may be missing needs an explicit if (is.na(x)) before it, and one that returns a value should always have a default alternative. See Control Flow for switch() in general.
# NA in switch(): a missing string, a missing number
sw_chr <- function(x) {
switch(x, a = "first", b = "second", "default")
}
sw_chr("a")
#> [1] "first"
sw_chr("z")
#> [1] "default"
sw_chr(NA_character_)
#> [1] "default"
sw_na_branch <- function(x) {
switch(x, a = "first", "NA" = "the string NA", "default")
}
sw_na_branch(NA_character_)
#> [1] "the string NA"
sw_na_branch("NA")
#> [1] "the string NA"
sw_num <- function(i) {
switch(i, "one", "two", "three")
}
sw_num(2L)
#> [1] "two"
is.null(sw_num(5L))
#> [1] TRUE
is.null(sw_num(NA_integer_))
#> [1] TRUE
is.null(sw_num(NA))
#> [1] TRUE
withVisible(sw_num(NA))$visible
#> [1] FALSE
tryCatch(sw_num(NULL), error = \(e) conditionMessage(e))
#> [1] "EXPR must be a length 1 vector"
tryCatch(sw_num(integer(0)), error = \(e) conditionMessage(e))
#> [1] "EXPR must be a length 1 vector"
tryCatch(sw_num(1:2), error = \(e) conditionMessage(e))
#> [1] "EXPR must be a length 1 vector"
is.null(switch(NA, a = 1, 2))
#> [1] TRUE
switch(NA_character_, a = 1, 2)
#> [1] 2
tryCatch(
switch(character(0), a = 1, 2),
error = \(e) conditionMessage(e)
)
#> [1] "EXPR must be a length 1 vector"
tryCatch(
switch(NULL, a = 1, 2),
error = \(e) conditionMessage(e)
)
#> [1] "EXPR must be a length 1 vector"
Source: R documentation: switch
NROW, NCOL and lengths
nrow() and ncol() return NULL for anything that is not a matrix or data frame, which is awkward when the object may be a plain vector or NULL. NROW() and NCOL() treat a vector as a one-column matrix: NROW(1:3) is 3 and NCOL(1:3) is 1, NROW(character(0)) is 0, and a data frame with no rows gives 0. R 4.4.0 made NCOL(NULL) return 0 instead of 1, consistently with cbind(), while NROW(NULL) was already 0 (R NEWS: Changes in R 4.4.0); code written for older versions that counted columns of a possible NULL should be checked. seq_len(NROW(x)) is a safe loop sequence for a vector, a matrix or a data frame that may be empty.
length() of a list(NULL) is 1, since the list holds one component that happens to be NULL, while length(NULL) and length(list()) are 0. lengths() gives the lengths of the components of a list at once, and is the quick way to find the empty ones. is.na() on a list is true only for the components that are single missing values (the help page: a length-one atomic vector whose element is NA or NaN), so is.na(list(1, NA, NULL, c(NA, 2), "a")) is FALSE TRUE FALSE FALSE FALSE. anyNA() gives the same for a list unless recursive = TRUE, which looks inside the components: anyNA(list(1, c(2, NA))) is FALSE and anyNA(list(1, c(2, NA)), recursive = TRUE) is TRUE. anyNA(NULL) is FALSE and is.na(NULL) is logical(0); R 3.1.0 introduced anyNA() as a fast version of any(is.na(x)), and is.na(NULL) stopped warning in 3.4.4 (R NEWS (R 3.x): Changes in R 3.1.0 and 3.4.4).
# NROW and NCOL, where nrow() and ncol() give NULL
nrow(1:3)
#> NULL
ncol(1:3)
#> NULL
NROW(1:3)
#> [1] 3
NCOL(1:3)
#> [1] 1
nrow(NULL)
#> NULL
NROW(NULL)
#> [1] 0
NCOL(NULL)
#> [1] 0
nrow(books)
#> [1] 16
NROW(books$title)
#> [1] 16
NROW(books[0L, ])
#> [1] 0
NROW(character(0))
#> [1] 0
NCOL(books)
#> [1] 6
dim(NULL)
#> NULL
dim(1:3)
#> NULL
nrow(books[books$year > 2100L & !is.na(books$year), ])
#> [1] 0
for (i in seq_len(NROW(NULL))) print(i)
Source: R documentation: nrow
# length() and lengths(), and what is.na() does to a list
length(NULL)
#> [1] 0
length(list(NULL))
#> [1] 1
length(list())
#> [1] 0
length(NA)
#> [1] 1
lengths(list(1:3, NULL, NA, character(0)))
#> [1] 3 0 1 0
is.na(list(1, NA, NULL, c(NA, 2), "a"))
#> [1] FALSE TRUE FALSE FALSE FALSE
anyNA(list(1, NA))
#> [1] TRUE
anyNA(list(1, c(2, NA)))
#> [1] FALSE
anyNA(list(1, c(2, NA)), recursive = TRUE)
#> [1] TRUE
anyNA(NULL)
#> [1] FALSE
sum(is.na(NULL))
#> [1] 0
Source: R documentation: lengths
Where older tutorials differ
Older tutorials and answers are right about the basics and wrong or out of date about a few details. When one of them disagrees with this page, the table says why.
| Older tutorial | Current practice | Why | Version |
|---|---|---|---|
|
|
a comparison with an unknown is unknown; the result is all |
always ( |
|
|
the same answer, simpler, and faster for atomic vectors; |
R 3.1.0 |
|
|
|
always |
a summary call that names |
name it once |
it is now an error ( |
R 4.4.0 |
|
|
the default of |
R 3.3.0 |
|
|
|
R 4.4.0 |
|
|
consistency with |
R 4.4.0 |
|
|
the warning was removed |
R 3.4.4 |
defining |
base |
it is in |
R 4.4.0 |
|
it gives a list |
consistency |
R 4.0.0 |
|
reduce with |
a condition of length greater than one is an error |
R 4.2.0 |
|
use |
it is an error: |
R 4.3.0 |
|
|
assigning |
always |
|
|
|
always |
|
the same, but with the right type ( |
a fill of the wrong type changes the type of the column |
always |
|
|
|
always |
For the full list of changes by release see What’s New and Migrating Older Code.
Where to go next
-
Atomic Vectors and Types for the six atomic types, coercion and why
NAhas several types. -
Numbers and Arithmetic for integer overflow to
NA,InfandNaNin arithmetic, and the floating-point limits. -
Subsetting and Indexing for
NAand zero in subscripts,[[versus[anddrop = FALSE. -
Lists and Data Frames for
NULLcomponents, columns withNAandcomplete.cases()on frames. -
Control Flow for the conditions of
ifandwhile,switch()andifelse(). -
Operators and Precedence for
&&,||,isTRUE()and%||%. -
Defensive Programming and Input Validation for
stopifnot()and checks on missing input. -
Idioms and Common Pitfalls for the same traps in a wider list.
References
-
An Introduction to R: Missing values
-
The R Language Definition: NA handling
-
The R Language Definition: Indexing by vectors
-
R FAQ 7.1: How can I set components of a list to NULL?
-
R NEWS: Changes in R 4.0.0, 4.0.1, 4.1.0, 4.2.0, 4.3.0 and 4.4.0
-
R NEWS (R 3.x): Changes in R 3.1.0, 3.3.0 and 3.4.4
-
NA (is.na, anyNA,
is.na<-, NA_integer_ and the typed constants) -
is.finite (is.nan, is.infinite)
-
NULL (is.null)
-
na.fail (na.omit, na.exclude, na.pass)
-
Control (if, while, switch conditions and %||%)
-
Logic (&, |, &&, ||, xor, isTRUE, isFALSE)
-
nrow (NROW, NCOL)
-
sum (prod, na.rm)
-
Extremes (max, min)
-
table (useNA)
-
nchar (nzchar, keepNA)
-
paste (recycle0)
-
Extract (NA and NULL subscripts,
[[<-) -
match (%in%)
-
read.table (na.strings)
-
lm (na.action)
-
options (na.action)