Reading and Writing Text Data
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. The reference material consulted while preparing these pages is listed in the section bibliography. |
Checked against R 4.6.1 (2026-06-24) on 2026-10-20; R 4.6.2 is scheduled for 2026-10-29. Text is the format in which most
data reaches R and in which most results leave it, and R has a small family of functions for each direction:
read.table() and its relatives turn a delimited file into a data frame, scan() and readLines() read values or lines,
write.table() and write.csv() write a data frame, and cat(), writeLines(), sprintf() and sink() write text for
people. This page covers every argument of the readers and writers that changes what you get, the conventions of CSV
files and how they differ between countries, the errors and warnings that a badly described file produces (with the exact
text R prints), why a round trip through a text file loses some types and how to keep them, the datasets that ship with R
and the way data() loads them, and which package to reach for when the data is in an Excel, SPSS, SAS, Stata, Octave,
Parquet or JSON file instead of a text file. Every example uses base R and the foreign package, which is a recommended
package shipped with R (checked at version 0.8-91, used for one short example). The add-on packages readr (checked at
version 2.2.0), vroom (1.7.1), readxl (1.6.0), haven (2.5.5) and data.table (1.18.6.1) are installed in the checking
environment and were called on the shelf files to check the statements made about them, but they are described in prose, and
none of their code appears in the examples; arrow, writexl, openxlsx and R.matlab are not
installed there and are named only as pointers.
Every example below is a complete script that was run with Rscript on R 4.6.1 with warnings printed as they happen
(options(warn = 1)), and the lines starting with #> are what it printed. The scripts also pass the default lintr
linters. Errors and warnings that a script shows on purpose are caught with tryCatch() or withCallingHandlers() and
printed as their conditionMessage(), so the text is exactly what R says. The examples use the library-shelf scenario
of the section: the files books.csv and loans.csv, the data frames books and loans, the constants shelf_today
and loan_days and the book class come from the scenario.R file described on the
section index, and the scripts start in the directory that holds those files, so
read.csv("books.csv") and source("scenario.R") work as written. books has 16 rows and the columns title,
author, isbn, year, genre and pages; loans has 60 rows and the columns loan_id, isbn, member,
loan_date and return_date. Both are deliberately untidy, because the interesting part of reading a file is what the
file does not say. Files that an example writes go to tempfile() or are removed before the script ends, and file names
that appear in messages are relative names such as draft.csv. Where an example needs a text that is too awkward to read
from the shelf files (a Latin-1 file, a file without a final newline, a Windows export with semicolons), the script
builds it in a few lines.
Text files in R: the big picture
An Introduction to R puts the design assumption plainly: R’s input facilities are simple and fairly strict, and the
designers presume that you will adjust an awkward input file with other tools (an editor, a scripting language) so that
it fits what R expects
(An Introduction to R: Reading data from files).
R Data Import/Export, the manual for this whole topic, adds that the primary function for importing from a text file is
scan() and that it underlies most of the more convenient functions, which then build on it
(R Data Import/Export: Imports). Everything on this
page follows from those two statements. A text file carries no description of itself, so every fact about it that R cannot
guess has to be said in an argument, and the readers differ in how much they guess.
| Function | Reads or writes | Returns or produces | Use it when |
|---|---|---|---|
|
a delimited file with one record per line |
a data frame, with a type guessed for each column |
the data is a table |
|
a stream of values separated by white space or one separator |
a vector of one type, or a list with one vector per field |
the data is a numeric matrix, a list of values, or you want the fast low-level reader |
|
a file, one string per line |
a character vector, nothing parsed |
the lines need your own parsing, or the file is not a table |
|
a file whose fields sit in fixed character positions |
a data frame |
there are no delimiters at all |
|
any of the above |
the number of fields on each line |
a read fails and you need to find the line |
|
a character vector or data frame already in memory |
the same, with each column converted |
the guessing of |
|
a data frame or matrix to a file or connection |
a delimited text file |
the data goes to another program |
|
strings or values to a file or connection |
plain text |
reports, logs, small outputs |
|
the printed output of other code |
a file or a character vector |
you want to keep what |
Source: R Data Import/Export: Imports
Seven facts that a text file does not state, and the argument that supplies each one, are the real content of
read.table(). The table lists them; the rest of this page takes them in turn.
| The file does not say | Wrong guess looks like | Argument that says it |
|---|---|---|
whether the first line is names |
the names are data, or the first row is lost |
|
what separates fields |
one wide column, or |
|
what spells a missing value |
a numeric column becomes text because of one |
|
the type of each column |
leading zeros lost, dates as text, a code of |
|
whether |
a title cut short, or a |
|
how many fields a short or long line has |
rows shifted or padded with |
|
which encoding the bytes are in |
|
|
Source: R documentation: read.table
A few related pages are the natural neighbours of this one.
Data Frames is where the result of read.csv() goes next,
Encodings, Locales and Internationalization
covers the bytes, declared encodings and iconv() in full (this page keeps to what the readers and writers need),
Connections and the File System covers the file(),
gzfile() and textConnection() objects that these functions open for you, and
Serialization: RDS, RData and JSON covers the
formats that keep R types exactly.
The two shelf files
Before reading a file, look at it. readLines() reads without interpreting, so it is the safe way to see the first
lines of a file, to count its lines, and to check for surprises such as a header with different quotes from the data.
The first script prints the first lines of both shelf files exactly as they are on disk.
# The first lines of the two shelf files, exactly as they are on disk
writeLines(readLines("books.csv", n = 6))
#> "title","author","isbn","year","genre","pages"
#> "The Dispossessed","Ursula K. Le Guin","978-0-306-40615-7",1974,"Science fiction",387
#> "Pride and Prejudice","Jane Austen","978-0-14-143951-8",1813,"Classic",432
#> " the quiet streets ","Mara Quill","9780001000100",2019,"Fiction",214
#> "Loud Roofs","Mara Quill","978-0-00-200029-1",2021,"fiction",198
#> "Midnight Ledger","Iris Fenwick","978-0-00-300031-3",2016,"Mystery",301
writeLines(readLines("loans.csv", n = 6))
#> "loan_id","isbn","member","loan_date","return_date"
#> "L001","9780141439518","Hugo Silva","2025-01-14","2025-02-04"
#> "L002","9780306406157","Ben Okafor","2025-01-16","2025-01-26"
#> "L003","9780002000291","Dev Patel","2025-01-21","2025-01-22"
#> "L004","9780005000571","Eva Lindqvist","2025-01-26","2025-02-21"
#> "L005","978030640615","Ben Okafor","2025-02-19","2025-03-17"
# One header line plus the records
length(readLines("books.csv"))
#> [1] 17
length(readLines("loans.csv"))
#> [1] 61Source: R documentation: readLines
Three things in the text are worth noticing before any parsing happens. Every text field is in double quotes and every
number is bare, which is what write.csv() produces. The ISBN in loans.csv is quoted although it is made of digits,
which is a hint that it is an identifier, not a quantity. And the last two columns of loans.csv hold dates in the
ISO 8601 form YYYY-MM-DD as plain text: CSV has no date type, so a date is always text until some function converts
it. The scenario also leaves empty fields in both files (an unreturned loan has an empty return_date, and some books
have no year or page count) and a handful of spelling and spacing problems that belong to the cleaning pages. The
reading functions treat those empty fields differently depending on the column type, and that difference is the first
thing to understand.
read.csv() and read.table()
read.table() is the main function for reading tabular data into R. The other four
readers are the same function with different defaults: the help page says that read.csv() and read.csv2() are
identical to read.table() except for the defaults, and likewise read.delim() and read.delim2(). Reading the
signatures from R itself shows what changes.
readers <- c("read.table", "read.csv", "read.csv2", "read.delim", "read.delim2")
settings <- c("header", "sep", "quote", "dec", "fill", "comment.char")
# The defaults of each reader, as the function signatures state them
defaults <- vapply(readers, \(name) {
formals_of <- formals(get(name))
vapply(settings, \(arg) {
paste(deparse(formals_of[[arg]]), collapse = "")
}, character(1))
}, character(length(settings)))
cat(sprintf("%-13s", ""), sprintf("%-19s", readers), "\n", sep = "")
#> read.table read.csv read.csv2 read.delim read.delim2
for (arg in settings) {
cat(sprintf("%-13s", arg), sprintf("%-19s", defaults[arg, ]), "\n", sep = "")
}
#> header FALSE TRUE TRUE TRUE TRUE
#> sep "" "," ";" "\t" "\t"
#> quote "\"'" "\"" "\"" "\"" "\""
#> dec "." "." "," "." ","
#> fill !blank.lines.skip TRUE TRUE TRUE TRUE
#> comment.char "#" "" "" "" ""
# read.csv() and its relatives are read.table() with other defaults
body(read.csv)
#> read.table(file = file, header = header, sep = sep, quote = quote,
#> dec = dec, fill = fill, comment.char = comment.char, ...)Source: R documentation: read.table
The table is worth reading column by column. read.table() expects white-space-separated fields (sep = ""), accepts
both kinds of quote, treats # as a comment character, and does not insist on a header; its fill default is the
opposite of blank.lines.skip, so with the default skipping of blank lines it does not pad short rows. The four
convenience functions assume that the first line holds the names (header = TRUE), pad short rows (fill = TRUE), and
switch comments off (comment.char = ""). read.csv() and read.delim() use the point as the decimal mark, and
read.csv2() and read.delim2() use the comma, together with the semicolon as the field separator for the CSV variant.
The body() of read.csv() shows that nothing else is going on: it is a call to read.table().
| Function | sep |
dec |
quote |
Typical source |
|---|---|---|---|---|
|
white space |
|
|
text tables written by R or by hand |
|
|
|
|
CSV from English-speaking locales, |
|
|
|
|
CSV from spreadsheets in locales with a decimal comma |
|
tab |
|
|
tab-separated files, which are the safest interchange text format |
|
tab |
|
|
tab-separated files with a decimal comma |
Source: R documentation: read.table
Reading books.csv
The shelf file books.csv is read with no arguments at all, and str() shows what the guessing made of each column.
books <- read.csv("books.csv")
str(books)
#> 'data.frame': 16 obs. of 6 variables:
#> $ title : chr "The Dispossessed" "Pride and Prejudice" " the quiet streets " "Loud Roofs" ...
#> $ author: chr "Ursula K. Le Guin" "Jane Austen" "Mara Quill" "Mara Quill" ...
#> $ isbn : chr "978-0-306-40615-7" "978-0-14-143951-8" "9780001000100" "978-0-00-200029-1" ...
#> $ year : int 1974 1813 2019 2021 2016 2008 NA 1999 2012 2023 ...
#> $ genre : chr "Science fiction" "Classic" "Fiction" "fiction" ...
#> $ pages : int 387 432 214 198 301 NA 96 280 352 256 ...
# Empty numeric fields became NA; the empty text field is still ""
colSums(is.na(books))
#> title author isbn year genre pages
#> 0 0 0 1 0 2
books[c(7L, 9L), c("title", "isbn", "year", "pages")]
#> title isbn year pages
#> 7 Garden of Small Fires 978-0-00-500057-1 NA 96
#> 9 Rivers of Glass 2012 352
sum(books$isbn == "")
#> [1] 1Source: R documentation: read.table
The result follows the rules that the help page gives. Unless colClasses says otherwise, every column is read as
text and then converted by type.convert() to logical, integer, numeric or complex, whichever is the first that accepts
every non-missing value; a column that none accepts stays character, because the default of stringsAsFactors has been
FALSE since R 4.0.0 (the section on column types below has the details and the NEWS line). year and pages are
integer. The two blank page counts and the blank year are NA, because the help page says that blank fields in logical,
integer, numeric and complex columns are missing values. The blank isbn of "Rivers of Glass" is not NA: in a text
column a blank field is the empty string, and only the text NA (the default of na.strings) is missing. The same
distinction shows up in every later example.
The arguments that describe the file’s shape
The help page lists a few arguments that change what a line means. A script with a small export that has a preamble is the clearest way to see them together.
report <- c(
"Library export, produced by the circulation desk",
"Counts are as of the end of the month",
"loan_id,loan date,pages (est.),member",
"L001,2025-01-14,214,Hugo Silva",
"L002,2025-01-16,387,Ben Okafor",
"L003,2025-01-21,432,Dev Patel"
)
path <- tempfile(fileext = ".csv")
writeLines(report, path)
# skip= drops the preamble; check.names = TRUE (the default) repairs the names
read.csv(path, skip = 2L)
#> loan_id loan.date pages..est.. member
#> 1 L001 2025-01-14 214 Hugo Silva
#> 2 L002 2025-01-16 387 Ben Okafor
#> 3 L003 2025-01-21 432 Dev Patel
names(read.csv(path, skip = 2L, check.names = FALSE))
#> [1] "loan_id" "loan date" "pages (est.)" "member"
# nrows= reads only the first records
read.csv(path, skip = 2L, nrows = 2L)
#> loan_id loan.date pages..est.. member
#> 1 L001 2025-01-14 214 Hugo Silva
#> 2 L002 2025-01-16 387 Ben Okafor
# header = FALSE keeps the first line as data and numbers the columns V1, V2
read.csv(path, skip = 3L, header = FALSE)
#> V1 V2 V3 V4
#> 1 L001 2025-01-14 214 Hugo Silva
#> 2 L002 2025-01-16 387 Ben Okafor
#> 3 L003 2025-01-21 432 Dev Patel
# col.names= supplies the names; with header = TRUE the file's names are dropped
read.csv(path, skip = 2L, col.names = c("id", "since", "pages", "who"))
#> id since pages who
#> 1 L001 2025-01-14 214 Hugo Silva
#> 2 L002 2025-01-16 387 Ben Okafor
#> 3 L003 2025-01-21 432 Dev Patel
# row.names= takes a column (by number or name) as the row names
read.csv(path, skip = 2L, row.names = 1L)
#> loan.date pages..est.. member
#> L001 2025-01-14 214 Hugo Silva
#> L002 2025-01-16 387 Ben Okafor
#> L003 2025-01-21 432 Dev Patel
# Without a header, header = TRUE is a guess that read.csv() does not make
unlink(path)Six rules are visible in the output. skip counts lines of the file before the header is looked for, which is how a
preamble is dropped. With check.names = TRUE, the default, the names loan date and pages (est.) become the
syntactically valid loan.date and pages..est..; this is make.names() at work, and R FAQ 7.14 (What are valid
names?) is the reference for why R does it
(R FAQ 7.14: What are valid names?).
check.names = FALSE keeps the original names, which then need backticks in a formula or $ access.
nrows reads only the first records; the help page recommends a mild over-estimate of the true number when it is used
for speed. header = FALSE makes the header line ordinary data and numbers the columns V1, V2 and so on. col.names
overrides the names that the header supplies. row.names = 1L makes the first column the row names, which is how a
file written by write.csv() with its default row names is read back.
The manual recommends stating header explicitly instead of relying on the guess. The help page says that when the
argument is missing, read.table() sets it to TRUE if and only if the first row has one field fewer than the number
of columns, which is the shape of a table printed by R, with row names but no name for them. The convenience functions
do not guess; they fix header = TRUE.
A last argument in this family is text, which replaces file with a character string. It is the way to put a small
table in a script, and many examples of this page use it for the awkward cases that the shelf files do not contain.
# text= reads a character string as if it were a file
loans_small <- read.csv(text = "
loan_id,member,days
L001,Hugo Silva,21
L002,Ben Okafor,10
")
loans_small
#> loan_id member days
#> 1 L001 Hugo Silva 21
#> 2 L002 Ben Okafor 10
# Leading and trailing blank lines are trimmed; the lines may be a vector
read.table(
text = c("title pages", "Dune 412", "Emma 474"),
header = TRUE
)
#> title pages
#> 1 Dune 412
#> 2 Emma 474
# A raw string keeps backslashes and quotes without escaping (R 4.0.0)
read.csv(text = r"(note,path
"say ""hi""",C:\data\books.csv)")
#> note path
#> 1 say "hi" C:\\data\\books.csv
# The same idea for numbers and text vectors
scan(text = "3 1 4 1 5", quiet = TRUE)
#> [1] 3 1 4 1 5
readLines(textConnection("alpha\nbeta"))
#> [1] "alpha" "beta"Source: R documentation: read.table
The first call shows that blank lines at the start and end of the string are trimmed. The vector form joins its elements
as lines. The raw string (an R 4.0.0 syntax, written with r"( and )") lets the CSV quotes and a Windows path appear
without a single escape; the backslashes in the printed data frame are doubled only because the console shows string
values in escaped form.
Separators and quotes
CSV is a convention more than a standard (RFC 4180 describes the common form, with a comma separator, double quotes and
doubled quotes inside quoted fields, and CRLF line endings: RFC 4180), and R
reads the common form. The relevant arguments are sep, quote and allowEscapes. When sep is given, a quoted field
may contain the separator and a doubled quote stands for one quote; when sep is the default white space, a backslash in
a quoted string escapes the next character instead. The manual says the same
(R Data Import/Export: Variations on read.table).
# Quoted fields can hold the separator, a doubled quote and a line break
csv <- c(
"title,note",
"\"Salt, Fire and Ice\",\"Said to be \"\"the\"\" best\"",
"\"Two\nLines\",plain"
)
path <- tempfile(fileext = ".csv")
writeLines(csv, path)
result <- read.csv(path)
result
#> title note
#> 1 Salt, Fire and Ice Said to be "the" best
#> 2 Two\nLines plain
nchar(result$title)
#> [1] 18 9
result$note[[1L]]
#> [1] "Said to be \"the\" best"
# read.csv() quotes only with the double quote, so an apostrophe is safe
read.csv(text = "name,place\nO'Brien,Cork")
#> name place
#> 1 O'Brien Cork
# read.table() also accepts the single quote, but only at the start of a field
read.table(text = "name place\nO'Brien Cork", header = TRUE)
#> name place
#> 1 O'Brien Cork
# The default separator of read.table() is any run of white space
read.table(text = "title pages\nDune 412\nEmma 474", header = TRUE)
#> title pages
#> 1 Dune 412
#> 2 Emma 474
# sep = "\t" separates only on tabs, so blanks inside a field survive
tsv <- "title\tpages\nThe Long Table\t256\nPaper Moons\t88"
read.delim(text = tsv)
#> title pages
#> 1 The Long Table 256
#> 2 Paper Moons 88
# A backslash escapes a quote only when no separator is given
read.table(text = "x\n\"say \\\"hi\\\"\"", header = TRUE)
#> x
#> 1 say "hi"
unlink(path)A quoted field can therefore hold a comma, a doubled quote and a line break; the second title in the output contains a
real newline, and nchar() shows 9 characters for it. read.csv() recognizes only the double quote, so an apostrophe
in O'Brien is plain text; read.table() recognizes both quotes, but only at the start of a field, so the same name
passes there too. A name that starts with the apostrophe would open a string that never closes, and the section on
common errors shows what R says then. quote = "" turns quoting off.
The help page adds that quoting is considered only for columns read as character. It also says that quoted fields with
embedded newlines are supported, except after a comment character. Embedded NUL bytes are not supported; skipNul = TRUE
may help.
The delimiter choice has its own pitfalls. A comma is a poor separator for data that contains commas, and in countries
that write the decimal comma it is not the separator at all. The manual says that a semicolon or a tab is probably the
safest option when writing, and a tab-separated file read with read.delim() keeps the blanks inside fields, as the
output shows for "The Long Table". If a tab-delimited file has empty fields, say sep = "\t" explicitly: with the
default white space a run of tabs counts as one separator and the empty field disappears.
Missing values
na.strings is a character vector of the spellings of a missing value. Two rules matter. First, blank fields in logical,
integer, numeric and complex columns are always missing, whatever na.strings says; in a text column a blank field is
"" unless "" is listed. Second, the comparison happens after white space is stripped (only if stripping is enabled),
so a field written as a blank followed by NA is not missing unless strip.white = TRUE removes the blank first.
csv <- c(
"title,year,pages,note",
"A Map of Salt,2008,,n/a",
"Garden of Small Fires,NA,96,",
"Ten Winters,2022,-,checked"
)
# By default only the text NA is missing, and empty numeric fields too
str(read.csv(text = csv))
#> 'data.frame': 3 obs. of 4 variables:
#> $ title: chr "A Map of Salt" "Garden of Small Fires" "Ten Winters"
#> $ year : int 2008 NA 2022
#> $ pages: chr "" "96" "-"
#> $ note : chr "n/a" "" "checked"
# na.strings= lists every spelling of missing; a blank text field joins in
# only when "" is listed
str(read.csv(text = csv, na.strings = c("NA", "", "-", "n/a")))
#> 'data.frame': 3 obs. of 4 variables:
#> $ title: chr "A Map of Salt" "Garden of Small Fires" "Ten Winters"
#> $ year : int 2008 NA 2022
#> $ pages: int NA 96 NA
#> $ note : chr NA NA "checked"
# The test happens after strip.white, so " NA" needs strip.white = TRUE
padded <- "a,b\n1, NA\n2,3"
str(read.csv(text = padded))
#> 'data.frame': 2 obs. of 2 variables:
#> $ a: int 1 2
#> $ b: chr " NA" "3"
str(read.csv(text = padded, strip.white = TRUE))
#> 'data.frame': 2 obs. of 2 variables:
#> $ a: int 1 2
#> $ b: int NA 3
# Text columns keep "" unless "" is in na.strings
x <- read.csv(text = "id,name\n1,\n2,Ana")
x$name
#> [1] "" "Ana"
is.na(x$name)
#> [1] FALSE FALSE
# An all-NA column becomes logical
class(read.csv(text = "a,b\n1,\n2,")$b)
#> [1] "logical"
# NaN, Inf and -Inf are accepted in numeric columns
str(read.csv(text = "z\n1\nNaN\nInf\n-Inf"))
#> 'data.frame': 4 obs. of 1 variable:
#> $ z: num 1 NaN Inf -InfSource: R documentation: read.table
The first call shows how a single non-number spoils a numeric column. The pages column contains a dash, so the whole
column is text, and its blank is the empty string. Listing the spellings in na.strings fixes all three problems at
once: pages becomes an integer column with two NA, and the note column has two NA and one genuine text. The
padded example is the case with a leading blank. NaN, Inf and -Inf are accepted spellings in a numeric column,
as the last call shows.
The same rule explains why the year column of books.csv is integer although one cell is empty, and why isbn has a
"" and no NA. If the downstream code wants NA there, say so with na.strings = c("NA", "").
Column types: the guess, and how to override it
The help page describes the algorithm: read every column as text, then call type.convert() on it. The manual states the
order as logical, then integer, then numeric, then complex, moving on whenever an entry that is not missing cannot be
converted; if all of those fail the column is text, or a factor when factors were requested
(R Data Import/Export: Variations on read.table).
A column made only of TRUE, FALSE, T, F and missing values is logical, a column of whole numbers within the
range of an integer is integer, and everything else numeric is double. Dates are never guessed: 2025-01-14 is just
text.
csv <- c(
"flag,count,ratio,label,when,mixed",
"TRUE,1,0.5,a,2025-01-14,1",
"FALSE,2,1.5,b,2025-02-03,x",
"NA,3,2.0,c,2025-03-17,3"
)
# read.table() tries logical, integer, numeric, complex in turn, then text
str(read.csv(text = csv))
#> 'data.frame': 3 obs. of 6 variables:
#> $ flag : logi TRUE FALSE NA
#> $ count: int 1 2 3
#> $ ratio: num 0.5 1.5 2
#> $ label: chr "a" "b" "c"
#> $ when : chr "2025-01-14" "2025-02-03" "2025-03-17"
#> $ mixed: chr "1" "x" "3"
# Dates stay text unless colClasses says Date
str(read.csv(text = csv, colClasses = c(when = "Date")))
#> 'data.frame': 3 obs. of 6 variables:
#> $ flag : logi TRUE FALSE NA
#> $ count: int 1 2 3
#> $ ratio: num 0.5 1.5 2
#> $ label: chr "a" "b" "c"
#> $ when : Date, format: "2025-01-14" "2025-02-03" ...
#> $ mixed: chr "1" "x" "3"
# stringsAsFactors = TRUE converts the text columns to factors (default FALSE)
str(read.csv(text = csv, stringsAsFactors = TRUE))
#> 'data.frame': 3 obs. of 6 variables:
#> $ flag : logi TRUE FALSE NA
#> $ count: int 1 2 3
#> $ ratio: num 0.5 1.5 2
#> $ label: Factor w/ 3 levels "a","b","c": 1 2 3
#> $ when : Factor w/ 3 levels "2025-01-14","2025-02-03",..: 1 2 3
#> $ mixed: Factor w/ 3 levels "1","3","x": 1 3 2
# as.is names the columns that must not become factors
str(read.csv(text = csv, stringsAsFactors = TRUE, as.is = "label"))
#> 'data.frame': 3 obs. of 6 variables:
#> $ flag : logi TRUE FALSE NA
#> $ count: int 1 2 3
#> $ ratio: num 0.5 1.5 2
#> $ label: chr "a" "b" "c"
#> $ when : Factor w/ 3 levels "2025-01-14","2025-02-03",..: 1 2 3
#> $ mixed: Factor w/ 3 levels "1","3","x": 1 3 2
# colClasses = "factor" is the direct way to ask for one factor column
str(read.csv(text = csv, colClasses = c(label = "factor")))
#> 'data.frame': 3 obs. of 6 variables:
#> $ flag : logi TRUE FALSE NA
#> $ count: int 1 2 3
#> $ ratio: num 0.5 1.5 2
#> $ label: Factor w/ 3 levels "a","b","c": 1 2 3
#> $ when : chr "2025-01-14" "2025-02-03" "2025-03-17"
#> $ mixed: chr "1" "x" "3"
# T and F alone are logical, which can be a surprise in a code column
str(read.csv(text = "grade\nF\nT\nF"))
#> 'data.frame': 3 obs. of 1 variable:
#> $ grade: logi FALSE TRUE FALSE
str(read.csv(text = "grade\nF\nT\nF", tryLogical = FALSE))
#> 'data.frame': 3 obs. of 1 variable:
#> $ grade: chr "F" "T" "F"
# type.convert() is the function read.table() calls on every column
type.convert(c("1", "2", "3"), as.is = TRUE)
#> [1] 1 2 3
type.convert(c("1", "2.5", "x"), as.is = TRUE)
#> [1] "1" "2.5" "x"
type.convert(c("1", "2", NA), as.is = TRUE)
#> [1] 1 2 NA
type.convert(c("a", "b", "a"), as.is = FALSE)
#> [1] a b a
#> Levels: a bSource: R documentation: read.table
Each call in the output makes one point. The default read gives the guessed types, and the mixed column, whose second value
is x, stays text. colClasses = c(when = "Date") converts a named column and leaves the others to the guess.
stringsAsFactors = TRUE converts every text column that the guess leaves as text into a factor (that is why label,
when and mixed all become factors), as.is = "label" takes one column back out, and colClasses = c(label = "factor") asks for
exactly one factor column without touching the others. The last two calls show tryLogical: a column consisting only of
F and T is read as logical by default, which is a classic surprise for a column of grade codes or of single-letter
flags. tryLogical = FALSE (an argument of read.table() and of type.convert()) keeps the letters as text. The
argument was added in R 4.3.0, with the old behaviour as its default.
colClasses
colClasses is the argument to know. It is a character vector of classes, one per column; if it is unnamed it is
recycled to the number of columns, and if it is named, the names pick columns and the others are guessed. The allowed
values are NA (guess; the default), "NULL" (skip the column), one of the atomic classes ("logical", "integer",
"numeric", "complex", "character", "raw"), "factor", "Date", "POSIXct", or any class for which a
methods::as() conversion from character exists. The help page notes that the argument is per column, not per variable,
so it also counts a row-names column if the file has one. The R FAQ gives the performance reason as well: stating the
classes avoids the guessing and is the first thing to do with a big file
(R FAQ 7.28: Why is read.table() so inefficient?).
# Identifiers that look like numbers are read as numbers
csv <- c(
"code,isbn10,pages",
"007,0306406152,320",
"010,080442957X,288"
)
read.csv(text = csv)
#> code isbn10 pages
#> 1 7 0306406152 320
#> 2 10 080442957X 288
str(read.csv(text = csv))
#> 'data.frame': 2 obs. of 3 variables:
#> $ code : int 7 10
#> $ isbn10: chr "0306406152" "080442957X"
#> $ pages : int 320 288
# Without the X, the whole ISBN column would have been numbers too
str(read.csv(text = "isbn10\n0306406152\n0804429575"))
#> 'data.frame': 2 obs. of 1 variable:
#> $ isbn10: int 306406152 804429575
# colClasses = "character" keeps every field exactly as written
str(read.csv(text = csv, colClasses = "character"))
#> 'data.frame': 2 obs. of 3 variables:
#> $ code : chr "007" "010"
#> $ isbn10: chr "0306406152" "080442957X"
#> $ pages : chr "320" "288"
# Name only the columns that must stay text; the others are still converted
str(read.csv(
text = csv,
colClasses = c(code = "character", isbn10 = "character")
))
#> 'data.frame': 2 obs. of 3 variables:
#> $ code : chr "007" "010"
#> $ isbn10: chr "0306406152" "080442957X"
#> $ pages : int 320 288
# A positional vector is matched to the columns in order; NA means "guess"
str(read.csv(text = csv, colClasses = c("character", "character", NA)))
#> 'data.frame': 2 obs. of 3 variables:
#> $ code : chr "007" "010"
#> $ isbn10: chr "0306406152" "080442957X"
#> $ pages : int 320 288
# A column that cannot be converted is an error, not a quiet NA
tryCatch(
read.csv(text = csv, colClasses = c(NA, "integer", NA)),
error = conditionMessage
)
#> [1] "scan() expected 'an integer', got '080442957X'"
# "NULL" skips a column altogether
str(read.csv(text = csv, colClasses = c("NULL", NA, NA)))
#> 'data.frame': 2 obs. of 2 variables:
#> $ isbn10: chr "0306406152" "080442957X"
#> $ pages : int 320 288Source: R documentation: read.table
The first call shows why identifiers must not be left to the guess: 007 and 010 come back as the integers 7 and 10,
the zero is gone for good, and the file can no longer be reproduced from the data frame. The ISBN column survives in
that file only because one of its values ends in X, which is not a digit; without the X the whole column would have
been integer too, as the next call shows. colClasses = "character" is the blunt remedy that keeps every field exactly as
written, a vector naming only the identifier columns is the precise one, and a conversion to the wrong type is an error,
not a quiet NA: scan() expected 'an integer', got '080442957X'. "NULL" drops a column before it is stored.
The shelf scenario has the same problem in a milder form. In loans.csv the ISBNs are 13 digits and the guess turns them
into doubles.
loans_plain <- read.csv("loans.csv")
vapply(loans_plain, \(x) class(x)[[1L]], character(1))
#> loan_id isbn member loan_date return_date
#> "character" "numeric" "character" "character" "character"
# The ISBN column became numbers: it prints in scientific notation, and a
# leading zero or an X check digit would not have survived
head(loans_plain$isbn, 3L)
#> [1] 9.780141e+12 9.780306e+12 9.780002e+12
format(head(loans_plain$isbn, 3L), scientific = FALSE)
#> [1] "9780141439518" "9780306406157" "9780002000291"
# An unreturned loan has an empty return_date, which stays "" in a text column
tail(loans_plain$return_date, 3L)
#> [1] "" "" ""Source: R documentation: read.table
A 13-digit ISBN fits in a double without loss, so format(scientific = FALSE) brings the digits back, but it prints as
9.780141e+12 by default, an ISBN with a leading zero or an X would not survive, and the dates stay text. The reading
used by the scenario, and by the other pages of this section, is therefore colClasses = "character".
loans <- read.csv("loans.csv", colClasses = "character")
str(loans)
#> 'data.frame': 60 obs. of 5 variables:
#> $ loan_id : chr "L001" "L002" "L003" "L004" ...
#> $ isbn : chr "9780141439518" "9780306406157" "9780002000291" "9780005000571" ...
#> $ member : chr "Hugo Silva" "Ben Okafor" "Dev Patel" "Eva Lindqvist" ...
#> $ loan_date : chr "2025-01-14" "2025-01-16" "2025-01-21" "2025-01-26" ...
#> $ return_date: chr "2025-02-04" "2025-01-26" "2025-01-22" "2025-02-21" ...
# Unreturned loans are empty strings until the dates are converted
sum(loans$return_date == "")
#> [1] 7
loans$loan_date <- as.Date(loans$loan_date, format = "%Y-%m-%d")
loans$return_date <- as.Date(loans$return_date, format = "%Y-%m-%d")
sum(is.na(loans$loan_date))
#> [1] 1
sum(is.na(loans$return_date))
#> [1] 7
# The impossible date 2025-02-30 is the one NA in loan_date
loans[is.na(loans$loan_date), c("loan_id", "member", "return_date")]
#> loan_id member return_date
#> 20 L020 Ana Ruiz 2025-07-15
# The same result in one step: name the class of each column
loans_typed <- read.csv(
"loans.csv",
colClasses = c(
loan_id = "character", isbn = "character", member = "character",
loan_date = "Date", return_date = "Date"
)
)
identical(loans_typed, loans)
#> [1] TRUESource: R documentation: read.table
Two details in this output matter later. The seven unreturned loans have an empty return_date, which is the empty
string in a text column, and as.Date() turns "" into NA, so the conversion produces the seven NA that mean "still
out". The impossible date 2025-02-30 becomes an NA in loan_date too, with no warning, which the cleaning code of
the data-frames page then has to find. The last call shows the same result in one step by naming a class for every
column; colClasses = "Date" calls as.Date() on the text, so it understands only the ISO form, and the next section
shows what happens with any other form.
Numbers and dates that turn into text
The most common complaint about read.csv() is a numeric column that arrives as text, nearly always because one entry
is not a number. Older tutorials and the R FAQ describe the same symptom as a factor, because before R 4.0.0 the default
turned all text into factors (see the version notes below).
csv <- "title,pages\nA Map of Salt,352\nLoud Roofs,n/a\nTen Winters,256"
# One stray word and the whole column is text, not a numeric column
pages <- read.csv(text = csv)$pages
class(pages)
#> [1] "character"
tryCatch(mean(pages), warning = conditionMessage)
#> [1] "argument is not numeric or logical: returning NA"
tryCatch(as.numeric(pages), warning = conditionMessage)
#> [1] "NAs introduced by coercion"
# Tell read.csv() what the word means, and the column is numeric again
pages <- read.csv(text = csv, na.strings = c("NA", "n/a"))$pages
class(pages)
#> [1] "integer"
mean(pages, na.rm = TRUE)
#> [1] 304
# With stringsAsFactors = TRUE the same column is a factor, and as.numeric()
# returns the codes of the levels (R FAQ 7.10)
as_factor <- read.csv(text = csv, stringsAsFactors = TRUE)$pages
as_factor
#> [1] 352 n/a 256
#> Levels: 256 352 n/a
as.numeric(as_factor)
#> [1] 2 3 1
suppressWarnings(as.numeric(as.character(as_factor)))
#> [1] 352 NA 256
suppressWarnings(as.numeric(levels(as_factor))[as_factor])
#> [1] 352 NA 256
# Dates that are not ISO 8601 are not understood by colClasses = "Date":
# as.Date() tries "%Y-%m-%d", then "%Y/%m/%d", and "14/01/2025" fits the
# second one as year 14, month 01, day 20, without any warning
csv_dates <- "id,when\n1,14/01/2025\n2,03/02/2025"
wrong <- read.csv(text = csv_dates, colClasses = c(when = "Date"))
class(wrong$when)
#> [1] "Date"
format(wrong$when)
#> [1] "14-01-20" "3-02-20"
# Read the column as text and give the format
dates <- read.csv(text = csv_dates)
as.Date(dates$when, format = "%d/%m/%Y")
#> [1] "2025-01-14" "2025-02-03"The first calls show the chain of consequences in a character column: mean() returns NA with a warning, and
as.numeric() produces NA with the warning NAs introduced by coercion. The remedy is in the reading, with
na.strings. The factor case is the one the R FAQ warns about: as.numeric() of a factor returns the integer codes of
the levels (here 2, 3, 1), never the numbers that the labels show, and the FAQ gives the two safe conversions,
as.numeric(as.character(f)) and the faster as.numeric(levels(f))[f]. The last part is a trap that does not warn.
colClasses = "Date" is not a date-format argument. as.Date() tries %Y-%m-%d and then %Y/%m/%d, and the text
14/01/2025 fits the second pattern as year 14, month 01, day 20, so a European date comes back as a valid Date in the
year 14 and no message at all. Read such a column as text and give the format
(R documentation: as.Date); the
Dates and Times page has the format codes.
stringsAsFactors and as.is
stringsAsFactors controls whether text columns become factors. R 4.0.0 changed its default from TRUE to FALSE in
data.frame() and read.table(). The NEWS entry says that R now uses a stringsAsFactors = FALSE default and hence by
default no longer converts strings to factors in calls to data.frame() and read.table(), and that a large number of
packages relied on the previous behaviour and needed updating
(R NEWS: Changes in R 4.0.0). The reason is in the examples
above: a factor is a poor default for free text, an invitation to the as.numeric() trap, and a quiet source of bugs when
two data sets with different levels are combined. The arguments interact as follows.
| Argument | Effect on a text column |
|---|---|
|
stays character |
|
becomes a factor, with levels sorted in the locale of the session (but |
|
overrides |
|
overrides both, and also stops the guessing of numbers and logicals |
|
the same switch for a vector already in memory |
Source: R documentation: read.table
Factors are the right type for a column that really is a set of categories, and the Factors
page covers building them from text deliberately, with factor(levels = ), instead of by the accident of an import.
Comments, blank lines and short rows
The three arguments comment.char, blank.lines.skip and fill decide what a line that is not a plain record means.
read.table() treats # as the start of a comment, and so the rest of the line, even inside a data field, is
discarded; the help page adds that quotes are not processed in comments and that comment lines before the header must
have the comment character in the first non-blank column. The four convenience functions turn comments off with
comment.char = "", and the manual advises setting it to the empty string whenever the data cannot contain comments,
because it is both safer and faster.
lines <- c(
"# exported from the circulation desk",
"title,rating",
"Loud Roofs #2,4",
"Ten Winters,5 # re-read"
)
# read.csv() has no comment character, so "#" is ordinary text: the comment
# line becomes the header, and since the data lines have one field more, the
# first field of each line turns into a row name
read.csv(text = lines)
#> X..exported.from.the.circulation.desk
#> title rating
#> Loud Roofs #2 4
#> Ten Winters 5 # re-read
# skip= removes the first line and the rest is read as it stands
read.csv(text = lines, skip = 1L)
#> title rating
#> 1 Loud Roofs #2 4
#> 2 Ten Winters 5 # re-read
# comment.char = "#" cuts every line at the first "#": the title is shortened
read.csv(text = lines, comment.char = "#")
#> title rating
#> 1 Loud Roofs NA
#> 2 Ten Winters 5
# read.table() has comment.char = "#" by default, which here cuts the rows
# short, and then a row has fewer fields than the header
tryCatch(
read.table(text = lines, sep = ",", header = TRUE),
error = conditionMessage
)
#> [1] "line 1 did not have 2 elements"
# comment.char = "" switches comments off, which is also a little faster
read.table(text = lines, sep = ",", header = TRUE, comment.char = "",
skip = 1L)
#> title rating
#> 1 Loud Roofs #2 4
#> 2 Ten Winters 5 # re-readSource: R documentation: read.table
Four different results from the same four lines. read.csv() ignores comments, so the comment line becomes the
header, and since the data lines have one field more, the first field of each line becomes a row name; the result is a
data frame with a strange header and no error. skip = 1L removes the comment line and the data are read as they stand,
including the # inside the title and the trailing note, which is part of the last field. comment.char = "#" cuts
every line at the #, and Loud Roofs #2,4 loses everything after the hash, leaving a row with a missing rating.
read.table() with its default comment character cuts the lines in the same way and then reports line 1 did not have 2
elements, because the first data line is now one field short and fill is FALSE. Turning comments off and skipping
the preamble line reads the data as written.
Short and long rows are a separate matter.
# A spreadsheet export that dropped the trailing empty fields
ragged <- c(
"title,year,pages,note",
"Loud Roofs,2021,198",
"",
"Ten Winters,2022,,reread",
"Paper Moons,2005"
)
# read.csv() has fill = TRUE: short rows are padded, blank lines are skipped
read.csv(text = ragged)
#> title year pages note
#> 1 Loud Roofs 2021 198
#> 2 Ten Winters 2022 NA reread
#> 3 Paper Moons 2005 NA
# blank.lines.skip = FALSE turns the blank line into a row of NA
read.csv(text = ragged, blank.lines.skip = FALSE)
#> title year pages note
#> 1 Loud Roofs 2021 198
#> 2 NA NA
#> 3 Ten Winters 2022 NA reread
#> 4 Paper Moons 2005 NA
# read.table() has fill = FALSE (when blank lines are skipped): it refuses
tryCatch(
read.table(text = ragged, sep = ",", header = TRUE),
error = conditionMessage
)
#> [1] "line 1 did not have 4 elements"
read.table(text = ragged, sep = ",", header = TRUE, fill = TRUE)
#> title year pages note
#> 1 Loud Roofs 2021 198
#> 2 Ten Winters 2022 NA reread
#> 3 Paper Moons 2005 NA
# A row with too MANY fields is not an error with fill = TRUE:
# the surplus wraps round into the next record
wide <- c("a,b", "1,2", "3,4,5", "6,7")
read.csv(text = wide)
#> a b
#> 1 2 NA
#> 3 4 5
#> 6 7 NA
count.fields(textConnection(wide), sep = ",")
#> [1] 2 2 3 2fill = TRUE pads short rows with blank fields; it is the default of the convenience functions, because spreadsheet
exports commonly drop the trailing empty fields, as the manual notes. In the output the Loud Roofs row has no note and
the Paper Moons row has no pages and no note, and a blank line is skipped by default. With blank.lines.skip = FALSE
the blank line becomes a row of missing values. read.table() has fill = FALSE and refuses a short row. The last
two calls show a quieter hazard: a row with more fields than the first lines, read with fill = TRUE, is not an error.
The surplus wraps into a new record, so three lines become four rows and the 3,4,5 line turns into the records
3,4 and 5,NA. count.fields() reports 2 2 3 2, which is how this is found. The help page explains why: the number of
columns is taken from the first five lines of input, or from the length of col.names, and a row that does not fit
that count is filled or split; if the first five lines are not representative, give col.names.
Decimal marks, thousands separators and the spreadsheet dialect
Spreadsheet programs write the CSV that their locale prefers. In many Western European locales that means a semicolon
as the field separator and a comma as the decimal mark, because the comma is already taken. The manual describes
read.csv2() and write.csv2() for exactly this and calls the semicolon or the tab the safest separator.
# The spreadsheet convention in many European locales: semicolon and decimal
# comma. Read with the comma defaults, the file falls apart.
euro <- c("title;price;pages", "Loud Roofs;12,50;198", "Ten Winters;9,90;256")
read.csv(text = euro)
#> title.price.pages
#> Loud Roofs;12 50;198
#> Ten Winters;9 90;256
str(read.csv2(text = euro))
#> 'data.frame': 2 obs. of 3 variables:
#> $ title: chr "Loud Roofs" "Ten Winters"
#> $ price: num 12.5 9.9
#> $ pages: int 198 256
str(read.csv(text = euro, sep = ";", dec = ","))
#> 'data.frame': 2 obs. of 3 variables:
#> $ title: chr "Loud Roofs" "Ten Winters"
#> $ price: num 12.5 9.9
#> $ pages: int 198 256
# read.delim() is read.table() for tab-separated files
tabbed <- "title\tprice\nLoud Roofs\t12.50\nTen Winters\t9.90"
str(read.delim(text = tabbed))
#> 'data.frame': 2 obs. of 2 variables:
#> $ title: chr "Loud Roofs" "Ten Winters"
#> $ price: num 12.5 9.9
str(read.delim2(text = "title\tprice\nLoud Roofs\t12,50"))
#> 'data.frame': 1 obs. of 2 variables:
#> $ title: chr "Loud Roofs"
#> $ price: num 12.5
# The thousands separator is not understood: the column stays text
str(read.csv(text = "n\n\"1,234\"\n\"5,678\""))
#> 'data.frame': 2 obs. of 1 variable:
#> $ n: chr "1,234" "5,678"
as.numeric(gsub(",", "", read.csv(text = "n\n\"1,234\"\n\"5,678\"")$n))
#> [1] 1234 5678
# numerals= decides what happens to digits a double cannot hold
long <- "id\n12345678901234567890"
str(read.csv(text = long))
#> 'data.frame': 1 obs. of 1 variable:
#> $ id: num 1.23e+19
str(read.csv(text = long, numerals = "no.loss"))
#> 'data.frame': 1 obs. of 1 variable:
#> $ id: chr "12345678901234567890"
tryCatch(
read.csv(text = long, numerals = "warn.loss"),
warning = conditionMessage
)
#> [1] "accuracy loss in conversion from \"12345678901234567890\" to numeric"Source: R documentation: read.table
Read with the comma defaults, the semicolon file falls apart into a single column whose name is made from the whole
header line, and the values are one string per line; there is no error, which is why this is the first thing to suspect
when a file produces one wide column. read.csv2() and read.csv(sep = ";", dec = ",") give the same result with
numeric price. read.delim() and read.delim2() are the tab-separated pair. R reads no thousands separator: a
value written as "1,234" stays text, and gsub() before as.numeric() (or the package readr’s parse_number()) is
the remedy. The last block shows numerals, an argument that decides what happens to digits a double cannot hold: the
default "allow.loss" converts with some loss, "warn.loss" warns with accuracy loss in conversion and the
number, and "no.loss" leaves the field as text. An identifier of twenty digits belongs in a character column.
Encodings and the byte order mark
A CSV file has no field for its encoding. The reading functions have two arguments that look alike and do opposite jobs,
and the help page spells out the difference: fileEncoding re-encodes the input to the encoding of the session through
a connection, and encoding only marks the strings as Latin-1 or UTF-8 without converting them. For a Latin-1 file in a
UTF-8 session, which is the commonest case, the manual’s remedy is fileEncoding = "latin1"
(R Data Import/Export: Variations on read.table).
The full treatment, with iconv(), Encoding() and the bytes mark, is on
Encodings, Locales and Internationalization;
this section keeps to what a reader needs.
# A Latin-1 file: one byte (e9) for each accented letter
text <- "name,city\nJosé,Café\n"
path <- "latin1.csv"
writeBin(iconv(text, "UTF-8", "latin1", toRaw = TRUE)[[1L]], path)
file.size(path)
#> [1] 20
nchar(text, type = "bytes")
#> [1] 22
# Without fileEncoding the bytes are taken as UTF-8, which they are not
bad <- read.csv(path)
validUTF8(bad$name)
#> [1] FALSE
tryCatch(nchar(bad$name), error = conditionMessage)
#> [1] "invalid multibyte string, element 1"
# fileEncoding converts to the session encoding while reading
good <- read.csv(path, fileEncoding = "latin1")
good
#> name city
#> 1 José Café
validUTF8(good$name)
#> [1] TRUE
nchar(good$name)
#> [1] 4
# encoding only declares what the bytes are; the strings stay Latin-1 inside
declared <- read.csv(path, encoding = "latin1")
Encoding(declared$name)
#> [1] "latin1"
declared$name == good$name
#> [1] TRUE
# A file saved by Excel as "CSV UTF-8" starts with the three bytes of a BOM
bom <- c(as.raw(c(0xef, 0xbb, 0xbf)), charToRaw("title,year\nA,2000\n"))
writeBin(bom, path)
names(read.csv(path))
#> [1] "title" "year"
# The same bytes read as Latin-1 show the mark as letters in the first name
names(read.csv(path, fileEncoding = "latin1"))
#> [1] "ï..title" "year"
names(read.csv(path, fileEncoding = "UTF-8-BOM"))
#> [1] "title" "year"
unlink(path)Source: R Data Import/Export: Encodings
The Latin-1 file is two bytes shorter than its UTF-8 text because each accented letter is one byte instead of two. Read
with the session’s encoding, the strings are not valid UTF-8, nothing fails at the time of reading, and the failure comes
later, in the first function that needs characters: nchar() stops with invalid multibyte string, element 1. With
fileEncoding = "latin1" the text is converted while it is read and is valid. With encoding = "latin1" the strings
are marked as Latin-1, which R converts when needed, so they compare equal to the converted ones, but the bytes are
not changed in memory.
A byte order mark (BOM) is the character U+FEFF at the start of a file, and Microsoft programs write it at the start
of "CSV UTF-8" files. Since R 3.0.0, readLines() and scan() (and hence read.table()) discard a UTF-8 BOM in a UTF-8
locale, so the first name is title, and the connection encoding name "UTF-8-BOM" discards it in any locale
(R NEWS: Changes in R 3.0.0). The third result shows what
the mark looks like when it is not recognized, which is the origin of the many questions about a first column name that
starts with ï..: the three bytes read as Latin-1 become the letters ï, » and ¿, and make.names() turns the last
two into dots. Names such as utf8 are not valid encoding names (UTF-8 is), and "UTF-16LE" is the one for the
"Unicode" text files of Windows.
Large files
Everything above is about getting the right result; for a big file the same arguments get it faster and in less memory.
The help page gives the list, and the R FAQ and the manual repeat it: give colClasses (as one of the six atomic classes
if the data allow it, since a column of many distinct numbers stored as strings can take up to 14 times the memory of
integers), give nrows (a mild over-estimate is better than nothing), set comment.char = "", and use scan() for large
matrices because read.table() is built for data frames with columns of different classes.
source("scenario.R")
# A bigger file: the 60 loans repeated 500 times (30 000 rows)
big <- loans[rep(seq_len(nrow(loans)), times = 500L), ]
big_path <- tempfile(fileext = ".csv")
write.csv(big, big_path, row.names = FALSE, na = "")
length(readLines(big_path))
#> [1] 30001
# Plain read: every column is read as text first and then converted
plain <- read.csv(big_path, colClasses = "character")
dim(plain)
#> [1] 30000 5
# Tuned read: the classes stated, a row count a little above the truth, and
# no comment scanning; the result is the same, with less work and memory
tuned <- read.csv(
big_path,
colClasses = c(
loan_id = "character", isbn = "character", member = "character",
loan_date = "Date", return_date = "Date"
),
nrows = 31000L,
comment.char = ""
)
dim(tuned)
#> [1] 30000 5
all.equal(tuned$loan_date, big$loan_date, check.attributes = FALSE)
#> [1] TRUE
# Columns that are not needed are skipped with "NULL"
slim <- read.csv(
big_path,
colClasses = c("character", "NULL", "NULL", "Date", "NULL")
)
names(slim)
#> [1] "loan_id" "loan_date"
# A chunk at a time from an open connection, for a file larger than memory
con <- file(big_path, "r")
header <- readLines(con, n = 1L)
chunk_rows <- integer()
repeat {
chunk <- readLines(con, n = 10000L)
if (length(chunk) == 0L) {
break
}
chunk_rows <- c(chunk_rows, length(chunk))
}
close(con)
header
#> [1] "\"loan_id\",\"isbn\",\"member\",\"loan_date\",\"return_date\""
chunk_rows
#> [1] 10000 10000 10000
# Compressed files are read as they are
gz_path <- tempfile(fileext = ".csv.gz")
gz <- gzfile(gz_path, "w")
write.csv(loans[1:3, ], gz, row.names = FALSE)
close(gz)
read.csv(gz_path, colClasses = "character")
#> loan_id isbn member loan_date return_date
#> 1 L001 9780141439518 Hugo Silva 2025-01-14 2025-02-04
#> 2 L002 9780306406157 Ben Okafor 2025-01-16 2025-01-26
#> 3 L003 9780002000291 Dev Patel 2025-01-21 2025-01-22
unlink(c(big_path, gz_path))Source: R documentation: read.table
The 30 000-row file is small enough to run in the time of a page build, but the options are the ones used for 30 million
rows. The plain read stores every field as text. The tuned read states the classes, caps the rows a little above the
truth, and switches off comment scanning; the dates come back as Date directly. "NULL" in colClasses skips a column
before it is stored, which saves memory when a file has columns you do not need. The loop reads a file in chunks from an
open connection, which is the pattern for a file larger than memory: each call of readLines(con, n) continues where
the last stopped, and an empty result means the end. The last call shows that a compressed file is read as it is:
file() checks a file that is opened for text-mode reading for compression and decompresses it (the raw = TRUE argument
suppresses the check; the details are on the
Connections and the File System page). For files of
hundreds of megabytes the add-on packages data.table (fread()) and vroom, and the Arrow tools, are faster than any
setting of read.table(); the section on those packages says how they differ.
Common errors and warnings when reading
A badly described file nearly always produces one of a few messages. They are collected here, with the text R prints, in the order of the script below; a warning, when there is one, is printed before the error that follows it.
# Report what an expression does: its warnings, then its error or its value
outcome <- function(expr) {
warnings <- character()
result <- withCallingHandlers(
tryCatch(
{
expr
"no error"
},
error = \(e) paste("error:", conditionMessage(e))
),
warning = \(w) {
warnings <<- c(warnings, paste("warning:", conditionMessage(w)))
invokeRestart("muffleWarning")
}
)
writeLines(c(warnings, result))
}
# 1. A header shorter than the data
writeLines(c("a", "1 2 3", "4 5 6"), "draft.txt")
outcome(read.table("draft.txt", header = TRUE))
#> error: more columns than column names
# 2. A row shorter than the others, when fill is FALSE
writeLines(c("a b c", "1 2", "4 5 6"), "draft.txt")
outcome(read.table("draft.txt", header = TRUE))
#> error: line 1 did not have 3 elements
# 3. A file that does not exist: a warning, then the error
outcome(read.csv("no-such-file.csv"))
#> warning: cannot open file 'no-such-file.csv': No such file or directory
#> error: cannot open the connection
# 4. A file without any line
outcome(read.csv(text = ""))
#> error: no lines available in input
# 5. A last line without a newline
cat("a,b\n1,2\n3,4", file = "draft.txt")
outcome(read.csv("draft.txt"))
#> warning: incomplete final line found by readTableHeader on 'draft.txt'
#> no error
outcome(readLines("draft.txt"))
#> warning: incomplete final line found on 'draft.txt'
#> no error
# 6. A quote that is never closed
writeLines(c("title,note", sprintf("T%d,fine", 1:7), "B,\"open", "C,x"),
"draft.txt")
outcome(read.csv("draft.txt"))
#> warning: EOF within quoted string
#> no error
unterminated <- suppressWarnings(read.csv("draft.txt"))
c(data_lines = length(readLines("draft.txt")) - 1L,
rows_read = nrow(unterminated))
#> data_lines rows_read
#> 9 8
# 7. Text where a number was declared
outcome(read.csv(text = "a,b\n1,x", colClasses = c("integer", "numeric")))
#> error: scan() expected 'a real', got 'x'
# 8. A duplicated or missing row name
outcome(read.table(text = "a b\nx 1\nx 2", header = TRUE, row.names = 1))
#> error: duplicate 'row.names' are not allowed
outcome(read.csv(text = "a,b\n1,2", row.names = 3L))
#> error: subscript out of bounds
# 9. Latin-1 bytes in a UTF-8 session
writeBin(as.raw(c(0x6e, 0x0a, 0x63, 0x61, 0x66, 0xe9, 0x0a)), "draft.txt")
cafe <- read.csv("draft.txt")$n
outcome(nchar(cafe))
#> error: invalid multibyte string, element 1
outcome(toupper(cafe))
#> error: invalid multibyte string 1
outcome(grepl("f", cafe))
#> warning: unable to translate 'caf<e9>' to a wide string
#> warning: input string 1 is invalid
#> no error
# 10. More colClasses than columns
outcome(read.csv(text = "a,b\n1,2", colClasses = rep("integer", 3L)))
#> warning: cols = 2 != length(data) = 3
#> no error
unlink("draft.txt")The messages, with what they mean and what to do, are in this table.
| Message | What happened | What to do |
|---|---|---|
|
The first lines of the file have more fields than the header can account for. A header one field short is accepted (the first column becomes the row names), two or more short is this error. |
Check |
|
A row is shorter than the others and |
|
|
The path is wrong, usually the working directory. Both messages come from |
|
|
The file is empty, or the |
Check the file; |
|
The last line has no newline. The data are all read. |
Harmless; add the newline, or ignore. |
|
Same cause. |
|
|
A quote was opened and never closed, so the rest of the file became one field. Fewer rows than lines come back. |
Find the stray quote; set |
|
A column declared numeric (by |
Fix the data, add the text to |
|
The column chosen by |
Use |
|
|
Check the number of columns. |
|
The strings were read in the wrong encoding, and the first function that needs characters ( |
|
|
|
Match the length to the file. |
Source: R documentation: read.table
Four of these deserve a second look.
The line number. In line N did not have M elements, N counts the lines after the header, so the 41st line of the
file is line 40, and M is the width found in the first five data lines. A bad line beyond the first five is named
correctly, as the script in the next section shows. A bad line among the first five sets the width itself, and then the
first correct line is the one reported, as in the stray-comma file below, which blames its first line. In the first
script the short row really is the first data row, so line 1 is right there.
The two incomplete-line messages. They look alike and come from two functions. readLines() accepts the final line
and warns incomplete final line found on 'draft.txt'; read.table() looks at the header with an internal helper and
its warning says incomplete final line found by readTableHeader on 'draft.txt'. In both cases the data are read in full.
The help page of readLines() explains that the behaviour depends on whether the connection is blocking: for a
non-blocking text-mode connection the incomplete line is pushed back silently, for all others it is accepted with a
warning.
The unclosed quote. It does not stop the read. The warning EOF within quoted string is printed, and all the text from
the stray quote to the end of the file becomes the value of one field, so the data frame has fewer rows than the file
has lines. The script above shows the symptom: the warning with no error, and eight rows from nine data lines. Nothing in the
data frame itself says that anything was wrong; the warning is the only signal, which is why a
script should not run with warnings switched off.
Latin-1 bytes. Reading never fails on an encoding problem; use is what fails. This is the same lesson as in the
encodings section above, and validUTF8() on the result is the quick check.
count.fields() and where a file goes wrong
count.fields() returns the number of fields on each line without building a data frame. The help page says that it
used to be used by read.table() and is still useful for discovering why a read fails. Embedded line breaks in quoted
fields are handled consistently with scan(): the line where such a field starts is reported as NA, and the line where
the record ends has the count for the whole record.
# A file in which one record has a stray comma
lines <- c(
"loan_id,member,loan_date",
"L001,Hugo Silva,2025-01-14",
"L002,Ben Okafor,2025-01-16",
"L003,Dev Patel, the younger,2025-01-21",
"L004,Eva Lindqvist,2025-01-26"
)
writeLines(lines, "draft.csv")
# One number per line: the number of fields it has
fields <- count.fields("draft.csv", sep = ",")
fields
#> [1] 3 3 3 4 3
table(fields)
#> fields
#> 3 4
#> 4 1
which(fields != fields[[1L]])
#> [1] 4
lines[which(fields != fields[[1L]])]
#> [1] "L003,Dev Patel, the younger,2025-01-21"
# read.table() with fill = FALSE blames line 1: the width of 4 comes from the
# stray line, which is among the first five
tryCatch(
read.table("draft.csv", sep = ",", header = TRUE),
error = conditionMessage
)
#> [1] "line 1 did not have 4 elements"
# A bad line after the first five is named by its place among the data lines:
# file line 41 is data line 40
wide <- c("a,b,c", sprintf("%d,%d,%d", 1:60, 1:60, 1:60))
wide[41L] <- "40,40,40,40"
writeLines(wide, "draft.csv")
tryCatch(
read.table("draft.csv", sep = ",", header = TRUE),
error = conditionMessage
)
#> [1] "line 40 did not have 3 elements"
which(count.fields("draft.csv", sep = ",") != 3L)
#> [1] 41
# read.csv() has fill = TRUE, so the surplus field starts a 61st record
nrow(read.csv("draft.csv"))
#> [1] 61
# Quoted line breaks: the start line shows NA, the end line the whole count
writeLines(c("a,b", "\"x\ny\",1", "z,2"), "draft.csv")
count.fields("draft.csv", sep = ",")
#> [1] 2 NA 2 2
# Read a file whose width is unknown: take the widest line as the column count
writeLines(c("1", "2,3", "4,5,6"), "draft.csv")
widest <- max(count.fields("draft.csv", sep = ","))
read.csv("draft.csv", header = FALSE, col.names = paste0("V", seq_len(widest)))
#> V1 V2 V3
#> 1 1 NA NA
#> 2 2 3 NA
#> 3 4 5 6
unlink("draft.csv")Source: R documentation: count.fields
In the stray-comma file the third data line has four fields where the others have three; count.fields() finds it,
table() summarizes how many lines have each width, and indexing lines shows the offending text. read.table() with
fill = FALSE fails with line 1 did not have 4 elements because the width of the table was taken from that stray
line, so the first, correct line is the one that is "short". In the 61-line file the bad line is beyond the first five,
the error names data line 40 although count.fields() finds it at file line 41 (the header counts there), and
read.csv() with fill = TRUE makes the surplus field a 61st record instead of failing. The quoted-line-break file
reports NA for the start of the record and 2 for its end. The last example is the idiom from the help page for a file of unknown width: read the
widest line count and pass col.names of that length.
Writing: write.csv() and write.table()
write.table() converts its argument to a data frame if it is not one (a matrix is also accepted) and writes it to a file
or connection. write.csv() and write.csv2() are wrappers that fix the arguments needed for a valid CSV file. The
manual lists the decisions for the writer: precision, header line, separator, missing values, quoting and encoding
(R Data Import/Export: Export to text files).
They are all arguments.
| Argument | Default | Meaning |
|---|---|---|
|
|
a file name, or a connection open for writing |
|
|
add to the file instead of overwriting it; only for a file name |
|
|
quote character and factor columns (and the names); |
|
|
the field separator |
|
|
the end-of-line string; |
|
|
the text for a missing value |
|
|
the decimal mark |
|
|
write the row names, or give a character vector of names |
|
|
write the header; |
|
|
how to write a double quote inside a quoted field |
|
|
re-encode the text while writing |
Source: R documentation: write.table
write.csv() in practice
books <- read.csv("books.csv")
path <- tempfile(fileext = ".csv")
first <- books[1:3, c("title", "year", "pages")]
# write.csv() with its defaults: quoted text, row names in an unnamed column
write.csv(first, path)
writeLines(readLines(path))
#> "","title","year","pages"
#> "1","The Dispossessed",1974,387
#> "2","Pride and Prejudice",1813,432
#> "3"," the quiet streets ",2019,214
# row.names = FALSE is what almost every reader of the file expects
write.csv(first, path, row.names = FALSE)
writeLines(readLines(path))
#> "title","year","pages"
#> "The Dispossessed",1974,387
#> "Pride and Prejudice",1813,432
#> " the quiet streets ",2019,214
# file = "" (the default) prints to the console instead of a file
write.csv(first, row.names = FALSE)
#> "title","year","pages"
#> "The Dispossessed",1974,387
#> "Pride and Prejudice",1813,432
#> " the quiet streets ",2019,214
# quote = FALSE removes the quotes; a field with a comma would then break
write.csv(first, path, row.names = FALSE, quote = FALSE)
writeLines(readLines(path))
#> title,year,pages
#> The Dispossessed,1974,387
#> Pride and Prejudice,1813,432
#> the quiet streets ,2019,214
# na = "" writes empty fields, as the shelf files do; the default is NA
na_rows <- books[c(6L, 7L, 9L), c("title", "isbn", "year", "pages")]
write.csv(na_rows, path, row.names = FALSE)
writeLines(readLines(path))
#> "title","isbn","year","pages"
#> "A Map of Salt","9780004000404",2008,NA
#> "Garden of Small Fires","978-0-00-500057-1",NA,96
#> "Rivers of Glass","",2012,352
write.csv(na_rows, path, row.names = FALSE, na = "")
writeLines(readLines(path))
#> "title","isbn","year","pages"
#> "A Map of Salt","9780004000404",2008,
#> "Garden of Small Fires","978-0-00-500057-1",,96
#> "Rivers of Glass","",2012,352
# eol = "\r\n" gives the line endings RFC 4180 asks for
write.csv(first, path, row.names = FALSE, eol = "\r\n")
bytes <- readBin(path, "raw", n = file.size(path))
tail(bytes, 2L)
#> [1] 0d 0a
sum(bytes == as.raw(0x0d))
#> [1] 4
unlink(path)Source: R documentation: write.table
The first output is the default, and it is almost never what the next program expects: a header cell with an empty name
for the row names, and a column of row numbers "1", "2", "3". A data frame that has the default row names has
nothing to write there, so row.names = FALSE is the right choice in nearly every call (and the one used for the shelf
files). To read a file that has the row-name column, use read.csv(row.names = 1). file = "", the default, prints to
the console. quote = FALSE removes the quotes, and then a title that contains a comma would break the file. The
na argument controls how missing values are written: the default is the text NA, which read.csv() reads back as a
missing value in every column; na = "" writes an empty field, which the shelf files use, and which read.csv() reads
back as NA only in numeric columns; a text column then needs na.strings = "". The line endings default to "\n";
eol = "\r\n" gives the CRLF that RFC 4180 specifies, and the script confirms it with the last two bytes (0d 0a) and
four carriage returns for four lines. The help page says that write.csv() writes compliant files on Windows and that
other platforms need eol = "\r\n", because on Windows a text-mode connection converts "\n" itself.
write.table() and its relatives
books <- read.csv("books.csv")
path <- tempfile(fileext = ".txt")
first <- books[1:3, c("title", "year", "pages")]
# write.table() defaults: a space, quoted text, header without a row-name cell
write.table(first)
#> "title" "year" "pages"
#> "1" "The Dispossessed" 1974 387
#> "2" "Pride and Prejudice" 1813 432
#> "3" " the quiet streets " 2019 214
# col.names = NA writes an empty header cell for the row names, which is the
# convention for CSV files that a spreadsheet will open
write.table(first, sep = ",", col.names = NA)
#> "","title","year","pages"
#> "1","The Dispossessed",1974,387
#> "2","Pride and Prejudice",1813,432
#> "3"," the quiet streets ",2019,214
# sep, quote, row.names and col.names all have their own arguments
write.table(first, sep = "\t", quote = FALSE, row.names = FALSE)
#> title year pages
#> The Dispossessed 1974 387
#> Pride and Prejudice 1813 432
#> the quiet streets 2019 214
# append = TRUE adds to an existing file; the header would be written again,
# so the second call switches the header off
write.table(first[1L, ], path, sep = ",", row.names = FALSE)
write.table(first[2L, ], path, sep = ",", row.names = FALSE,
col.names = FALSE, append = TRUE)
writeLines(readLines(path))
#> "title","year","pages"
#> "The Dispossessed",1974,387
#> "Pride and Prejudice",1813,432
# A double quote inside a field: backslash by default, doubled for CSV
quoted <- data.frame(note = "said \"hello\"")
write.table(quoted, row.names = FALSE)
#> "note"
#> "said \"hello\""
write.table(quoted, row.names = FALSE, qmethod = "double")
#> "note"
#> "said ""hello"""
write.csv(quoted, row.names = FALSE)
#> "note"
#> "said ""hello"""
unlink(path)Source: R documentation: write.table
The defaults of write.table() are the space separator, quoted text and a header that has no cell for the row names,
which R itself reads back with the automatic header detection. col.names = NA is the CSV convention, and as the help
page notes it is the form to use when the file will be opened in a spreadsheet, which expects a header cell above the
row names. sep = "\t", quote = FALSE, row.names = FALSE writes tab-separated text. append = TRUE adds to an existing
file; the header would be written again, so it is switched off on the second call. The last three calls show the
difference between the two ways of escaping a double quote inside a field: a backslash by default, a doubled quote with
qmethod = "double" and in write.csv(). The manual calls the doubled form the one that spreadsheets use. It has a
consequence for reading: a backslash-escaped quote is understood only when no separator is given, and doubling is the rule when
one is, so a file written with qmethod = "double" must be read with its separator named, as the example on the help
page notes.
What write.csv() will not let you change
first <- read.csv("books.csv")[1:2, c("title", "year")]
# write.csv() fixes sep, dec, qmethod, col.names and append on purpose;
# asking for something else is a warning, and the request is ignored
captured <- character()
withCallingHandlers(
write.csv(first, sep = ";", row.names = FALSE),
warning = \(w) {
captured <<- c(captured, conditionMessage(w))
invokeRestart("muffleWarning")
}
)
#> "title","year"
#> "The Dispossessed",1974
#> "Pride and Prejudice",1813
captured
#> [1] "attempt to set 'sep' ignored"
captured <- character()
withCallingHandlers(
write.csv(first, col.names = FALSE, row.names = FALSE),
warning = \(w) {
captured <<- c(captured, conditionMessage(w))
invokeRestart("muffleWarning")
}
)
#> "title","year"
#> "The Dispossessed",1974
#> "Pride and Prejudice",1813
captured
#> [1] "attempt to set 'col.names' ignored"
# write.csv2() is the semicolon-and-decimal-comma variant
write.csv2(data.frame(title = "Loud Roofs", price = 12.5), row.names = FALSE)
#> "title";"price"
#> "Loud Roofs";12,5
# For any other separator, use write.table() directly
write.table(first, sep = "|", row.names = FALSE, quote = FALSE)
#> title|year
#> The Dispossessed|1974
#> Pride and Prejudice|1813Source: R documentation: write.table
The wrappers are deliberately inflexible, because their purpose is to guarantee a valid CSV file. An attempt to change
append, col.names, sep, dec or qmethod is ignored, with a warning. The warnings are attempt to set 'sep' ignored and attempt to set 'col.names' ignored, and
the output in both cases is the ordinary CSV. A script that wants a semicolon separator and the decimal point uses
write.table(sep = ";"), and a script that wants the full European convention uses write.csv2(), which sets the
semicolon and the decimal comma together.
What each column type becomes
# What write.csv() does to each kind of column
tbl <- data.frame(
id = c("007", "010"),
genre = factor(c("Poetry", "Mystery")),
borrowed = as.Date(c("2025-01-14", "2025-02-03")),
price = c(12.5, 1 / 3),
big = c(123456789.123, 1e-10),
flag = c(TRUE, NA)
)
write.csv(tbl, row.names = FALSE)
#> "id","genre","borrowed","price","big","flag"
#> "007","Poetry",2025-01-14,12.5,123456789.123,TRUE
#> "010","Mystery",2025-02-03,0.333333333333333,1e-10,NA
# Numbers go out with up to 15 significant digits
write.csv(data.frame(x = c(pi, 1e15, 1e5, 100000L)), row.names = FALSE)
#> "x"
#> 3.14159265358979
#> 1e+15
#> 1e+05
#> 1e+05
# scipen decides when a number switches to scientific notation
old <- options(scipen = 100)
write.csv(data.frame(x = c(1e15, 1e-10)), row.names = FALSE)
#> "x"
#> 1000000000000000
#> 0.0000000001
options(old)
# format() gives control over every column before writing
tbl$price <- sprintf("%.2f", tbl$price)
tbl$borrowed <- format(tbl$borrowed, "%d/%m/%Y")
write.csv(tbl[c("id", "borrowed", "price")], row.names = FALSE)
#> "id","borrowed","price"
#> "007","14/01/2025","12.50"
#> "010","03/02/2025","0.33"Source: R documentation: write.table
Text and factor columns are quoted, and numbers, logicals and NA are not. Dates (and any other column with a class) are
converted by their as.character() method and are not quoted, so they appear as 2025-01-14. Numbers are written with
up to 15 significant digits and follow the scipen option, so 1e5 is written 1e+05 and 1e15 as 1e+15: a number that a spreadsheet may read as text. Setting
options(scipen = 100) writes the digits out in full. For anything more specific, convert the column to text with
format() or sprintf() first, which is what the last call does with a date format and two decimals: write.table()
cannot format a number for you, and the converted columns are then written as the text they have become.
Round trips: what survives a text file
A CSV round trip keeps text, numbers and the empty/missing distinction if the arguments are chosen with care, and loses everything that is not text or a number unless it is put back.
source("scenario.R")
path <- tempfile(fileext = ".csv")
# books survives a CSV round trip unchanged, including NA and the empty ISBN
write.csv(books, path, row.names = FALSE, na = "")
identical(read.csv(path), books)
#> [1] TRUE
# loans does not: the dates were written as text and come back as text
write.csv(loans, path, row.names = FALSE, na = "")
back <- read.csv(path, colClasses = "character")
vapply(back, \(x) class(x)[[1L]], character(1))
#> loan_id isbn member loan_date return_date
#> "character" "character" "character" "character" "character"
back$loan_date[19:21]
#> [1] "2025-07-02" "" "2025-07-05"
# colClasses puts the types back
typed <- read.csv(
path,
colClasses = c(
loan_id = "character", isbn = "character", member = "character",
loan_date = "Date", return_date = "Date"
)
)
identical(typed, loans)
#> [1] TRUE
# What a CSV round trip cannot keep: factor levels, row names, time zones
tbl <- data.frame(
size = factor(c("small", "large"), levels = c("small", "medium", "large")),
code = c("007", "010")
)
rownames(tbl) <- c("first", "second")
write.csv(tbl, path)
back <- read.csv(path)
str(back)
#> 'data.frame': 2 obs. of 3 variables:
#> $ X : chr "first" "second"
#> $ size: chr "small" "large"
#> $ code: int 7 10
levels(back$size)
#> NULL
str(read.csv(path, row.names = 1L, colClasses = c(code = "character")))
#> 'data.frame': 2 obs. of 2 variables:
#> $ size: chr "small" "large"
#> $ code: chr "007" "010"
unlink(path)books survives unchanged: identical(read.csv(path), books) is TRUE because the file was written with na = "" and
read with the guessing that produced the data frame in the first place. loans does not. The dates were written as text,
and a plain read gives text back; the empty return_date fields are empty strings, not NA. With the classes named
in colClasses the data frame is identical() to the original. The last part lists what no choice of arguments
restores: the factor comes back as text, and its levels, including the unused level medium, and their order are gone
(levels() is NULL), the row names become an extra column called X (restored with row.names = 1L), and a code with leading zeros turns into
an integer unless it is read as text. A time zone, a POSIXct class and an attribute are lost too; formats that keep R
objects exactly are on the Serialization: RDS, RData and JSON
page.
Writing for a spreadsheet
The help page of write.table() is frank that CSV files do not record an encoding, which is a problem for any file that
is not plain ASCII, and lists what the Excel versions of its time accepted: ASCII, UTF-16 (fileEncoding = "UTF-16LE") or the
current Windows code page ("CP1252"), with the Text Import Wizard allowing far more. A semicolon-and-decimal-comma file is
what write.csv2() produces, the Excel convention in some Western European locales.
first <- data.frame(title = c("Café des Sables", "Loud Roofs"),
price = c(12.5, 9.9))
path <- tempfile(fileext = ".csv")
# Semicolon and decimal comma, the way a spreadsheet in many European
# locales writes and reads "CSV"
write.csv2(first, path, row.names = FALSE)
writeLines(readLines(path))
#> "title";"price"
#> "Café des Sables";12,5
#> "Loud Roofs";9,9
read.csv2(path)
#> title price
#> 1 Café des Sables 12.5
#> 2 Loud Roofs 9.9
# UTF-16LE, the "Unicode text" of Windows Excel: two bytes per character
write.csv(first, path, row.names = FALSE, fileEncoding = "UTF-16LE")
bytes <- readBin(path, "raw", n = 8L)
bytes
#> [1] 22 00 74 00 69 00 74 00
read.csv(path, fileEncoding = "UTF-16LE")
#> title price
#> 1 Café des Sables 12.5
#> 2 Loud Roofs 9.9
# Latin-1 for programs that predate UTF-8
write.csv(first, path, row.names = FALSE, fileEncoding = "latin1")
any(readBin(path, "raw", n = file.size(path)) == as.raw(0xe9))
#> [1] TRUE
read.csv(path, fileEncoding = "latin1")
#> title price
#> 1 Café des Sables 12.5
#> 2 Loud Roofs 9.9
unlink(path)Source: R documentation: write.table
The first block is the semicolon dialect and its reader. The second is the UTF-16LE file, where the bytes 22 00 74 00
show two bytes per character, and the third is a Latin-1 file whose accented letters are one byte (e9). Each is read back
with the same fileEncoding. Which one a given Excel version opens correctly depends on the version, the locale and
whether it is opened by double click or through the import wizard; the safest course is to export from R in the format
the receiving program names, and to check by opening the file there.
readLines() and writeLines()
readLines() reads text lines from a file name or a connection and returns a character vector of one element per line,
with no parsing at all; writeLines() is its counterpart. The arguments of readLines() are n (the maximum number of
lines, a negative number meaning all), ok (whether a short read is allowed), warn (warn about a missing final newline
and about embedded NUL bytes), encoding (a declared encoding, not a conversion) and skipNul. writeLines() has
text, con, sep (the terminator written after each line) and useBytes.
path <- tempfile(fileext = ".txt")
# writeLines() writes one string per line and adds the newline
writeLines(c("first line", "second line", "", "fourth line"), path)
readLines(path)
#> [1] "first line" "second line" "" "fourth line"
# n= limits how many lines are read; the rest is not touched
readLines(path, n = 2L)
#> [1] "first line" "second line"
# sep= changes the line terminator that writeLines() adds
writeLines(c("a", "b"), path, sep = ";")
readBin(path, "raw", n = file.size(path))
#> [1] 61 3b 62 3b
readLines(path, warn = FALSE)
#> [1] "a;b;"
# Without a file, writeLines() writes to the console
writeLines(c("alpha", "beta"))
#> alpha
#> beta
# readLines() does not parse: numbers are still text
writeLines(c("3", "4.5", "x"), path)
lines <- readLines(path)
lines
#> [1] "3" "4.5" "x"
suppressWarnings(as.numeric(lines))
#> [1] 3.0 4.5 NA
# readLines() accepts LF, CRLF and CR as line endings
writeBin(charToRaw("one\r\ntwo\rthree\nfour"), path)
readLines(path, warn = FALSE)
#> [1] "one" "two" "three" "four"
# An empty file gives character(0)
writeLines(character(), path)
file.size(path)
#> [1] 0
readLines(path)
#> character(0)
unlink(path)Source: R documentation: readLines
A few things to see in the output. readLines(path, n = 2L) reads two lines and leaves the rest. A blank line in the file
is the empty string in the vector, so the file with four lines gives four elements. sep in writeLines() replaces the
terminator: with sep = ";" the file has no newline at all, and readLines() then returns one element "a;b;". The line
endings accepted by readLines() are LF, CRLF and a lone CR, whatever the platform, which the example with
"one\r\ntwo\rthree\nfour" demonstrates; that is why a Windows file read on Linux has no stray carriage returns. writeLines()
writes the default "\n" and, on a text-mode connection on Windows, the platform converts it to CRLF. Reading an empty
file gives character(0).
Reading from an open connection
A character string as con opens the file for the duration of the call and closes it. A connection that is opened once
reads on from where the previous call stopped, which is how a file is processed in chunks and how a function consumes the
header before the data.
path <- tempfile(fileext = ".txt")
writeLines(sprintf("record %02d", 1:7), path)
# A connection opened once reads on from where it stopped
con <- file(path, "r")
readLines(con, n = 3L)
#> [1] "record 01" "record 02" "record 03"
readLines(con, n = 2L)
#> [1] "record 04" "record 05"
readLines(con)
#> [1] "record 06" "record 07"
readLines(con)
#> character(0)
close(con)
# on.exit() closes the connection even when reading fails
first_lines <- function(path, n) {
con <- file(path, "r")
on.exit(close(con))
readLines(con, n = n)
}
first_lines(path, 2L)
#> [1] "record 01" "record 02"
# ok = FALSE turns a short read into an error
con <- file(path, "r")
tryCatch(readLines(con, n = 10L, ok = FALSE), error = conditionMessage)
#> [1] "too few lines read in readLines"
close(con)
# A file name is enough for a one-off read; the file is opened and closed
length(readLines(path))
#> [1] 7
# readLines() on a text connection is the in-memory version
tc <- textConnection("a\nb\nc")
readLines(tc)
#> [1] "a" "b" "c"
close(tc)
unlink(path)Source: R documentation: readLines
Each call of readLines(con, n) continues from the previous position, and at the end the result is character(0).
on.exit(close(con)) closes the connection even if the code after file() fails, which is the pattern for any function
that opens a connection itself. ok = FALSE turns a short read into an error whose text is too few lines read in
readLines. A textConnection() is the in-memory counterpart that lets code that reads lines be tested without a file.
The rest of the connection family is on
Connections and the File System.
The incomplete final line
path <- "incomplete.txt"
# A file that does not end with a newline
cat("123\nabc", file = path)
# The default is to accept the last line and warn about it
result <- withCallingHandlers(
readLines(path),
warning = \(w) {
cat("warning:", conditionMessage(w), "\n")
invokeRestart("muffleWarning")
}
)
#> warning: incomplete final line found on 'incomplete.txt'
result
#> [1] "123" "abc"
# warn = FALSE silences the warning for files you know are like this
readLines(path, warn = FALSE)
#> [1] "123" "abc"
# A non-blocking connection keeps the incomplete line back until it is complete
con <- file(path, "r", blocking = FALSE)
readLines(con)
#> [1] "123"
cat(" def\n", file = path, append = TRUE)
readLines(con)
#> [1] "abc def"
close(con)
# A NUL byte ends the line it is in, with a warning of its own
writeBin(as.raw(c(0x61, 0x00, 0x62, 0x0a)), path)
outcome <- withCallingHandlers(
readLines(path),
warning = \(w) {
cat("warning:", conditionMessage(w), "\n")
invokeRestart("muffleWarning")
}
)
#> warning: line 1 appears to contain an embedded nul
outcome
#> [1] "a"
readLines(path, skipNul = TRUE)
#> [1] "ab"
unlink(path)Source: R documentation: readLines
A file that does not end with a newline is read in full, with the warning incomplete final line found on 'incomplete.txt';
warn = FALSE silences it for files known to end that way. The second part shows the other behaviour from the help page.
A non-blocking connection treats the last line as not complete yet and returns only "123"; when more text arrives
(here appended with cat(append = TRUE)) the next call returns the finished line "abc def". This is the rule that a tail-style
reader follows. The NUL byte case is the third form of the warning, line 1 appears to contain an embedded nul: the
line is cut at the NUL ("a"), and skipNul = TRUE ignores the byte ("ab").
Input from the console and from standard input
A script that asks a question has two ways to get an answer, and only one works under Rscript.
# interactive() is FALSE under Rscript, so readline() does not wait: it prints
# the prompt, returns "" at once, and the script goes on
interactive()
#> [1] FALSE
answer <- readline("Member name: ")
#> Member name:
answer
#> [1] ""
nchar(answer)
#> [1] 0
# A script that needs input should take it from the command line or a file
args <- commandArgs(trailingOnly = TRUE)
length(args)
#> [1] 0
member <- if (length(args) > 0L) args[[1L]] else "Ana Ruiz"
member
#> [1] "Ana Ruiz"Source: R documentation: readline
readline() reads from the terminal in an interactive session. Under Rscript interactive() is FALSE, and the help page
says the result is then as if the user pressed RETURN: the prompt is printed, the value is "", and the script goes on
without waiting. A script that needs a value should take it from the command line (commandArgs(trailingOnly = TRUE),
described on Scripts and the Command Line), from an
environment variable, or from standard input. Standard input is read with file("stdin"), not with stdin().
# Read every line the process was given on its standard input
con <- file("stdin")
lines <- readLines(con)
close(con)
cat("lines read:", length(lines), "\n")
cat(toupper(lines), sep = "\n")Source: R documentation: connections
The script is saved as shout.R and run with the data piped into it. With nothing piped in, readLines() returns
character(0) and the message is lines read: 0.
printf 'ana ruiz\nben okafor\n' | Rscript shout.RSource: R documentation: connections
The real output of that command, on R 4.6.1:
lines read: 2
ANA RUIZ
BEN OKAFORSource: R documentation: stdin
stdin() is different. The help page says it refers to the console of R and not to the C-level standard input of the
process, and that when R reads a script from a file, the file is the console. A readLines(stdin(), n = 3L) in a script
file therefore reads the next three lines of the script itself, which R does not run as code; the idiom is how very
old tutorials embedded data in a script (the appendix "Scripting with R" of An Introduction to R, An Introduction to R: Scripting with R, shows it). It was run once to check;
the script is not valid R, so it cannot be linted, and it is shown here only to explain what stdin() does under
Rscript.
inline <- readLines(stdin(), n = 3L)
Ana Ruiz,4
Ben Okafor,5
Dev Patel,3
inline
#> [1] "Ana Ruiz,4" "Ben Okafor,5" "Dev Patel,3"
read.csv(text = inline, header = FALSE, col.names = c("member", "rating"))
#> member rating
#> 1 Ana Ruiz 4
#> 2 Ben Okafor 5
#> 3 Dev Patel 3Source: R documentation: stdin
Use file("stdin") for piped input, and commandArgs() for arguments.
scan()
scan() is the primitive under read.table(). It reads a stream of values of the type given by what and returns either
a vector or, if what is a list, a list with one vector per field. The manual’s chapter on using scan() directly gives the
reason to know it: the arguments that read.table() has are mostly those of scan(), and scan() is the faster and more
economical tool for a big numeric matrix, where read.table() pays the cost of one column per field
(R Data Import/Export: Using scan directly).
# scan() reads a stream of values of one type; text= stands in for a file
scan(text = "2 3 5 7 11")
#> Read 5 items
#> [1] 2 3 5 7 11
# quiet = TRUE removes the "Read N items" line
scan(text = "2 3 5 7 11", quiet = TRUE)
#> [1] 2 3 5 7 11
# what= gives the type of the values; sep= the separator
scan(text = "ana,ben,carla", what = "", sep = ",", quiet = TRUE)
#> [1] "ana" "ben" "carla"
scan(text = "1,2,,4", sep = ",", quiet = TRUE)
#> [1] 1 2 NA 4
# A character value in a numeric stream is an error
tryCatch(scan(text = "1 2 three", quiet = TRUE), error = conditionMessage)
#> [1] "scan() expected 'a real', got 'three'"
# n= and nlines= limit what is read; skip= drops leading lines
text <- c("header line", "1 2 3", "4 5 6", "7 8 9")
scan(text = text, skip = 1L, nlines = 2L, quiet = TRUE)
#> [1] 1 2 3 4 5 6
scan(text = text, skip = 1L, n = 4L, quiet = TRUE)
#> [1] 1 2 3 4
# what= may be a list: one entry per field, so each record is a row
records <- c("L001 Hugo 21", "L002 Ben 10", "L003 Dev 1")
inp <- scan(text = records, what = list(id = "", who = "", days = 0L),
quiet = TRUE)
str(inp)
#> List of 3
#> $ id : chr [1:3] "L001" "L002" "L003"
#> $ who : chr [1:3] "Hugo" "Ben" "Dev"
#> $ days: int [1:3] 21 10 1
as.data.frame(inp)
#> id who days
#> 1 L001 Hugo 21
#> 2 L002 Ben 10
#> 3 L003 Dev 1
# A NULL entry skips a field but still counts it
skipped <- scan(text = records, what = list(id = "", NULL, days = 0L),
quiet = TRUE)
str(skipped)
#> List of 3
#> $ id : chr [1:3] "L001" "L002" "L003"
#> $ : NULL
#> $ days: int [1:3] 21 10 1
# fill = TRUE pads short records; flush = TRUE drops trailing fields
str(scan(text = c("a 1", "b"), what = list("", 0), fill = TRUE, quiet = TRUE))
#> List of 2
#> $ : chr [1:2] "a" "b"
#> $ : num [1:2] 1 NA
str(scan(text = "a 1 x", what = list("", 0), flush = TRUE, quiet = TRUE))
#> List of 2
#> $ : chr "a"
#> $ : num 1
# scan() is the fast way to read a numeric matrix
values <- scan(text = "1 2 3 4 5 6", quiet = TRUE)
matrix(values, nrow = 2L, byrow = TRUE)
#> [,1] [,2] [,3]
#> [1,] 1 2 3
#> [2,] 4 5 6
# One value per line from a file
path <- tempfile(fileext = ".txt")
writeLines(c("10", "20", "30"), path)
sum(scan(path, quiet = TRUE))
#> [1] 60
unlink(path)The main arguments are visible here. Without quiet = TRUE, scan() prints a line saying how many items it read
(Read 5 items), which goes to the message stream, not to the output; quiet = TRUE silences it, and scripts normally want that.
what gives the type ("" for text, 0L for integers, 0 for doubles) and sep the separator, with the default being any
white space. An empty numeric field between two separators is NA, and a non-number in a numeric stream is the error
scan() expected 'a real', got 'three'. skip drops leading lines, nlines limits the lines read, n and nmax limit the
number of values or records. A list for what reads one record per line: each field is converted to the type of its
element, the list names become the names of the result, and as.data.frame() turns it into a data frame; a NULL
element skips a field but keeps its place in the result. fill = TRUE pads short records, flush = TRUE drops the fields
after the last one requested (which also allows comments after the last field), and multi.line = FALSE requires each
record to sit on one line. Reading a matrix is matrix(scan(file), nrow, byrow = TRUE), as in the manual.
Two defaults differ from read.table(), and they are the usual cause of surprises when code is moved between the two. The
default comment.char of scan() is the empty string, whereas read.table() uses #; and a quoted string in scan() with
the default separator treats a backslash as an escape.
Fixed-width files
Some old systems, and some government data, write fields in fixed character positions with no separators. read.fwf()
reads them from a vector of field widths. The manual describes how it works: the function reads the whole file into memory
as lines, splits them by position, writes a temporary tab-separated file and calls read.table() on it. That is adequate for
small files; the manual suggests a preprocessing tool in another language for anything more complicated, and mentions
read.fortran() for Fortran-style column specifications
(R Data Import/Export: Fixed-width-format files).
# A fixed-width export from an old circulation system: no separators at all
loans <- read.csv("loans.csv", colClasses = "character")[1:3, ]
records <- sprintf(
"%-4s %-10s %-12s %s", loans$loan_id, loans$loan_date, loans$member,
loans$isbn
)
path <- tempfile(fileext = ".txt")
writeLines(records, path)
writeLines(readLines(path))
#> L001 2025-01-14 Hugo Silva 9780141439518
#> L002 2025-01-16 Ben Okafor 9780306406157
#> L003 2025-01-21 Dev Patel 9780002000291
# widths= counts characters; a negative width skips that many characters
widths <- c(4, -1, 10, -1, 12, -1, 13)
read.fwf(path, widths = widths)
#> V1 V2 V3 V4
#> 1 L001 2025-01-14 Hugo Silva 9.780141e+12
#> 2 L002 2025-01-16 Ben Okafor 9.780306e+12
#> 3 L003 2025-01-21 Dev Patel 9.780002e+12
read.fwf(path, widths = widths,
col.names = c("loan_id", "loan_date", "member", "isbn"))
#> loan_id loan_date member isbn
#> 1 L001 2025-01-14 Hugo Silva 9.780141e+12
#> 2 L002 2025-01-16 Ben Okafor 9.780306e+12
#> 3 L003 2025-01-21 Dev Patel 9.780002e+12
# colClasses and strip.white go through to read.table()
str(read.fwf(
path,
widths = widths,
col.names = c("loan_id", "loan_date", "member", "isbn"),
colClasses = c("character", "Date", "character", "character"),
strip.white = TRUE
))
#> 'data.frame': 3 obs. of 4 variables:
#> $ loan_id : chr "L001" "L002" "L003"
#> $ loan_date: Date, format: "2025-01-14" "2025-01-16" ...
#> $ member : chr "Hugo Silva" "Ben Okafor" "Dev Patel"
#> $ isbn : chr "9780141439518" "9780306406157" "9780002000291"
# A zero width or a field beyond the end of a short line is NA
short <- tempfile(fileext = ".txt")
writeLines(c("123456", "9876"), short)
read.fwf(short, widths = c(1, 0, 2, 3))
#> V1 V2 V3 V4
#> 1 1 NA 23 456
#> 2 9 NA 87 6
unlink(c(path, short))Source: R documentation: read.fwf
The widths count characters. A negative width skips that many characters, and the skipped fields are not part of the
result, so they are not given in col.names or colClasses. The extra arguments pass through to read.table(), so
colClasses can keep an ISBN as text, and strip.white = TRUE removes the padding that the fixed width put around
"Hugo Silva"; without it the member names keep their trailing blanks. A field of zero width, or one beyond the end of a
short line, is NA. sep is the character that read.fwf() uses internally between fields (a tab by default); it must
not occur in the file.
type.convert()
type.convert() is the function that read.table() calls on every column, and it is also the way to apply the same
guessing to a data frame whose columns are all text (the result of readLines() plus strsplit(), or of colClasses =
"character"). It works on a vector, matrix, array, data frame or list, and recurses into data frames and lists.
# type.convert() is the function behind the guessing of read.table()
x <- c("12", "7", NA, "30")
type.convert(x, as.is = TRUE)
#> [1] 12 7 NA 30
# The first type that accepts every value wins: logical, integer, numeric,
# complex, and then text (a factor only when as.is = FALSE)
class(type.convert(c("TRUE", "FALSE"), as.is = TRUE))
#> [1] "logical"
class(type.convert(c("1", "2"), as.is = TRUE))
#> [1] "integer"
class(type.convert(c("1", "2.5"), as.is = TRUE))
#> [1] "numeric"
class(type.convert(c("1+2i"), as.is = TRUE))
#> [1] "complex"
class(type.convert(c("1", "x"), as.is = TRUE))
#> [1] "character"
class(type.convert(c("1", "x"), as.is = FALSE))
#> [1] "factor"
# A column of only NA is logical, because NA is logical
class(type.convert(c(NA, NA), as.is = TRUE))
#> [1] "logical"
class(type.convert(c("", ""), as.is = TRUE))
#> [1] "logical"
# na.strings decides which texts are missing
type.convert(c("1", "-", "3"), na.strings = "-", as.is = TRUE)
#> [1] 1 NA 3
# On a data frame or a list it works column by column
messy <- data.frame(
n = c("1", "2"), x = c("0.5", "1e3"), s = c("a", "b"),
stringsAsFactors = FALSE
)
str(type.convert(messy, as.is = TRUE))
#> 'data.frame': 2 obs. of 3 variables:
#> $ n: int 1 2
#> $ x: num 5e-01 1e+03
#> $ s: chr "a" "b"
# Leaving as.is out warns, and then behaves as as.is = TRUE
caught <- withCallingHandlers(
type.convert(c("a", "b")),
warning = \(w) {
cat("warning:", conditionMessage(w), "\n")
invokeRestart("muffleWarning")
}
)
#> warning: 'as.is' should be specified by the caller; using TRUE
class(caught)
#> [1] "character"
# An exponent needs digits after the e, since R 4.4.1
type.convert(c("1e5", "2"), as.is = TRUE)
#> [1] 1e+05 2e+00
type.convert(c("1e", "2"), as.is = TRUE)
#> [1] "1e" "2"
# tryLogical = FALSE keeps T, F, TRUE and FALSE as text
type.convert(c("T", "F"), as.is = TRUE)
#> [1] TRUE FALSE
type.convert(c("T", "F"), as.is = TRUE, tryLogical = FALSE)
#> [1] "T" "F"Source: R documentation: type.convert
The first type that accepts every non-missing value wins, in the order logical, integer, numeric and complex, then text;
as.is = FALSE makes the last step a factor. A column of only NA is logical, because NA is a logical value, and so is
a column of empty strings. na.strings names the spellings of missing, and on a data frame the function works column by
column. The warning in the middle is the one added in R 4.1.0: when as.is is not given, type.convert() warns
'as.is' should be specified by the caller; using TRUE. The NEWS entry says that the help file always asked callers to
specify it, and that the default changed to TRUE to match the change of read.table() in R 4.0.0; before R 4.1.0 the
silent default produced factors from text. tryLogical = FALSE (R 4.3.0) stops T and F from becoming logical. Since R 4.4.1 an exponent must have digits, so 1e stays text.
Text for people: cat(), format(), sprintf(), formatC() and prettyNum()
write.table() is for machines; for a report, a log line, or a message the output is built as text. The tools are
cat(), which prints values with a separator and no quotes or index; format(), which turns a vector into aligned
strings; sprintf(), which fills a C-style template; formatC(), which does the same with named arguments; and
prettyNum(), which inserts big marks into numbers. The manual’s note on export is the reason to care: cat()
underlies the exporting functions, takes a file and an append argument, and is the tool for writing text a piece at a
time.
# cat() joins its arguments with sep= and prints no quotes and no [1]
cat("Loans:", 60L, "returned:", 53L, "\n")
#> Loans: 60 returned: 53
cat("a", "b", "c", sep = "")
#> abc
cat("\n")
#>
cat("one", "two", sep = "\n")
#> one
#> two
cat("\n")
#>
# cat(fill = TRUE) wraps at the line width and ends the line
cat(paste("word", 1:12), fill = 30)
#> word 1 word 2 word 3 word 4
#> word 5 word 6 word 7 word 8
#> word 9 word 10 word 11
#> word 12
# Numbers: cat() uses up to 7 significant digits, like print()
cat(pi, 1 / 3, 1e6, 123456789, 1e-5, "\n")
#> 3.141593 0.3333333 1e+06 123456789 1e-05
cat(format(pi, digits = 10), "\n")
#> 3.141592654
# format(): align and round a vector for printing
format(c(1, 10, 100))
#> [1] " 1" " 10" "100"
format(3.14159, nsmall = 2)
#> [1] "3.14159"
format(2, nsmall = 2)
#> [1] "2.00"
format(1234567.891, big.mark = ",")
#> [1] "1,234,568"
format(0.000012345, scientific = TRUE)
#> [1] "1.2345e-05"
format(TRUE)
#> [1] "TRUE"
format(c("a", "bbb"), width = 5)
#> [1] "a " "bbb "
# sprintf(): C-style templates, vectorised over every argument
sprintf("%s has %d pages (%.1f%% read)", "Dune", 412L, 37.5)
#> [1] "Dune has 412 pages (37.5% read)"
sprintf("%05d", 42L)
#> [1] "00042"
sprintf("%-10s|%8.2f|", c("Dune", "Emma"), c(9.9, 12.5))
#> [1] "Dune | 9.90|" "Emma | 12.50|"
sprintf("%e", 12345.6789)
#> [1] "1.234568e+04"
sprintf("%3d.", 1:3)
#> [1] " 1." " 2." " 3."
# formatC(): the same idea with named arguments
formatC(3.14159, digits = 2, format = "f")
#> [1] "3.14"
formatC(42L, width = 8, flag = "0")
#> [1] "00000042"
formatC(c(1, 10, 100), width = 6, flag = " ")
#> [1] " 1" " 10" " 100"
formatC(1234567.891, format = "f", big.mark = ",", digits = 1)
#> [1] "1,234,567.9"
formatC("left", width = -8)
#> [1] "left "
formatC(0.000123, format = "e", digits = 2)
#> [1] "1.23e-04"
# prettyNum(): insert separators into numbers or strings that are already text
prettyNum(1234567, big.mark = ",")
#> [1] "1,234,567"
prettyNum(c(123456, 1234), big.mark = ",")
#> [1] "123,456" "1,234"
prettyNum("1234567.5", big.mark = ",")
#> [1] "1,234,567.5"
# toString() and paste0() build labels from vectors
toString(1:5)
#> [1] "1, 2, 3, 4, 5"
paste0("L", sprintf("%03d", 1:3))
#> [1] "L001" "L002" "L003"Source: R documentation: sprintf
Several details are visible in the output. cat() uses up to 7 significant digits for numbers, as print() does, so
pi is 3.141593 and a number like 123456789 is written in full; a precise number needs format(digits = ). fill
wraps at the given width and adds a final newline. format(c(1, 10, 100)) pads to a common width so that the numbers align;
nsmall is a minimum number of decimals, not a rounding; big.mark inserts thousands marks. sprintf() is vectorised
over every argument, so sprintf("%-10s|%8.2f|", names, prices) formats a column in one call; %% is a literal percent,
%05d pads with zeros and %e is scientific. formatC() has the same power with width, flag, digits, format
and big.mark, and a negative width means left-justify. prettyNum() works on numbers and on strings that already hold
digits. Which one to prefer is a matter of taste; sprintf() is the most compact for a fixed template and format() for
aligning a whole column.
sink() and capture.output()
Sometimes the text you want to keep is what print() already shows: a model summary, a table, a str(). The R Data
Import/Export manual’s section on exporting notes that sink() diverts R’s standard output to a file, that it is not
usually the most efficient route, and that options(width) may need increasing. capture.output() is the safer front-end: it
evaluates its arguments and returns what they printed as a character vector, or writes it to a file.
books <- read.csv("books.csv")
path <- tempfile(fileext = ".txt")
# capture.output() returns what the expressions would have printed
text <- capture.output(summary(books$year))
text
#> [1] " Min. 1st Qu. Median Mean 3rd Qu. Max. NAs "
#> [2] " 1813 2006 2016 1999 2020 2023 1 "
capture.output(cat("one\n"), print(1:3))
#> [1] "one" "[1] 1 2 3"
# It is the right tool to write printed output to a file
writeLines(capture.output(str(books[1:3, c("title", "year")])), path)
writeLines(readLines(path))
#> 'data.frame': 3 obs. of 2 variables:
#> $ title: chr "The Dispossessed" "Pride and Prejudice" " the quiet streets "
#> $ year : int 1974 1813 2019
# sink() diverts all output to a file until sink() is called again
sink(path)
print(summary(books$pages))
cat("pages known:", sum(!is.na(books$pages)), "\n")
sink()
writeLines(readLines(path))
#> Min. 1st Qu. Median Mean 3rd Qu. Max. NAs
#> 88.0 202.0 268.0 268.2 344.5 432.0 2
#> pages known: 14
# sink(append = TRUE) adds to the file; sink.number() counts the open sinks
sink(path, append = TRUE)
cat("appended; open sinks:", sink.number(), "\n")
sink()
tail(readLines(path), 1L)
#> [1] "appended; open sinks: 1 "
# A sink that is never closed swallows the rest of the output; on.exit()
# inside a function is the safe pattern
log_to <- function(path, code) {
sink(path)
on.exit(sink())
force(code)
}
log_to(path, print("inside the file"))
readLines(path)
#> [1] "[1] \"inside the file\""
# cat(file =) and write() are the simple alternatives for small outputs
cat("title\n", file = path)
cat("Dune\n", "Emma\n", file = path, sep = "", append = TRUE)
readLines(path)
#> [1] "title" "Dune" "Emma"
write(c(1, 2, 3, 4, 5, 6), path, ncolumns = 3L)
readLines(path)
#> [1] "1 2 3" "4 5 6"
unlink(path)Source: R documentation: sink
The first part is the whole of capture.output(): the printed lines of summary() come back as two strings, and several
expressions can be captured in one call. The writeLines(capture.output(x), path) form is the usual way to write a
printed object to a file. sink(path) diverts all the output that follows until sink() is called again, which is why
the print() and cat() calls in the script printed nothing to the console; append = TRUE adds to the file, and
sink.number() reports how many diversions are open (one, inside the sink). A sink that is never closed swallows the
rest of the output, and an error between sink(path) and sink() leaves it open, so inside a function the pattern is
sink(path); on.exit(sink()). The help page warns against sinking the message stream (type = "message") unless the
effects are understood, because there is no stack and the target must be an already open connection. split = TRUE
sends output to the file and the console both, like the Unix tee, but splits only R’s own printing and the default
output of writeLines(). The last block shows the plain alternatives: cat(file = , append = TRUE) and write(), which
writes a vector in rows of ncolumns values.
The two tools differ in what they capture. capture.output() works through a sink, so the code it evaluates must not
close connections, and if the code fails, the output collected so far is lost unless a file is given. Neither captures
messages and warnings by default.
Built-in data: datasets and data()
R ships a package of example data, datasets, that is attached at start-up. An Introduction to R says that around a
hundred data sets are supplied with R and that all of them are available directly by name. The same manual notes that
many packages still use an older convention in which data() was also used to load them, as in data(infert), which
still works with the standard packages
(An Introduction to R: Accessing builtin datasets).
The two ways meet in the next script.
# Every dataset of the datasets package is available by name, lazily
head(mtcars, 3L)
#> mpg cyl disp hp drat wt qsec vs am gear carb
#> Mazda RX4 21.0 6 160 110 3.90 2.620 16.46 0 1 4 4
#> Mazda RX4 Wag 21.0 6 160 110 3.90 2.875 17.02 0 1 4 4
#> Datsun 710 22.8 4 108 93 3.85 2.320 18.61 1 1 4 1
class(mtcars)
#> [1] "data.frame"
dim(mtcars)
#> [1] 32 11
# data() with no argument lists the datasets of the attached packages;
# the result is an object of class "packageIQR"
listing <- data(package = "datasets")
class(listing)
#> [1] "packageIQR"
dim(listing$results)
#> [1] 108 4
head(listing$results[, c("Item", "Title")], 4L)
#> Item Title
#> [1,] "AirPassengers" "Monthly Airline Passenger Numbers 1949-1960"
#> [2,] "BJsales" "Sales Data with Leading Indicator"
#> [3,] "BJsales.lead (BJsales)" "Sales Data with Leading Indicator"
#> [4,] "BOD" "Biochemical Oxygen Demand"
# The same for any installed package
rpart_sets <- data(package = "rpart")
rpart_sets$results[, "Item"]
#> [1] "car.test.frame" "car90"
#> [3] "cu.summary" "kyphosis"
#> [5] "solder" "solder.balance (solder)"
#> [7] "stagec"
# data(name) also copies the object into the global environment
exists("mtcars", envir = globalenv(), inherits = FALSE)
#> [1] FALSE
data(mtcars)
exists("mtcars", envir = globalenv(), inherits = FALSE)
#> [1] TRUE
identical(get("mtcars", envir = globalenv()), datasets::mtcars)
#> [1] TRUE
# A modified copy (here: km per litre) shadows the dataset; the dataset
# in the package is untouched
mtcars$mpg <- mtcars$mpg * 1.60934 / 3.78541
head(mtcars$mpg, 3L)
#> [1] 8.928000 8.928000 9.693257
head(datasets::mtcars$mpg, 3L)
#> [1] 21.0 21.0 22.8
rm(mtcars)
# Use a character string and envir= in code that must not touch the workspace
local_sets <- new.env()
data("airquality", envir = local_sets)
ls(local_sets)
#> [1] "airquality"
exists("airquality", envir = globalenv(), inherits = FALSE)
#> [1] FALSE
# Asking for a dataset that does not exist is a warning, not an error
caught <- withCallingHandlers(
data("no_such_dataset"),
warning = \(w) {
cat("warning:", conditionMessage(w), "\n")
invokeRestart("muffleWarning")
}
)
#> warning: data set ‘no_such_dataset’ not found
caught
#> [1] "no_such_dataset"
# package= names where to look when two packages use the same name
data("Puromycin", package = "datasets", envir = local_sets)
sort(ls(local_sets))
#> [1] "airquality" "Puromycin"Source: R documentation: data
What each part shows:
-
head(mtcars, 3L)works at once. The data sets ofdatasetsare lazy-loaded: the object is not in memory until first used, and it lives in the package, not in your workspace.data()is not needed, and the help page says that the need it once met, to avoid having large data sets in memory, has been almost entirely superseded by lazy loading. -
data(package = "datasets")returns an object of class"packageIQR"whose$resultsmatrix has one row per item (108 in this version, including entries such asBJsales.lead (BJsales), which says that the objectBJsales.leadis loaded bydata(BJsales)). The help page marks the structure of the class as experimental.data(package = )works for any installed package, as the listing ofrpart, one of the recommended packages, shows. -
data(mtcars)copies the object into the global environment, where it shadows the package copy. The next lines changempgin the copy and show that the data set in the package is untouched.rm(mtcars)removes the copy. -
data("airquality", envir = local_sets)puts the object into a chosen environment instead. The help page recommends this in programming:data()called inside a function withoutenvirhas the almost always undesirable side effect of putting an object in the user’s workspace and replacing any object of that name. -
A name that does not exist is a warning, not an error: the text is
data set ‘no_such_dataset’ not found(with the typographic quotes of a UTF-8 session), and the call returns the name. -
package =says where to look when the same name exists in several packages.
The help page lists four file formats that data() understands, in the order it tries them: .R files (sourced, with the
working directory changed temporarily), .RData, .rdata and .rda files (loaded), .tab, .txt and .TXT files (read
with read.table(header = TRUE, as.is = FALSE)), and .csv and .CSV files (read with read.table(header = TRUE,
sep = ";", as.is = FALSE)). The semicolon is not a typo: a .csv file in the data directory of a package is
semicolon-separated, and Writing R Extensions notes that .csv there is not the standard CSV format. data() also
documents a warning that applies to all four: loading a data set can silently create objects other than the one named, and
there have been published packages in which data() replaced .Random.seed.
Lazy loading and the LazyData field
A package of your own makes its data available in the same way. The data subdirectory holds files of the three kinds
above (.R, .rda, .tab/.txt/.csv), and the LazyData field of the DESCRIPTION file decides whether they are
lazy-loaded (available by name once the package is attached) or loaded on request with data(); the default is not to
lazy-load. The manual also names inst/extdata as the place for data files that are not R data sets.
Package: shelf
Version: 0.1.0
LazyData: trueWith LazyData: true, shelf::books or a plain books after library(shelf) would work; with the field absent a user
would run data(books, package = "shelf") first. The Creating Packages
page builds the shelf package with such a data directory, and
Packages, Libraries and Namespaces explains the
:: form.
penguins and penguins_raw
R 4.5.0 added two data sets to datasets: penguins, a table of 344 adult penguins of three species on three islands of
the Palmer Archipelago in Antarctica, with their size measurements and sex, and penguins_raw, the more extensive table
(17 columns, with nesting observations and blood isotope data) from which the eight columns of penguins are taken, with
different names and data types. The R NEWS entry names the authors of the data sets
(R NEWS: Changes in R 4.5.0). They are a better teaching
data set than mtcars for categorical columns and missing values, and tutorials written before 4.5.0 had to install a
package to get them, so code written against that package may use other column names.
# penguins and penguins_raw joined the datasets package in R 4.5.0
str(penguins)
#> 'data.frame': 344 obs. of 8 variables:
#> $ species : Factor w/ 3 levels "Adelie","Chinstrap",..: 1 1 1 1 1 1 1 1 1 1 ...
#> $ island : Factor w/ 3 levels "Biscoe","Dream",..: 3 3 3 3 3 3 3 3 3 3 ...
#> $ bill_len : num 39.1 39.5 40.3 NA 36.7 39.3 38.9 39.2 34.1 42 ...
#> $ bill_dep : num 18.7 17.4 18 NA 19.3 20.6 17.8 19.6 18.1 20.2 ...
#> $ flipper_len: int 181 186 195 NA 193 190 181 195 193 190 ...
#> $ body_mass : int 3750 3800 3250 NA 3450 3650 3625 4675 3475 4250 ...
#> $ sex : Factor w/ 2 levels "female","male": 2 1 1 NA 1 2 1 2 NA NA ...
#> $ year : int 2007 2007 2007 2007 2007 2007 2007 2007 2007 2007 ...
colSums(is.na(penguins))
#> species island bill_len bill_dep flipper_len body_mass
#> 0 0 2 2 2 2
#> sex year
#> 11 0
table(penguins$species, penguins$island)
#>
#> Biscoe Dream Torgersen
#> Adelie 44 56 52
#> Chinstrap 0 68 0
#> Gentoo 124 0 0
# penguins_raw is the untidy original, with awkward column names
dim(penguins_raw)
#> [1] 344 17
names(penguins_raw)[1:6]
#> [1] "studyName" "Sample Number" "Species" "Region"
#> [5] "Island" "Stage"
# Complete rows only
nrow(penguins[complete.cases(penguins), ])
#> [1] 333
# Group means need na.rm = TRUE because of the missing measurements
tapply(penguins$body_mass, penguins$species, mean, na.rm = TRUE)
#> Adelie Chinstrap Gentoo
#> 3700.662 3733.088 5076.016Source: R documentation: penguins
penguins has three factor columns (species, island, sex) and numeric measurements, with two missing
measurements in each of four columns and 11 missing values of sex; colSums(is.na()) finds them, and 333 rows are
complete. penguins_raw has 17 columns, several of whose names include spaces and units, such as Culmen Length (mm), which would need backticks. Group
means need na.rm = TRUE, as the last call shows. Neither is part of the shelf scenario; the shelf analyses use the
books and loans tables.
Excel and other systems
Text is the interchange format of last resort, and often of first resort. R Data Import/Export says it plainly for
Excel: the most common question about R data import is how to read a spreadsheet, the first piece of advice is to avoid
reading it directly if possible, and if Excel is at hand the data should be exported as tab-delimited or
comma-separated text and read with read.delim() or read.csv() (read.delim2() or read.csv2() in a locale with a
decimal comma)
(R Data Import/Export: Reading Excel spreadsheets).
The same chapter notes that an .xls file is not just a table (it can hold many sheets, formulae and macros), that not all
readers can read other than the first sheet, and, for Windows, that read.table() can read the clipboard. The advice
dates from the days of .xls; for the modern .xlsx format there is now a package that reads both without external
dependencies. The table puts the options side by side.
| System or format | Base R and the recommended package foreign |
Add-on packages (CRAN) | Notes |
|---|---|---|---|
Excel |
no direct reader; export CSV or tab-separated text and use |
readxl 1.6.0 reads both formats ( |
a CSV round trip loses formulae, formats and sheets; the |
SPSS |
|
haven 2.5.5: |
haven keeps value labels in a |
SAS |
|
haven 2.5.5: |
|
Stata |
|
haven 2.5.5: |
value labels become factors in |
Minitab, S-PLUS, EpiInfo, Systat |
|
none needed |
the manual warns that these were written for versions from the early 2000s |
Octave and MATLAB |
|
R.matlab (CRAN, not installed here): |
a MAT file is binary; |
Parquet and Feather |
none |
arrow (not installed here): |
columnar formats for large data; the package wraps the Arrow C++ library |
SQL databases |
none (the DBI interface is a package) |
DBI and a backend such as RSQLite |
Databases with DBI and the SQL Reference |
JSON |
none |
jsonlite |
|
XML and YAML |
none |
xml2 and yaml, which the manual names for XML and YAML |
described on the same serialization page |
Fixed-width, DIF, flat contingency tables |
|
none needed |
|
Two comments on the table. First, the foreign package is a recommended package, installed with every standard R, and the
manual’s chapter on importing from other systems says it provides import for the files of the systems in the table and
export to Stata, with write.foreign() for SAS, SPSS and Stata. The chapter also warns that the functions were written
for the file versions of the time and have not necessarily been updated for the current ones
(R Data Import/Export: Importing from other statistical systems),
which is the reason to use haven for modern SAS, SPSS and Stata files. Second, none of the "other systems" readers is
needed for text: an .sav file is not a text file, and a CSV exported from SPSS is read with read.csv() like any other.
A foreign round trip
The foreign package works without any file from another system, because it can also write Stata files and the text and
program files that SAS, SPSS or Stata use to read a text file.
library(foreign)
books <- read.csv("books.csv")
path <- tempfile(fileext = ".dta")
# Stata's .dta format: foreign reads and writes Stata versions 5 to 12
write.dta(books[1:4, c("title", "year", "genre")], path)
back <- read.dta(path)
vapply(back, \(x) class(x)[[1L]], character(1))
#> title year genre
#> "character" "integer" "character"
identical(back$title, books$title[1:4])
#> [1] TRUE
back$year
#> [1] 1974 1813 2019 2021
# Value labels in the file become factors; convert.factors = FALSE keeps codes
labelled <- data.frame(
grade = factor(c("low", "high", "low"), levels = c("low", "high"))
)
write.dta(labelled, path)
read.dta(path)$grade
#> [1] low high low
#> Levels: low high
read.dta(path, convert.factors = FALSE)$grade
#> [1] 1 2 1
# write.foreign() writes a text file plus the code that reads it in SAS,
# SPSS or Stata
data_file <- tempfile(fileext = ".txt")
code_file <- tempfile(fileext = ".sps")
write.foreign(
books[1:3, c("title", "year", "pages")],
datafile = data_file, codefile = code_file, package = "SPSS"
)
writeLines(readLines(data_file))
#> The Dispossessed,1974,387,
#> Pride and Prejudice,1813,432,
#> the quiet streets ,2019,214,
code <- readLines(code_file)
writeLines(code[!grepl("FILE=", code, fixed = TRUE) & nzchar(code)])
#> SET DECIMAL=DOT.
#> ENCODING="UTF8"
#> / title (A20) year pages
#> .
#> VARIABLE LABELS
#> title "title"
#> year "year"
#> pages "pages"
#> .
#> VARIABLE LEVEL year, pages
#> (scale).
#> EXECUTE.
unlink(c(path, data_file, code_file))write.dta() writes a Stata file and read.dta() reads it back; the text columns stay text, the integers stay
integers, and a factor is written as Stata value labels, which read.dta() turns back into a factor
(convert.factors = FALSE keeps the integer codes 1, 2, 1). write.foreign() writes two files, a text file of the data and
a program in the language of the target package; with package = "SPSS" the program is a DATA LIST and VARIABLE
LABELS script (the line that names the data file is dropped from the output above because it holds a temporary path).
The data file is written with write.table(), which is why it has the quoting and trailing-comma features of that
function.
readr, vroom, readxl, haven, arrow and fread
Five packages are named so often in data-import advice that a reader should know what they are. readr
(readr package documentation: read_delim) is the
tidyverse reader for delimited files: read_csv(), read_delim() and write_csv() return tibbles, report the column
types they guessed, trim white space and treat both the empty string and NA as missing by default, and read the shelf’s
year and pages as doubles where base R reads integers. vroom (vroom package documentation)
indexes a delimited file and reads its columns lazily through the ALTREP framework, so that you only pay for the columns you use. readxl (readxl package documentation: read_excel)
reads .xls and .xlsx with no external dependencies. haven (haven package documentation: read_spss)
reads and writes SAS, SPSS and Stata files by wrapping the ReadStat C library and keeps their value labels.
arrow (not installed in the checking environment; arrow package documentation) reads and
writes Parquet and Feather files and provides an Arrow C++ backend for dplyr; this page does not run it.
Outside the tidyverse, data.table::fread()
(data.table package documentation: fread) detects the separator and header by itself
and returns a data.table. All of these are described in the documentation sites named above, and readr, readxl and arrow
also have chapters in the community book "R for Data Science"; the pages
The tidyverse and dplyr and
data.table cover the ecosystems around them. Base R is enough for the
files of this page, needs nothing to install, and its defaults are the ones that the rest of the R documentation assumes.
Putting it together: clean, save, reload
The pieces combine into the usual cycle for the shelf data: read the raw files with the types stated, clean the text
columns, write a tidy CSV with the conventions the next reader needs, and prove that the file reads back to the same
data frame. The cleaning steps (trimws(), collapsing double spaces, lower-casing the genre, turning the empty ISBN into
NA) are the ones the Data Frames page explains.
source("scenario.R")
# The cleaning steps of the data-frames page, applied to the text columns
tidy_books <- books
tidy_books$title <- trimws(tidy_books$title)
tidy_books$author <- gsub("\\s+", " ", trimws(tidy_books$author))
tidy_books$genre <- tolower(trimws(tidy_books$genre))
tidy_books$isbn <- ifelse(nzchar(tidy_books$isbn), tidy_books$isbn, NA)
# Write the clean table: ISO dates are text, so they survive any reader
out_path <- tempfile(fileext = ".csv")
write.csv(tidy_books, out_path, row.names = FALSE, na = "")
writeLines(readLines(out_path, n = 4L))
#> "title","author","isbn","year","genre","pages"
#> "The Dispossessed","Ursula K. Le Guin","978-0-306-40615-7",1974,"science fiction",387
#> "Pride and Prejudice","Jane Austen","978-0-14-143951-8",1813,"classic",432
#> "the quiet streets","Mara Quill","9780001000100",2019,"fiction",214
# Read it back with the types stated, and compare
back <- read.csv(
out_path,
colClasses = c(
title = "character", author = "character", isbn = "character",
year = "integer", genre = "character", pages = "integer"
),
na.strings = ""
)
identical(back, tidy_books)
#> [1] TRUE
# The loans table with its dates, written and read back with Date columns
loans_path <- tempfile(fileext = ".csv")
write.csv(loans, loans_path, row.names = FALSE, na = "")
loans_back <- read.csv(
loans_path,
colClasses = c(
loan_id = "character", isbn = "character", member = "character",
loan_date = "Date", return_date = "Date"
)
)
identical(loans_back, loans)
#> [1] TRUE
sum(is.na(loans_back$loan_date))
#> [1] 1
range(loans_back$loan_date, na.rm = TRUE)
#> [1] "2025-01-14" "2026-05-28"
unlink(c(out_path, loans_path))The file shows the point of each choice. row.names = FALSE keeps the first column meaningful. na = "" writes missing
values as empty fields, which is how the raw shelf files look and what a spreadsheet shows as an empty cell; the matching
na.strings = "" on the way back turns the empty text fields (including the ISBN) into NA again. The column classes
are stated, so nothing is guessed and the isbn text, with its hyphens, cannot be mistaken for a number. With those two
settings the cleaned books table is identical() to the original; the loans table, with its two date columns,
round-trips the same way once colClasses names "Date", including the impossible date that is NA in the original and
empty in the file.
Pitfalls
-
Trusting the guess for identifiers. ISBNs, postcodes, phone numbers and account numbers are labels, not quantities.
colClasses = "character"or a named vector keeps their zeros; the guess does not (the007example). -
One bad value turns a column into text. The mean of the column is
NAwith a warning, and arithmetic fails later, far from the read. Look atstr()of every imported data frame; list the spellings of missing inna.strings. -
Leaving the factor trap to old habits.
as.numeric(f)on a factor returns the codes of the levels; the safe conversion isas.numeric(as.character(f))(R FAQ 7.10), and since R 4.0.0 a text column is not a factor unless asked. -
Names that need repair.
check.names = TRUErenamesloan datetoloan.date, and a name that starts with a digit gets anX. R FAQ 7.14 explains what a valid name is;check.names = FALSEkeeps the original, which then needs backticks. -
The default
write.csv()row-name column. Writing a data frame with the default row names and reading it withread.csv()gives an extra columnX. Userow.names = FALSEto write,row.names = 1Lto read. -
Wrong separator, no error. A semicolon file read by
read.csv()is one wide column. Look at the first lines withreadLines()before choosing the reader. -
Silent record splitting. With
fill = TRUEa line with too many fields wraps into a new record; compare the number of rows withlength(readLines())minus the header, and usecount.fields()when they differ. -
An unclosed quote. The warning
EOF within quoted stringis the only signal. Never silence warnings in an import. -
Dates. CSV has no date type.
as.Date()understandsYYYY-MM-DDandYYYY/MM/DD; a day-first date parsed withcolClasses = "Date"is silently wrong. Read as text and give theformat. -
Encoding.
fileEncodingconverts andencodingonly labels. A Latin-1 file in a UTF-8 session fails at the first function that needs characters, not at the read. -
Windows paths. A backslash in a string is an escape character. The R FAQ 7.8 advice is to write
"C:/data/books.csv"or double each backslash; the raw stringr"(C:\data\books.csv)"is the third way since R 4.0.0.
# A Windows path pasted into a string: the backslash starts an escape
code <- "path <- \"C:\\data\\books.csv\""
writeLines(code)
#> path <- "C:\data\books.csv"
tryCatch(parse(text = code), error = conditionMessage)
#> [1] "'\\d' is an unrecognized escape in character string (<input>:1:13)"
# Forward slashes work on every platform, and so do doubled backslashes
nchar("C:/data/books.csv")
#> [1] 17
nchar("C:\\data\\books.csv")
#> [1] 17
# file.path() joins the parts with "/"; raw strings keep a backslash as typed
file.path("data", "shelf", "books.csv")
#> [1] "data/shelf/books.csv"
r"(C:\data\books.csv)"
#> [1] "C:\\data\\books.csv"
# A relative name is looked up in the working directory, a missing file is a
# warning and then an error
file.exists("books.csv")
#> [1] TRUE
file.exists("data/books.csv")
#> [1] FALSE
tryCatch(read.csv("data/books.csv"), error = conditionMessage,
warning = conditionMessage)
#> [1] "cannot open file 'data/books.csv': No such file or directory"
# Tilde expansion is done by file(), so read.csv("~/books.csv") works too
nchar(path.expand("~")) > 1L
#> [1] TRUEThe first call reproduces what R says to the unescaped path: '\d' is an unrecognized escape in character string, with the
position in the text. A path with forward slashes is the same string length as the doubled-backslash form. A missing file
gives the warning cannot open file 'data/books.csv': No such file or directory first (caught here by the warning
handler), and an error cannot open the connection after it when the warning is not caught.
Version notes
The rows for R 3.0.0 come from the NEWS of the 3.x series; the others from the NEWS of the 4.x series.
| Version | Change |
|---|---|
R 3.0.0 |
|
R 4.0.0 |
|
R 4.1.0 |
|
R 4.2.0 |
|
R 4.3.0 |
|
R 4.4.1 |
|
R 4.5.0 |
The data sets |
R 4.5.1 |
|
A search of the release NEWS for the functions of this page found nothing in R 4.6.0 or 4.6.1. The R-devel NEWS (the
future 4.7.0, upcoming and not run here) adds one thing: data() will accept text data files with the further extension
.zst or .zstd and read them on builds with zstd support
(R-devel NEWS).
Where older tutorials differ
Many pages on the web were written before R 4.0.0 or for the idioms of S. When one disagrees with this page, the table says why.
| Older tutorial | Current practice | Why | Version |
|---|---|---|---|
|
leave the default; pass |
the default changed in R 4.0.0, and setting the option to |
R 4.0.0 |
|
nothing |
defunct |
R 4.3.0 |
numeric columns that arrive as factors, |
find the stray value, add it to |
a text column is character, not a factor, so the conversion is not needed unless factors were requested |
R 4.0.0 |
|
|
|
always |
|
|
|
always |
no |
|
FAQ 7.28; guessing costs time and can destroy leading zeros |
always |
a call to |
relative paths from the project directory, or |
a script that sets the working directory cannot run anywhere else |
always |
|
|
an unclosed sink swallows later output |
always |
|
|
|
always |
|
|
|
always |
|
|
|
always |
|
use the name directly; |
the |
always |
|
|
added in R 4.5.0 |
R 4.5.0 |
|
|
no Java or Perl needed; the manual lists the older tools for |
always |
|
haven (or readstata13) |
the manual says |
always |
The first rows can be checked in a session:
# Older tutorials start scripts with options(stringsAsFactors = FALSE). Since
# R 4.0.0 that is the default; the opposite setting is deprecated
caught <- character()
old <- withCallingHandlers(
options(stringsAsFactors = TRUE),
warning = \(w) {
caught <<- c(caught, conditionMessage(w))
invokeRestart("muffleWarning")
}
)
caught
#> [1] "'options(stringsAsFactors = TRUE)' is deprecated and will be disabled"
# ... and read.csv() ignores it, because stringsAsFactors is an argument
str(read.csv(text = "a\nx\ny"))
#> 'data.frame': 2 obs. of 1 variable:
#> $ a: chr "x" "y"
options(old)
getOption("stringsAsFactors")
#> NULL
# default.stringsAsFactors() was deprecated in R 4.2.0 and is defunct in 4.3.0
tryCatch(default.stringsAsFactors(), error = conditionMessage)
#> [1] "'default.stringsAsFactors' is defunct.\nUse '`stringsAsFactors = FALSE`' instead.\nSee help(\"Defunct\")"
# The explicit argument is still the way to ask for factors
str(read.csv(text = "a\nx\ny", stringsAsFactors = TRUE))
#> 'data.frame': 2 obs. of 1 variable:
#> $ a: Factor w/ 2 levels "x","y": 1 2Source: R NEWS: Changes in R 4.0.0
The warning is 'options(stringsAsFactors = TRUE)' is deprecated and will be disabled, read.csv() ignores the option
because its own argument decides, and default.stringsAsFactors() is defunct. For the full list of changes by release
see What’s New and Migrating Older Code.
Where to go next
-
Data Frames for what to do with the table once it is in memory, and Missing and Special Values for the
NArules that run through the import options. -
Factors and Dates and Times for converting the text columns that CSV leaves as text.
-
Strings and Regular Expressions for
trimws(),gsub()andsprintf()in more depth, and Encodings, Locales and Internationalization foriconv(), declared encodings and locale effects. -
Connections and the File System for
file(),gzfile(),url(),textConnection()and the file and path functions. -
Serialization: RDS, RData and JSON for formats that keep R types exactly, and for JSON.
-
Databases with DBI and HTTP and Web APIs for data that lives in a database or behind a URL.
-
Scripts and the Command Line for
Rscript,commandArgs()and standard input in a script. -
The tidyverse and dplyr and data.table for the add-on readers.
References
-
R Data Import/Export: Imports, Encodings, Export to text files, Variations on read.table, Fixed-width-format files, Using scan directly, Importing from other statistical systems, Reading Excel spreadsheets
-
An Introduction to R: Reading data from files, Accessing builtin datasets, Scripting with R
-
R NEWS: Changes in R 4.0.0, 4.1.0, 4.2.0, 4.3.0, 4.4.1, 4.5.0 and 4.5.1; R-devel NEWS; Changes in R 3.0.0
-
read.table, read.csv and relatives, write.table and write.csv, readLines, writeLines, scan, read.fwf, count.fields, type.convert
-
data, datasets, mtcars, penguins, sink, capture.output, readline, cat, format, sprintf, formatC and prettyNum, connections, stdin, as.Date, write
-
RFC 4180 (the CSV format description cited by the write.table help page)
-
readr package documentation: read_delim, vroom package documentation, readxl package documentation: read_excel, haven package documentation: read_spss, data.table package documentation: fread, arrow package documentation (add-on packages, described but not run)