Design Philosophy and Coming from Other Languages

R is two things at once: a language very close in appearance to S, and an implementation whose semantics come from Scheme. The R FAQ says exactly that: the design of R was heavily influenced by Becker, Chambers & Wilks' S and by Sussman’s Scheme, and R was initially written by Ross Ihaka and Robert Gentleman at the University of Auckland. Since mid-1997 a core group (the R Core Team) can modify the source archive, and John Chambers, the principal designer of S, is a member of it (R FAQ 2.1).

S is a very high level language and an environment for data analysis and graphics. The R FAQ lists the four books that mark its evolution, which are also the best map of where R’s vocabulary comes from (R FAQ 3.1).

The FAQ treats S as a language with several "engines" (the old S engine, the new S engine and R), so that "the differences between R and S" really means the differences between the engines (R FAQ 3.3). The ones that still matter to a newcomer are these.

Source: R FAQ 3.3.3: Others

Two consequences shape everything that follows. First, because the semantics are Scheme’s, R functions are first-class values with lexical scope, and the language is built from a small number of ideas applied everywhere. Second, because the surface syntax is S’s, code that looks familiar to a C or Python programmer often means something different: assignment is <-, a single number is a vector, and x[1] is the first element rather than the second.

John Chambers, the designer of S, summarised the language in two slogans in his book Extending R (2016): everything that exists in R is an object, and everything that happens in R is a function call. The official manuals ground both halves.

The consequence is that nothing in R is syntax in the way +, [ or if are syntax in Java. Each of them is a function with a name you can call, pass around or redefine, and the infix and bracket forms are only a convenient way of writing the call. That is why a function name can stand where a value is expected (the script calls vapply() with the function - as an argument), and why S3 and S4 methods can be written for operators, [, $ and length().

Source: The R Language Definition: Function calls

The backquotes protect a name that is not a syntactic name. The assignment x[2L] <- 99 and the explicit call to the replacement function [<- in the script are the same operation: the replacement function receives the object and the new value and returns the modified object, which is then assigned back to x. That is why the right-hand-side argument of a replacement function has to be called value (R FAQ 3.3.3). Even if is a function here, which explains why if returns a value and can be used inside an expression. See Functions for replacement functions you write yourself and Object-Oriented Programming Overview and Base Types for how every object has a type that typeof() reports.

The basic way to compute in R is to write one expression over a whole vector. The manual introduces arithmetic on vectors as element-by-element operations, and recycling extends this to operands of different lengths: the shorter vector is reused until it matches the longer, with a warning when the longer length is not a multiple of the shorter (R-lang: Recycling rules and R-intro: Vector arithmetic). Comparison, logical operators, ifelse(), sum(), mean() and most mathematical functions are vectorised the same way. The loop still exists, but it runs inside compiled code, so the idiomatic R program has far fewer visible loops than the equivalent Java, C# or Python without NumPy.

Source: R documentation: Arithmetic operators

Three things in this output are worth noticing. The explicit loop and the vectorised line give an identical() result, so the loop adds nothing but length. mean(books$pages) returns NA because two books have a missing page count and missing values propagate through arithmetic; na.rm = TRUE is the explicit opt-in to ignore them (see Missing and Special Values). And books$pages > 300 contains NA for those two books too, which is why the example wraps sum() in na.rm = TRUE and selects titles with which(), which drops NA positions. Indexing with a logical vector that contains NA returns NA rows, a trap taken up below.

Vectorisation also explains why the most common R performance advice is "do not grow a vector in a loop" and "replace the loop with a vector operation". The pages Vectorisation and Iteration and The apply Family and Functionals cover the cases that cannot be vectorised (vapply(), Map(), Reduce()).

R has no scalar type. A single number such as 4.2 or a string such as "four point two" is a vector of length one, and the manual states there are no more basic types than the vector types (R-lang: Vectors). Vectors of length zero are possible and useful: numeric(0) is the result of selecting nothing, and NULL is the length-zero object that stands for "no value" (see below). The six atomic vector types are logical, integer, double, complex, character and raw.

Indexing starts at 1, as it does in MATLAB, Fortran and Julia and unlike Java, C#, C and Python. A positive index selects that position, a negative index drops it, 0 selects nothing, and an index past the end yields NA for [ and an error for [[ (R-intro: Index vectors).

Source: R documentation: vector and is.vector

A string is one element of a character vector, so length("shelf") is 1; nchar() counts characters, and strsplit() or substr() take a string apart. Programmers from Java or Python often write length(s) and are surprised by the 1. See Atomic Vectors and Types and Subsetting and Indexing for the full rules ([, [[, $, negative and logical indices, drop).

Assigning a vector to a second name does not copy it, and modifying one of the names does not change the other. From the programmer’s side R has value semantics: the manual describes argument passing as call-by-value, so changing a supplied argument inside a function does not affect the variable in the calling frame (R-lang: Argument evaluation), and it adds that some extra effort is made to avoid unnecessary duplication of data (R-lang: Function calls). Internally R shares the storage between names until one of them is modified, and copies at that moment. R 4.0.0 replaced the older NAMED mechanism by reference counting to decide when an object can safely be mutated, which, according to the NEWS for R 4.0.0, reduces the need for copying in some cases.

tracemem() marks an object so that a message is printed whenever the internal code copies it. The message holds the addresses of the old and the new object in hex, followed by a stack trace (empty at the top level); tracemem() itself returns the address enclosed in angle brackets. The addresses are different on every run, so the output below shows 0x... in their place. tracemem() is available only when R was compiled with memory profiling; the help page says it is enabled in the CRAN macOS and Windows builds, and it is in the Linux build used for these examples.

Source: R documentation: tracemem

No message appeared for y <- x, because no copy was made. The message appeared at y[[1L]] <- 99, when the shared vector had to be duplicated, and afterwards x still held its original values. The same thing happens when a function modifies its argument: bump() works on a private copy and returns it, so the caller keeps both a and b. The exception is the environment. The language definition says that environments are not copied when passed to functions or assigned, so if you assign the same environment to several names and change one, the others change too (R-lang: Environments); counter above is modified in place by increment(). Reference Classes and R6 objects are environments, which is why they have the reference semantics that S3 lists do not.

Copy-on-modify is the cost side of the design. Growing a vector one element at a time, or modifying a large data frame inside a loop, copies the data many times. The Writing R Extensions manual describes tracemem() as one of three memory-profiling methods (R-exts: Tracing copies of an object). The page Memory and Copy Semantics goes further (reference counting, ALTREP, in-place modification and object.size()).

When a function is called, its arguments are matched and each formal argument is bound to a promise, an object that holds the expression the caller wrote and the environment to evaluate it in. The promise is evaluated the first time the function needs the value, and the result is saved, so it is evaluated at most once (R-lang: Promise objects). An argument that is never used is never evaluated. Default arguments are promises whose environment is the function’s own frame, which is why a default can refer to another argument or to a local variable created later (R-lang: Argument evaluation).

Source: R documentation: force

The default end = start + days in loan_end() was not evaluated when the function was entered. By the time end was returned, the line days <- days * 2L had run, so the result is 42 days after the start, not 21. missing() tells a function whether the caller supplied an argument, which is how R expresses optional parameters without overloading. force() evaluates an argument immediately; it matters in function factories, where a promise left unevaluated would capture a variable that later changes. delayedAssign() makes a promise by hand. Because the promise remembers the caller’s expression, substitute() can recover it, which is the basis of non-standard evaluation, covered in Lazy Evaluation and Promises and Evaluation and Non-Standard Evaluation.

The practical consequences for a programmer from a strict language are: do not rely on side effects in arguments (the manual calls it bad style to use arguments to cause side effects, since there is no guarantee they are evaluated), an error in an argument can surface far from the call, and checking the arguments up front with stopifnot() or an explicit validation call is a common defensive habit.

Functions are ordinary values. They can be stored in lists, passed to other functions, returned from them and created anywhere. Since R 4.1.0 the shorthand \(x) is an alternative to function(x), which makes short anonymous functions as compact as a Python lambda or a Java x -> ... expression. A function created inside another function is a closure: it carries a pointer to the environment where it was created, and free variables are looked up there first. That is the lexical scoping the R FAQ lists as the major difference from S (R-lang: Lexical environment).

Source: R documentation: environment

<<- assigns to a variable in an enclosing environment, which is how make_counter() keeps state between calls; the default lintr style flags it, as the # nolint comment shows, because it is easy to overuse. local() evaluates its body in a new environment, so a and b never reach the workspace. The last two lines show that the global environment is where interactive assignments land, and that a function from a package, such as sd(), has the package namespace as its environment. The R FAQ separates the two environments a function can see: the enclosing environment, fixed when the function is created and returned by environment(f), and the parent environment, which is where the function was called from and is returned by parent.frame() (R FAQ 7.12). Variables are resolved through the enclosing chain, while things like model formulae are evaluated in the caller’s frame. The page Scoping, Environments and Closures covers the search path, sys.function() and the call stack, and the page Function Factories and Higher-Order Functions covers the patterns built on them.

R has more than one way to define classes, because the language grew them in layers. The choice is a design decision, and a newcomer meets all of them quickly because the standard library uses S3 and S4 and popular packages use R6 and S7.

The example creates the same domain concept three ways with base R only. The book class is the S3 class of the scenario; Loan is an S4 class with a generic and a method; and shelf_class is a Reference Class whose methods can be chained.

Source: R documentation: setClass

Note how b is a list underneath (is.list(b) is TRUE) with a class attribute "book", which is why S3 objects can be inspected with unclass() and why nothing prevents a user from modifying a field directly. The S4 object has typed slots read with @. The Reference Class object s was changed through t2 as well, because t2 <- s copies the reference, not the object. See Object-Oriented Programming Overview and Base Types for how to choose between them.

The mapping below pairs one concept per row with the form it takes in each language and in R. It is a lookup table, not a translation guide: the right-hand column is the R way of expressing the idea, which is not always the literal translation of the left-hand ones. Entries marked "dialect" differ between database products. Each R construct is explained on the page named in the section it belongs to.

Source: R documentation: Arithmetic operators

The next two scripts run the R column against the shelf data. The first filters, groups and joins with base R; the second shows the arithmetic and string operators that differ most from the other languages.

Source: R documentation: subset

Two details of this output repay attention. books[!is.na(books$pages) & books$pages > 300, ] has to exclude the missing page counts explicitly, because indexing with NA returns an all-NA row; subset() removes them for you and lets you write bare column names. And merge() produced 59 rows from 60 loans because one loan carries a malformed ISBN (the scenario has deliberately dirty data); a SQL inner join behaves the same way. The chain written with |>, the native pipe added in R 4.1.0, passes the left-hand side as the first argument of the next call; see Pipes and the Native Pipe. The script uses tapply() and aggregate() from base R and mentions dplyr only in the table: see The Tidyverse and dplyr and data.table for the two big alternatives, and Data Frames for the base-R data frame.

Source: R documentation: Arithmetic operators

/ on two integers is always floating point (5L / 2L is 2.5), and %/% on two integers stays an integer. %/% and %% round toward negative infinity, so -7 %/% 2 is -4 and -7 %% 2 is 1, matching Python and unlike Java and C#, where % takes the sign of the dividend. See Numbers and Arithmetic and Operators and Precedence for the details.

R was designed for data analysis, and it shows in both directions. The following list is a summary of design consequences, each with the page that covers it.

Most first-week errors in R are consequences of the design above. The messages below were produced by the script, with try(silent = TRUE) capturing what the console would have printed, so the text is the real text of R 4.6.1.

Source: R documentation: Control flow

The if and && rows are the most visible change for people who learned R from older material. Until R 4.1 an if on a length-greater-than-one condition warned and used the first element; the NEWS entry for R 4.2.0 turned it into an error and made &&/|| with a longer operand a warning; R 4.3.0 turned that into an error with the "length = 4 in coercion" message (NEWS: Changes in R 4.2.0 and R 4.3.0). See Control Flow and Conditions and Error Handling for the machinery.

The R FAQ’s chapter 7 collects questions that bite newcomers. The ones below follow directly from the design choices described earlier.

A large part of the R material on the web predates R 4.0 and shows habits that are now discouraged or invalid. When a tutorial and this reference disagree, the version in the table is the one that explains why.

Source: R documentation: data.frame

Source: R documentation: seq and seq_along

The fragment below is not a runnable script; it shows what a leftover T does in the older style. With T still bound to TRUE the call works. After an unrelated line assigns T, the same call silently changes meaning, because na.rm = 0 is treated as FALSE. With the explicit TRUE, the problem cannot occur, which is why the FAQ tells writers to always spell the constants out.

Source: R FAQ 3.3.3: Others

The page What’s New and Migrating Older Code collects every change by release, from stringsAsFactors in 4.0.0 to the upcoming features of R-devel.

For the same question asked in other languages of this site, see the Ruby page on design philosophy and Fortran for Programmers of Other Languages, which has a similar structure.