Content Negotiation & HTTP Caching

Negotiation is driven by request headers: Accept (media types), Accept-Language, Accept-Encoding and Accept-Charset. Each header is a comma-separated list of preferences, optionally weighted with a quality value, so text/html, application/json;q=0.9, */*;q=0.1 means "HTML if you can, otherwise JSON, otherwise anything". Express does not implement the matching itself: it delegates to the accepts package, which in turn uses negotiator, so the behaviour is the one defined by HTTP semantics (RFC 9110) and is the same in Express 4 and Express 5.

A cache stores a response keyed by its URL (and method). If the server’s answer also depended on a request header, the cache must know, or it will hand a JSON body to the next client that asked for HTML. Vary lists those request headers: Vary: Accept, Accept-Language tells every cache between the server and the client to store a separate copy per combination of values of those headers. A missing Vary is a classic cache-poisoning bug, and Vary: * effectively disables shared caching.

Express offers three ways to maintain it:

Rules of thumb: add Accept-Encoding if you compress, Origin if CORS echoes the origin, Accept and Accept-Language if you negotiate on them, and Cookie (or better, Cache-Control: private) when the response is personalised. Every field you list multiplies the number of cached copies, so negotiate on as few headers as possible.

HTTP caching has two independent mechanisms that are normally combined:

A response with a freshness lifetime is served from the cache with no network traffic until it expires. After that, the cache sends a conditional request carrying the validators it remembers. If the resource is unchanged the server answers 304 Not Modified with no body, and the cache refreshes the stored copy; otherwise it sends the new representation. The sequence is below. The semantics are defined in RFC 9110 (conditional requests) and RFC 9111 (caching).

An important consequence appears in steps 6 and 7: for a dynamic route, a 304 saves bandwidth, not server work. The handler still runs and res.send() still renders the body, because Express has to know the new ETag to compare it. If the expensive part is loading data, check a cheap validator (a version column, an update timestamp) before loading and answer 304 yourself, as in Complete example: conditional GET, Last-Modified and optimistic concurrency.

An ETag is an opaque identifier for one version of a representation. When a handler finishes with res.send(), res.json() or res.jsonp(), and the response has a body and no ETag header yet, Express computes one from the body and sets the header. It also checks the request, as described in req.fresh and req.stale, and turns the response into 304 when the client’s copy is current.

The application setting etag controls the generation:

Details that surprise people:

req.fresh is true when the client’s cached copy is still valid according to the validators of the response being built, and req.stale is its inverse. Both are getters that read the request’s If-None-Match and If-Modified-Since headers and the response’s ETag and Last-Modified headers, using the fresh package. They are meaningful only for GET and HEAD requests (Express 5.3 extends the same check to the HTTP QUERY method, see its entry in History.md) and only after the response validators are set. They return false when the request carried Cache-Control: no-cache (a forced refresh) or when the status is not a 2xx or 304.

Inside res.send(), Express does the check for you: once the ETag exists it tests req.fresh, and if it is true it sets the status to 304 and strips the entity headers (Content-Type, Content-Length, Transfer-Encoding) and the body. This is why a plain res.json(value) already gives you conditional-GET support. Use the properties directly when you want to short-circuit earlier:

Source: Express API: Request (req.fresh, req.stale)

The comparison rules follow HTTP: If-None-Match uses weak comparison, so W/"abc" and "abc" match; a list of tags or * is allowed; and when If-None-Match is present, If-Modified-Since is ignored.

The following program serves a book with a cheap validator derived from its version number, answers 304 before touching the data store, sets Cache-Control for the representation and uses If-Match to reject lost updates with 412 (precondition failed). Express handles If-None-Match and If-Modified-Since for you through req.fresh; it does not evaluate If-Match or If-Unmodified-Since on dynamic routes, so the handler must.

Source: Express API: Request (req.fresh, req.stale), Express API: Response (res.set, res.vary), RFC 9110: Conditional requests

Run it, then read the validator with curl -i http://localhost:3000/books/1, repeat the request with curl -i -H 'If-None-Match: "book-1-v1"' http://localhost:3000/books/1 to get the empty 304, and send a PUT with a stale If-Match to see 412. Note the Last-Modified timestamp has one-second precision, which is why a precise ETag is the better validator when updates can happen several times per second.

Express adds no caching policy to dynamic responses, apart from validators. Without Cache-Control, browsers apply heuristics (typically a fraction of the time since Last-Modified), which makes behaviour hard to predict, so set the header explicitly with res.set('Cache-Control', ...) or a small middleware such as the cacheFor() factory in Middleware Fundamentals. The directives that matter:

Choosing a policy per kind of response keeps things understandable:

express.static() and res.sendFile() are both built on the send package (Express 5: send 1.x, with serve-static 2.x), and they implement the whole conditional and range machinery without your help: ETag and Last-Modified from the file’s size and modification time, If-None-Match, If-Modified-Since, If-Match, If-Unmodified-Since, If-Range and Range. They answer 304, 412, 206 and 416 as appropriate. The options that control caching:

A common production layout serves hashed assets with a very long lifetime and everything else with revalidation:

Source: Express: Serving static files, Express API: express.static, Express API: Response (res.sendFile)

Serving files is covered in more depth in Serving Static Files. Two caveats apply here. In Express 5 the dotfiles option of express.static() defaults to ignore, which also hides /.well-known, and the hidden and from options were removed. In production, a reverse proxy or CDN is usually a better place to serve and cache static files; the headers set here tell it how.

A Range header asks for part of a representation, used by video players, download managers and resumable downloads. The static and sendFile machinery answers 206 Partial Content with a Content-Range header, 416 Range Not Satisfiable for impossible ranges, and honours If-Range, which resumes only if the validator still matches (hence the preference for strong validators on ranged resources).

For dynamic bodies, req.range(size, options) parses the header. It returns undefined when there is no Range header, -1 when no range is satisfiable, -2 when the header is malformed, and otherwise an array of { start, end } objects (inclusive offsets) with a type property, normally bytes. Pass { combine: true } to merge overlapping ranges. A minimal ranged response for an in-memory buffer:

Source: Express API: Request (req.range)

Real applications stream large files (res.sendFile() or fs.createReadStream()) instead of buffering them. Multiple ranges require a multipart/byteranges response and are rarely worth supporting.

Content negotiation and the etag setting themselves behave the same in Express 4 and 5, so existing caching code usually moves across unchanged; check your use of the removed acceptsLanguage() style methods and of hidden and from options. The migration steps are in Migrating & What Changed.

The four books consulted for this section target Express 4.0 to 4.18 and Node.js 0.10 to 20. They say little about negotiation and caching, but the practices below, common in code of that era, have changed.