File Upload & Streaming

Source: NestJS documentation: File upload and streaming, "Fastify".

On Fastify the controller keeps its shape; only the import path and the file type change. This example also adds the streaming interceptor discussed later.

Source: NestJS documentation: File upload and streaming, "Fastify".

The magic-number check reads the file’s bytes, so it only works when the file is in memory (file.buffer). With disk storage there is no buffer. With the module-wide disk storage described below, the cover route’s validator answers with this message:

That is why the controller above overrides the storage on that route with memoryStorage(): covers are small, and verifying the real type is worth the memory. For large files stored on disk, use skipMagicNumbersValidation: true and accept that only the client-provided MIME type is compared (documented on the official page), or validate after the fact. The type declarations of the installed 12.1.2 package additionally list fallbackToMimetype (fall back to the MIME type when the magic-number check fails), overrideMimeType (replace the client’s MIME type with the detected one) and errorMessage (a string, or a function of the validation context) on FileTypeValidatorOptions; these are not described on the official page, so check the declarations of the version you install before relying on them.

Subclass FileValidator for your own rules. The validator receives the file (or, for @UploadedFiles(), an array or an object of arrays), so narrow the type before using it:

Without a file the route answers 201 {"received":false}; with a 3-byte file it answers 400 {"message":"a cover must be at least 8 bytes", …​}. To change the error body, pass exceptionFactory; on Fastify the route below returns 422 {"reason":"Validation failed (current file type is text/plain, expected type is text/csv)"} for a text file posted to a CSV-only route:

When you use several validators in a reusable form, define the array once in a constant (for example fileValidators) and share it across routes. The same pipe works on arrays of files: pass a pipe built with ParseFilePipeBuilder to @UploadedFiles() and every validator runs against each file, as in .addFileTypeValidator({ fileType: /^image\/(png|jpeg|gif)$/ }) with FilesInterceptor('files', 5).

Source: NestJS documentation: File upload and streaming, "File validation" and Validating an array of files.

The validation pipe is a parameter pipe, so it runs after the upload interceptor has already stored the file. With memory storage that costs only memory. With disk storage, a file that is rejected by ParseFilePipe stays on disk: in the Express run, a request rejected by CoverValidator with the 400 above left one file in the upload directory, whereas a request rejected by the interceptor itself (four files against maxCount 3) left none. Rely on limits and fileFilter (next sections) to reject early, and clean up rejected files in an exception filter or a scheduled sweep of the upload directory.

On Express, diskStorage and memoryStorage come from the multer package (import { memoryStorage } from 'multer' and import multer from 'multer'; multer.diskStorage(…​) both work in the ESM layout). On Fastify they are exported from @nestjs/platform-fastify/multipart. Multer’s own diskStorage() and third-party Multer storage engines can be passed as storage to the Fastify interceptors unchanged, because they follow the same _handleFile() / _removeFile() contract.

The Fastify documentation shows disk storage with a timestamped name. Do not use the client’s originalname as the stored name without sanitizing it; the examples here generate a UUID and keep only the extension.

Source: NestJS documentation: File upload and streaming, "Storage".

To set defaults for every interceptor of a module instead of repeating options on each route, import MulterModule (Express) or MultipartModule (Fastify, 12.1). Both have register() and registerAsync(); registerAsync() supports useFactory (with imports and inject), useClass (a class implementing MulterOptionsFactory with createMulterOptions(), or MultipartOptionsFactory with createMultipartOptions()) and useExisting. MulterModule.registerAsync() also accepts extraProviders. The upload folder comes from configuration, never a literal path, so the same code runs in every environment:

The Fastify module is the same with different imports and names:

With these modules, the gallery route stores its files on disk under UUID names (.png files in the upload folder), and a 1 MiB + 10 byte upload is rejected before reaching the handler with 413 {"message":"File too large","error":"Payload Too Large","statusCode":413} on both adapters. A file named x.exe is skipped by the fileFilter, so the handler receives no file and ParseFilePipe answers File is required (422). Note that fileFilter only sees the client-declared name and MIME type; it is a convenience filter, not a security control. Defaults can also be given statically:

Source: NestJS documentation: File upload and streaming, "Default options", Async configuration and Fastify.

@fastify/multipart applies stricter defaults than Multer, which limits nothing. Unless configured, the maximum file size (fileSize) is the Fastify bodyLimit (1 MiB) and the maximum number of parts (parts) is 1000. Larger files are rejected with 413 File too large. Raise the limits at the level that fits:

Limits set on a route are merged over the ones from MultipartModule key by key, and both are merged over the application-wide ones. The only exception is fieldNameSize, which the interceptors do not read from the plugin options.

A 2 MiB upload to a route without module or route limits fails with 413 File too large under the defaults and succeeds (201 {"size":2097152}) with the adapter option set to 3 MiB, while a route-level limits: { fileSize: 16 } rejects a 100-byte file in both cases. The module-wide 1 MiB limit above still wins over a 10 MiB adapter-wide option, which is the merge order described above.

Source: NestJS documentation: File upload and streaming, "Limits".

The multipart option of FastifyAdapter controls registration of @fastify/multipart:

Set the option to true (or an options object) when an upload interceptor is instantiated outside dependency injection, such as new (FileInterceptor('file'))() passed to app.useGlobalInterceptors(), or when upload interceptors live only in lazy-loaded modules that load after start-up, since plugins can no longer be registered then. To register the plugin yourself, pass it to app.register(); your options take precedence over the multipart option:

Registering the plugin directly on the Fastify instance (app.getHttpAdapter().getInstance().register(multipart)), or passing a dynamic import() of it to app.register(), only works before app.init() (which app.listen() calls). After that, Fastify fails with FST_ERR_DEC_ALREADY_PRESENT. With multipart: false the application refuses to start with an error stating that the interceptors require the plugin, which the adapter registers unless multipart is false.

Source: NestJS documentation: File upload and streaming, "Plugin registration".

Besides the default limits, the Fastify interceptors differ from Multer in these ways, which matter when migrating or sharing code between adapters:

Source: NestJS documentation: File upload and streaming, "Differences from Multer".

On Fastify, FileStreamInterceptor(fieldName, options?) hands the handler a MultipartFileStream, a file with a stream property instead of buffer. Its options are the same as for the other interceptors except dest and storage.

Source: NestJS documentation: File upload and streaming, "Streaming uploads".

Because the file is never buffered, a few rules apply:

Multer has no streaming mode and the Nest docs describe no Express counterpart of FileStreamInterceptor. If you must stream on Express, send the file as the request body with a content type that Nest’s default parsers leave alone: only json and urlencoded are registered by default, so a request with Content-Type: application/octet-stream reaches the handler as an unread stream, and @Req() is a Node.js readable you can pipe. This is plain Node.js streams, not a Nest feature; it trades the form fields and the interceptor’s limits for simplicity, so enforce a size limit yourself (for example with a guard on Content-Length, or by counting bytes in a Transform).

The fixed file name above is only for the demonstration; a real route should generate a unique name per request. A multipart/form-data streaming alternative on Express is a streaming parser such as Busboy used directly, which is outside what Nest documents.

Source: NestJS documentation: Raw body, "Registering a different parser" (the default parsers) and Streaming uploads.

When headers depend on the request, inject the response with @Res({ passthrough: true }). Without passthrough: true Nest hands the response over to you and would not send the returned StreamableFile. The response object differs per platform:

Both versions return the catalog-dynamic.json disposition. The Express snippet is from the official page; the Fastify one replaces res.set() with res.header().

Source: NestJS documentation: File upload and streaming, "Response headers".

With the Express adapter, if the stream emits an error before any data has been sent, the client receives 400 Bad Request with the error message as its body (for a missing file, the message is the ENOENT text, which also leaks the server path, so replace it). If data has already been sent, the response is ended. Customize this with setErrorHandler(); setErrorLogger() replaces the function that logs errors emitted by the response stream.

With a missing report.pdf this answers 404 Report not found, while the same handler without setErrorHandler() answers 400 with the ENOENT message. With the Fastify adapter, a stream error is handled by the exceptions layer, so the default response is 500 {"statusCode":500,"message":"Internal server error"}; error handler and logger apply to the Express adapter. The portable approach is to check that the file exists, with stat() or by opening it, before creating the StreamableFile, and throw NotFoundException yourself.

Fastify can send streams without stream.pipe(res), so StreamableFile is not strictly required there; Nest supports it on both platforms, so code that returns it keeps working if you switch adapters.

Source: NestJS documentation: File upload and streaming, "Stream errors" and Cross-platform support.

Raw body is about the body parsed by the built-in parsers. To accept another content type, register a parser; to accept larger JSON or form bodies, raise the limit (Express default 100 kB, Fastify default 1 MiB):

With the larger limit, a 2 MiB JSON body is accepted and reaches route validation. Body limits are separate from upload limits: Multer and @fastify/multipart read the request themselves, so raising the JSON limit does not raise the maximum file size, and on Fastify the file limit defaults to the body limit only when no fileSize limit is set. With the same bootstrap, a 2 MiB upload still fails with 413 because the module-level fileSize limit applies. The platform sections of a production setup, including bodyLimit for the whole Fastify server, are covered on Fastify & HTTP Platforms.

Source: NestJS documentation: Raw body, "Registering a different parser" and Body parser size limit.