Image Processing with S3, Lambda and CloudFront
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. See the section bibliography for the reference material consulted while preparing these pages. |
Baseline date: 2026-10-07. Bookshelf already has a thumbnailer Lambda function that S3 triggers when a cover is uploaded
to bookshelf-uploads-111122223333, see
Lambda: event-driven apps and
S3 object storage. This page makes that function production-shaped, and shows the
on-demand alternative. The variants to produce and the formats are in
responsive images and thumbnails and
image formats and encoding.
Eager resizing: S3 triggers Lambda
Use a separate destination bucket (here bookshelf-thumbs-111122223333). The existing Bookshelf example filters on the
originals/ prefix and writes to thumbnails/ in the same bucket, which is safe only while the prefix filter excludes the
output, and a separate bucket removes that risk. If the function writes into the bucket that
triggers it, every output triggers another invocation, which is an infinite loop that is also a cost incident. The AWS
tutorial does the same: see the tutorial: using an Amazon
S3 trigger to create thumbnail images. The separate bucket also lets the public side stay read-only and private to
CloudFront, see CloudFront.
Python with Pillow in a layer
import io
from urllib.parse import unquote_plus
import boto3
from PIL import Image, ImageOps
s3 = boto3.client("s3")
DEST_BUCKET = "bookshelf-thumbs-111122223333"
WIDTHS = (320, 640, 1280)
FORMATS = {"webp": ("WEBP", {"quality": 78, "method": 5}), "jpg": ("JPEG", {"quality": 80, "optimize": True})}
def handler(event, context):
for record in event["Records"]:
bucket = record["s3"]["bucket"]["name"]
key = unquote_plus(record["s3"]["object"]["key"]) # for example covers/b81f/original.jpg
prefix = key.rsplit("/", 1)[0]
body = s3.get_object(Bucket=bucket, Key=key)["Body"].read()
with Image.open(io.BytesIO(body)) as image:
image = ImageOps.exif_transpose(image).convert("RGB") # orientation, then sRGB, no metadata
for width in WIDTHS:
target = min(width, image.width) # never upscale
resized = image.resize((target, round(image.height * target / image.width)), Image.Resampling.LANCZOS)
for ext, (fmt, options) in FORMATS.items():
out = io.BytesIO()
resized.save(out, fmt, **options)
s3.put_object(
Bucket=DEST_BUCKET, Key=f"{prefix}/w{width}.{ext}", Body=out.getvalue(),
ContentType=f"image/{'webp' if ext == 'webp' else 'jpeg'}",
CacheControl="public, max-age=31536000, immutable",
)
AVIF is omitted because it is slow to encode in a function with limited CPU, and Pillow’s AVIF support depends on how the
wheel was built (see the Python page). Use a container
image function, or a queue and a worker, for AVIF. Build Pillow into a layer for the function’s platform with the
pip install --platform command shown on the frame
capture page. Size the function: image work is CPU-bound, and Lambda allocates CPU in proportion to memory, so 1024 to
2048 MB is a sensible start, with a timeout of 30 to 60 seconds. Reference:
layers for Python functions.
Node.js with sharp
npm install --cpu=x64 --os=linux sharp
import { S3Client, GetObjectCommand, PutObjectCommand } from '@aws-sdk/client-s3';
import sharp from 'sharp';
const s3 = new S3Client({});
const DEST_BUCKET = 'bookshelf-thumbs-111122223333';
export const handler = async (event) => {
for (const record of event.Records) {
const bucket = record.s3.bucket.name;
const key = decodeURIComponent(record.s3.object.key.replace(/\+/g, ' '));
const prefix = key.slice(0, key.lastIndexOf('/'));
const { Body } = await s3.send(new GetObjectCommand({ Bucket: bucket, Key: key }));
const original = Buffer.from(await Body.transformToByteArray());
for (const width of [320, 640, 1280]) {
const base = sharp(original).rotate().resize({ width, withoutEnlargement: true });
const variants = { webp: await base.clone().webp({ quality: 78 }).toBuffer(),
jpg: await base.clone().jpeg({ quality: 80, mozjpeg: true }).toBuffer() };
for (const [ext, data] of Object.entries(variants)) {
await s3.send(new PutObjectCommand({
Bucket: DEST_BUCKET, Key: `${prefix}/w${width}.${ext}`, Body: data,
ContentType: ext === 'webp' ? 'image/webp' : 'image/jpeg',
CacheControl: 'public, max-age=31536000, immutable',
}));
}
}
}
};
sharp ships a prebuilt libvips binary per platform, so install it for Linux and the function’s CPU architecture
(--cpu=arm64 for Graviton), and package it in the deployment zip or in a layer, as the
sharp installation notes for AWS Lambda describe. Reference:
building Lambda functions with Node.js.
Operating it
-
Give the function role read on the source and write on the destination bucket only.
-
Configure an on-failure destination or a dead-letter queue, and send S3 events through SQS when you need retries with backoff, see Messaging and Integration.
-
S3 events are delivered at least once, so keep the function idempotent: deterministic output keys, overwritten on retry.
-
Limit concurrency with reserved concurrency so a bulk upload does not exhaust the account limit.
On-demand resizing: Dynamic Image Transformation for Amazon CloudFront
Eager resizing needs a decision about every width in advance. The AWS Solution
Dynamic
Image Transformation for Amazon CloudFront (formerly Serverless Image Handler) resizes on the first request and lets
CloudFront cache the result. It is deployed from a CloudFormation template and consists of a CloudFront distribution, an
API Gateway, a Lambda function based on sharp and your S3 source bucket. The request carries the transformation, for
example as a base64-encoded JSON body in the path or as query parameters, and it can choose the output format from the
Accept header, so a single URL serves AVIF or WebP or JPEG. It supports resizing, cropping, smart crop with Amazon
Rekognition, format conversion and quality.
-
Choose it when you have many images or sizes that change, and do not want to pre-generate every variant.
-
Restrict the allowed transformations (whitelist sizes, or sign the URLs), otherwise anyone can request every size of every image at your cost, see security and operations.
-
Because CloudFront caches the result, add
Vary: Accepthandling (the solution forwards theAcceptheader into the cache key when automatic format selection is on) and use versioned source keys instead of invalidating.
The AWS image optimization sample shows a lighter pattern: CloudFront in front of a Lambda function URL (or an S3 origin) that resizes and converts with sharp, and stores the transformed images in a second bucket as a cache. Treat samples as a starting point, since they are not maintained like a product.
S3 Object Lambda
S3 Object Lambda also transforms an object as it is read, which used to be a way to resize images on the fly behind S3. As AWS states, S3 Object Lambda is closed to new customers as of November 2025, so do not build new designs on it, and use CloudFront with a Lambda function (as above) instead. Existing users should read the AWS announcement and plan a migration. Reference: transforming objects with S3 Object Lambda.
Which one
| S3 trigger and Lambda (eager) | Dynamic Image Transformation (on demand) | |
|---|---|---|
Variants |
A fixed list per image, produced at upload. |
Any size, produced at the first request. |
Latency |
None at read time. The first view after upload may wait for processing. |
The first request for a size is slow, later ones are cached. |
Cost |
Compute once per image, storage for every variant. |
Compute per distinct request, storage in the cache only. |
Risk |
Backfill needed when sizes change. |
Open endpoint if not restricted. |