Performance, Benchmarking and Profiling

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

This page is written against Ruby 4.0 (4.0.7, checked on 2026-10-18); features newer than Ruby 3.0 name the version that introduced them. Performance work in Ruby follows the same discipline as in any language: decide what "fast enough" means, measure the program as it is, find the part that dominates, change that part, and measure again. This page gives the tools for each step: clocks and the Benchmark library for timing, community benchmark runners for stable comparisons, GC.stat and ObjectSpace for counting allocations, sampling profilers for finding hot spots, and a way to put performance expectations into a test suite without making it flaky. The two biggest levers, algorithmic choices and caching (memoization), get their own sections, because they usually beat any micro-optimisation by orders of magnitude. The JIT compilers and the virtual machine are in YJIT, ZJIT and the VM; the garbage collector, its tuning variables and memory bloat are in Garbage Collection and Memory; what the GVL means for threaded speed-ups is in Threads and the GVL.

Every example on this page was syntax-checked with ruby -wc, run with ruby -w on Ruby 4.0.7 and linted with RuboCop 1.92 using the configuration of the running shelf gem (target Ruby 4.0, double-quoted strings). The values shown after # => were checked by running the file; where a file prints something, the output block is the real output. Timings are illustrative of one ten-core Apple-silicon Mac and will differ on other machines and between runs: read them as orders of magnitude, never as promises. Each file is self-contained, with a trimmed copy of the shelf classes it uses.

A measuring workflow

Step What to do

Step	What to do
1. Set a target	Write down the goal in the unit users care about (a request in 200 ms, an import of a million books in a minute, a process under 512 MB). Without a target there is no end to optimising.
2. Reproduce	Build a representative, repeatable workload: realistic data sizes, the same Ruby version, the same flags (JIT on or off), a production-like environment. A benchmark of 10 records says nothing about 10 million.
3. Measure	Time the whole operation first (a monotonic clock or `Benchmark.realtime`), then find where the time goes with a profiler. Count allocations when memory or garbage-collection time is the suspect.
4. Change one thing	Change the dominant cost only, and keep the change small enough to attribute the result to it.
5. Measure again	Compare against the same workload, repeat to see the variance, and keep the number. If the gain is below the noise, revert.
6. Guard	Keep a cheap, stable test (an allocation budget, an algorithmic bound) so the regression does not come back silently.

1. Set a target

Write down the goal in the unit users care about (a request in 200 ms, an import of a million books in a minute, a process under 512 MB). Without a target there is no end to optimising.

2. Reproduce

Build a representative, repeatable workload: realistic data sizes, the same Ruby version, the same flags (JIT on or off), a production-like environment. A benchmark of 10 records says nothing about 10 million.

3. Measure

Time the whole operation first (a monotonic clock or Benchmark.realtime), then find where the time goes with a profiler. Count allocations when memory or garbage-collection time is the suspect.

4. Change one thing

Change the dominant cost only, and keep the change small enough to attribute the result to it.

5. Measure again

Compare against the same workload, repeat to see the variance, and keep the number. If the gain is below the noise, revert.

6. Guard

Keep a cheap, stable test (an allocation budget, an algorithmic bound) so the regression does not come back silently.

The order matters. Profile before you optimise: the line that looks slow is rarely the one that is. Optimise the algorithm before the code, then the allocation pattern, then the expression-level details, and only then consider the VM (YJIT) or more processes. The books this section draws on agree on this order; what they do not always show is how much the answers depend on the Ruby version, which is why the version and build are part of the workload description above.

Timing code

Clocks

Measure durations with a monotonic clock. Process.clock_gettime(Process::CLOCK_MONOTONIC) returns seconds as a Float from an arbitrary starting point that never goes backwards. Time.now reads the wall clock, which the operating system or an operator can move (NTP corrections, daylight-saving shifts handled by tools, manual changes), so subtracting two Time.now values can give a negative or wrong duration. Other clock ids measure CPU time instead of elapsed time: Process::CLOCK_PROCESS_CPUTIME_ID counts the CPU seconds of the whole process and Process::CLOCK_THREAD_CPUTIME_ID those of the calling thread, which separates computing from waiting (a thread blocked on a socket accumulates wall time but no CPU time). A third argument selects the unit: Process.clock_gettime(Process::CLOCK_MONOTONIC, :millisecond) returns an Integer in milliseconds, and :float_millisecond, :microsecond, :nanosecond and the other units exist too. See also the time guidance in Time and Dates and the mistake of timing with Time.now in Common Mistakes and Gotchas.

Source: Process.clock_gettime

The Benchmark library

Benchmark wraps those clocks in a small API. In Ruby 4.0 it is a bundled gem, not a default gem: it still installs with Ruby, so a plain ruby script.rb can require "benchmark" with no warning, but under Bundler the gem must be in the Gemfile (or the gemspec), exactly like logger or csv. Ruby 4.0’s own NEWS lists benchmark 0.5.0 among the bundled gems, and the warning that appears when it is missing is explicit. The check was run on 4.0.7 with a Gemfile that does not list it:

$ bundle exec ruby -e 'require "benchmark"'
-e:1: warning: benchmark used to be loaded from the standard library, but is not part of the default gems since Ruby 4.0.0.
You can add benchmark to your Gemfile or gemspec to fix this error.
... in 'Kernel.require': cannot load such file -- benchmark (LoadError)

Source: NEWS for Ruby 4.0 (bundled gems)

After adding gem "benchmark" to the Gemfile the same command prints nothing and Benchmark::VERSION is "0.5.0". On Ruby 3.3 and 3.4 the same gem is a default gem (checked with Gem::Specification#default_gem?), which is why code written for those versions fails only after the upgrade. The reference documentation of the library moved with it: the 4.0 documentation site has no Benchmark page, and the last core-library page is the 3.4 one linked in the references.

The methods, from smallest to largest:

Benchmark.realtime { ... } returns the elapsed wall-clock seconds as a Float. It is the right tool for "how long did this take", for logging, and for assertions that are not about speed.
Benchmark.measure { ... } returns a Benchmark::Tms with utime and stime (CPU time in user and kernel mode), cutime and cstime (children), total (the CPU sum) and real (wall clock). Tms supports arithmetic (+, -, *, /) and format, so results can be summed over several runs. Benchmark.ms is the millisecond counterpart of realtime.
Benchmark.bm(label_width) { |x| x.report("label") { ... } } prints a header and one row per report with the user, system, total and real columns. Reports run in the order written, so the first one pays for cold caches and for any lazy initialisation.
Benchmark.bmbm runs every report twice: a "rehearsal" pass whose result is discarded (to warm up and trigger the garbage collector) and then the measured pass. Use it instead of bm whenever the reports are close. (Benchmark.benchmark is the general form with a custom caption and format.)

The example compares finding a book by ISBN with a scan against a hash lookup, using the four forms.

# frozen_string_literal: true

require "benchmark"

# Two ways to find a book by ISBN among 5,000 books.
Book = Data.define(:isbn, :title)
books = Array.new(5_000) { |i| Book.new(isbn: format("978%010d", i), title: "Title #{i}") }
wanted = books.last.isbn

# Process.clock_gettime with a monotonic clock is the primitive for durations.
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
books.find { |book| book.isbn == wanted }
elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started
elapsed.class # => Float

# Benchmark.realtime returns that elapsed time (seconds) for a block.
Benchmark.realtime { books.find { |book| book.isbn == wanted } }.class # => Float

# Benchmark.measure returns a Tms with CPU and wall-clock times.
tms = Benchmark.measure { 200.times { books.find { |book| book.isbn == wanted } } }
tms.class # => Benchmark::Tms
tms.real.negative? # => false

# Benchmark.bm prints one row per report; bmbm runs the code twice to warm up.
by_isbn = books.to_h { |book| [book.isbn, book] }
Benchmark.bmbm do |x|
  x.report("Array#find") { 200.times { books.find { |book| book.isbn == wanted } } }
  x.report("Hash#[]") { 200.times { by_isbn[wanted] } }
end

Output, produced by ruby -w timing.rb (the times are illustrative of this machine):

Rehearsal ----------------------------------------------
Array#find   0.029343   0.000007   0.029350 (  0.029351)
Hash#[]      0.000011   0.000000   0.000011 (  0.000010)
------------------------------------- total: 0.029361sec

                 user     system      total        real
Array#find   0.030193   0.000009   0.030202 (  0.030203)
Hash#[]      0.000010   0.000000   0.000010 (  0.000009)

The scan is three orders of magnitude slower here, and the ratio would grow with the catalogue: Array#find is linear, Hash#[] is constant on average. No amount of micro-tuning of the block would close that gap, which is the point of the next section.

The user column is CPU time spent in Ruby and native code, system is kernel time (I/O, memory mapping), total is their sum and the parenthesised real is elapsed time. A program that waits (sleeping, reading a socket, joining a thread) shows a real far larger than total; a program that parallelises across cores in child processes shows children’s time only in cutime and cstime.

Source: Benchmark (last core-library page; bundled gem since 4.0, source)

benchmark-ips and benchmark-memory

Benchmark.bm has two weaknesses: it runs each block a fixed number of times that you must choose, and a single run is a single sample. The community gem benchmark-ips (iterations per second, evanphx/benchmark-ips) fixes both: it first warms up each report, then runs it for a time budget, and reports iterations per second with a standard deviation, and compare! prints the ratio between reports. It is the usual tool for comparing two implementations of the same operation. benchmark-memory (michaelherold/benchmark-memory) has the same shape but reports allocated and retained memory per report, built on memory_profiler. Neither gem is part of Ruby and neither was installed or run for this page; the snippet below is the documented usage, shown for orientation and not verified here:

# Not run: benchmark-ips and benchmark-memory are community gems.
require "benchmark/ips"
require "benchmark/memory"

books = Array.new(5_000) { |i| format("978%010d", i) }
by_isbn = books.to_h { |isbn| [isbn, true] }

Benchmark.ips do |x|
  x.config(time: 3, warmup: 1) # seconds measured, seconds of warm-up
  x.report("Array#include?") { books.include?(books.last) }
  x.report("Hash#key?") { by_isbn.key?(books.last) }
  x.compare! # prints "Hash#key?: ... i/s - N.NNx faster"
end

Benchmark.memory do |x|
  x.report("map") { books.map(&:upcase) }
  x.compare!
end

Source: benchmark-ips and benchmark-memory

Under Bundler both gems belong in the development group of the Gemfile, together with benchmark itself in 4.0 (see RubyGems and Bundler).

Algorithmic hot spots

The first thing a profile usually shows is not a slow line but a wrong data structure: a quadratic loop, a linear search inside a loop, a collection rebuilt on every call, a query repeated inside a block. The fixes are the classical ones, expressed in Ruby’s collections:

Search by key, not by scan. Array#include?, Array#find and Array#index are linear. Membership tests belong in a Set (no require needed in current Ruby; see Sets); lookups by key belong in a Hash (see Hashes). Build the index once with to_set, to_h, group_by or each_with_object, then query it many times.
Do not repeat work in a loop. Hoist invariants (a computed regexp, a parsed date, a constant lookup) out of the block; compile patterns once into a constant; call Array#sum, minmax or tally instead of reducing by hand.
Avoid intermediate arrays in long chains. Each map/select allocates an array; on large inputs use each_with_object, filter_map, sum with a block, or a lazy enumerator (see Enumerators and Lazy Evaluation); on huge inputs stream the file line by line instead of reading it whole.
Mind the quadratic idioms. array.uniq inside a loop, string = other` in a loop (each ` copies; use << or join), array.shift/unshift in a loop over a large array, Array#- and Array#& repeated against a large array inside a loop.
Sort once. Sorting costs O(n log n); use sort_by (computes each key once) over sort { ... }, min_by/max_by over sorting to take one element, and bsearch on already sorted data.

Memoization

Memoization stores the result of a pure computation so that later calls with the same arguments skip it. In Ruby it is a few lines around a Hash. Three details separate a correct memo from a leaking or recomputing one:

@value ||= compute recomputes whenever the cached value is nil or false, because ||= tests truthiness. Use defined?(@value) for a single value, or cache in a hash and test with fetch (with a block) or key?, which distinguish "absent" from "present and `nil`".
A hash cache keyed by arguments grows without bound. For long-running processes bound it (an LRU, a size cap, or ObjectSpace::WeakMap or WeakKeyMap for caches that must not keep their keys alive).
A memoized method shared between threads can run twice and, for class-level state, can publish a half-built value. See the ||= race discussion in Threads and the GVL; use a Mutex, build the value eagerly, or accept the duplicate computation if it is pure.

The example times a membership test against an array and a set, and shows the ||= pitfall by counting how often the computation runs. It also shows Hash.new with a block, the compact form for recursive pure functions.

# frozen_string_literal: true

require "benchmark"

# A catalogue lookup two ways: scan an Array, test a Set.
isbns = Array.new(20_000) { |i| format("978%010d", i) }
isbn_set = isbns.to_set
queries = isbns.last(50)

array_time = Benchmark.realtime { queries.each { |q| isbns.include?(q) } }
set_time = Benchmark.realtime { queries.each { |q| isbn_set.include?(q) } }
(array_time / set_time).round.positive? # => true

# Two memoization styles. Both cache a discount per ISBN; only the first
# also caches nil, which `||=` treats as "not computed yet".
class Pricing
  attr_reader :calls

  def initialize
    @cache = {}
    @calls = 0
  end

  def discount(isbn)
    @cache.fetch(isbn) do
      @calls += 1
      @cache[isbn] = compute(isbn)
    end
  end

  def naive_discount(isbn)
    @cache[isbn] ||= begin
      @calls += 1
      compute(isbn)
    end
  end

  private

  def compute(isbn) = isbn.end_with?("7") ? nil : 10
end

pricing = Pricing.new
3.times { pricing.discount("9780306406157") }
pricing.calls # => 1

naive = Pricing.new
3.times { naive.naive_discount("9780306406157") }
naive.calls # => 3

# Hash.new with a block is the compact form for a pure function.
fib = Hash.new { |memo, n| memo[n] = n < 2 ? n : memo[n - 1] + memo[n - 2] }
fib[80] # => 23416728348467685

The assertions print nothing; the values after # => were verified by running the file. In a separate run the array scan took about a thousand times longer than the set lookup; the ratio is the point rather than the figure.

Source: Hash#fetch, Hash.new and Set

Counting allocations

Time is only one cost. Every object Ruby allocates must later be marked or swept by the garbage collector, and a program that allocates a lot spends a lot of time collecting even when each allocation is cheap. Counting allocations is also far more stable than measuring time: the count of objects a piece of code allocates is a property of the code, not of the machine, so it can be asserted in a test.

GC.stat(:total_allocated_objects) is the number of objects allocated since the process started; the difference before and after a block is the number the block allocated. GC.stat(:total_freed_objects), GC.stat(:count) (collections run), GC.stat(:malloc_increase_bytes) and the heap_* keys describe the rest of the picture; the full list of keys is in Garbage Collection and Memory.
GC.stat takes a hash to fill, or a key to return one value; passing a key avoids allocating a new hash on each call (a detail that matters when measuring tiny blocks).
ObjectSpace.count_objects returns the number of live slots per type (:T_STRING, :T_ARRAY, :T_HASH, :FREE, …).
require "objspace" adds ObjectSpace.memsize_of(obj) (the shallow size of one object in bytes), memsize_of_all, count_objects_size, reachable_objects_from, and the allocation tracer ObjectSpace.trace_object_allocations, which records the file, line, class and generation of every object allocated inside the block so that ObjectSpace.allocation_sourcefile(obj) and allocation_sourceline(obj) answer "where was this object created". ObjectSpace.dump(obj) and dump_all write heap snapshots as JSON for offline analysis. Tracing slows the program down, so it belongs in diagnosis and not in production.
The warning category :performance is a switch for the VM’s own hints about slow constructs, enabled with -W:performance or Warning[:performance] = true.

The helper below measures allocations per call. The warm-up run matters: the first call of a method may fill caches, and the first call through GC.stat itself can allocate. The total is rounded to a per-call figure because the VM can add a few unrelated objects during a long loop.

# frozen_string_literal: true

require "objspace"

# Objects allocated per iteration, from GC.stat. The block runs once first to
# warm up, and the total is rounded because other threads and the VM itself
# can add a few stray objects.
def allocations_per_call(times: 1_000, &block)
  block.call
  GC.start
  before = GC.stat(:total_allocated_objects)
  times.times(&block)
  ((GC.stat(:total_allocated_objects) - before) / times.to_f).round
end

Point = Struct.new(:x, :y)
Pair = Data.define(:x, :y)

allocations_per_call { Point.new(1, 2) } # => 1
allocations_per_call { Pair.new(x: 1, y: 2) } # => 3
allocations_per_call { { x: 1, y: 2 } } # => 1

# Strings: interpolation builds one string, + builds an intermediate.
first = "Dune"
second = "Frank Herbert"
allocations_per_call { "#{first} by #{second}" } # => 1
allocations_per_call { first + " by " + second } # rubocop:disable Style/StringConcatenation
# => 2

# A frozen literal is allocated once, so repeating it costs nothing.
allocations_per_call { "constant" } # => 0

# ObjectSpace.memsize_of reports the bytes of one object.
ObjectSpace.memsize_of("x" * 1_000).between?(1_000, 1_100) # => true

# ObjectSpace.count_objects counts live objects per type.
ObjectSpace.count_objects[:T_STRING].positive? # => true

# trace_object_allocations records where an object was allocated.
line = nil
ObjectSpace.trace_object_allocations do
  tracked = Object.new
  line = ObjectSpace.allocation_sourceline(tracked)
end
puts line # prints 42

The program prints the line number of the traced allocation:

Three things are worth reading in those numbers. A Struct instance costs one object, and a Data instance built with new costs three in this build (the extra objects are an implementation detail, so measure rather than assume); the same figures came out on Ruby 3.3 and 3.4. The 4.0 NEWS says Class#new is faster "in all cases, but especially when passing keyword arguments", so the time difference between the two may be smaller than the allocation difference, which is why the count and a timing both belong in the report. Interpolation builds one string while + builds an intermediate per operator. And a string literal under # frozen_string_literal: true is allocated once and costs nothing in a loop, which is a free win on hot paths (see Strings). Without the magic comment, Ruby 3.4 and 4.0 treat plain literals as chilled strings: they still allocate a new object each time, and writing to one emits a deprecation warning when deprecation warnings are enabled (checked on 4.0.7 with -W:deprecated).

For the comparison of Struct, Data and Hash as record types, and the cost of OpenStruct (a bundled gem since 4.0, slow and discouraged), see Struct, Data and Value Objects.

Source: GC.stat, ObjectSpace (including the objspace methods) and Ruby 4.0.0 NEWS

Profilers

A benchmark compares two versions of code you already suspect; a profiler tells you which code to suspect. Profilers fall in two families.

Sampling profilers interrupt the program many times per second, record the call stack, and report how often each method appears. Their overhead is small, so the profile resembles the real run, and they scale to production. The price is statistical: functions shorter than the sampling interval are seen rarely.
Tracing (instrumenting) profilers hook every method call and return. They are exact about call counts and call graphs but slow the program by a large factor and distort what they measure, because the cost they add is proportional to the number of calls, not to the work done.

Also decide what to measure: wall time includes waiting (I/O, sleeping, lock contention), CPU time counts only computing, and allocations count objects created. A slow wall-clock profile with low CPU usage points at I/O and locks, not at Ruby code.

The profilers below are community gems (none ships with Ruby, none was installed for this page); the descriptions come from their documentation and the snippets are not run here. Check each project’s README for the Ruby versions it supports.

Tool Kind What it gives

Tool	Kind	What it gives
stackprof	Sampling, native; modes `:wall`, `:cpu` and `:object`	Per-method and per-line sample counts, call graphs, and flame-graph output; command-line reader (`stackprof file --text`).
vernier	Sampling, native; wall time, thread-aware, with allocation and retained-memory modes	JSON profiles to open in the Firefox Profiler, with per-thread timelines, GC pauses and GVL waits.
ruby-prof	Tracing, native; wall, process and allocation measure modes	Flat, graph, call-stack and flame-graph reports with exact call counts; high overhead.
memory_profiler	Allocation tracking, built on `ObjectSpace` tracing	Report of allocated and retained objects and bytes by gem, file, location and class, with `top:` limits and an `ignore_files` filter.

stackprof

Sampling, native; modes :wall, :cpu and :object

Per-method and per-line sample counts, call graphs, and flame-graph output; command-line reader (stackprof file --text).

vernier

Sampling, native; wall time, thread-aware, with allocation and retained-memory modes

JSON profiles to open in the Firefox Profiler, with per-thread timelines, GC pauses and GVL waits.

ruby-prof

Tracing, native; wall, process and allocation measure modes

Flat, graph, call-stack and flame-graph reports with exact call counts; high overhead.

memory_profiler

Allocation tracking, built on ObjectSpace tracing

Report of allocated and retained objects and bytes by gem, file, location and class, with top: limits and an ignore_files filter.

The typical sessions look like this (documented usage, not run here):

# Not run: stackprof, vernier and memory_profiler are community gems.
require "stackprof"
profile = StackProf.run(mode: :cpu, interval: 1000, raw: true) { import_catalog }
File.binwrite("tmp/stackprof-cpu.dump", Marshal.dump(profile))
# shell: stackprof tmp/stackprof-cpu.dump --text --limit 10

require "vernier"
Vernier.profile(out: "tmp/time_profile.json") { import_catalog }
# open tmp/time_profile.json in https://profiler.firefox.com/

require "memory_profiler"
report = MemoryProfiler.report { import_catalog }
report.pretty_print(to_file: "tmp/memory.txt", scale_bytes: true)

Source: stackprof, vernier and memory_profiler

How to read a profile, whichever the tool:

Start with the top of the report sorted by self time (time in the method itself) and by total time (including callees). A method that is high in total but low in self is a caller of the real cost.
Prefer a flame graph for the shape of the whole run: wide boxes are the time, tall stacks are depth and mean nothing by themselves.
Native frames (Integer#times, String#to_i, Array#each) are attributed to the Ruby code that called them; the question is usually why the caller calls so often, not why the C function is slow.
Profile a workload long enough to collect a few hundred samples, with the production Ruby version, flags and data volume. A profile taken with the JIT off does not describe a run with it on.
Re-profile after every change: the dominant cost moves.

The Thread API is enough to build a tiny sampler, which also shows how the real tools work and why they are written in C. A second thread reads the main thread’s stack with Thread#backtrace_locations and counts the innermost method defined in the script:

# frozen_string_literal: true

# A toy sampling profiler: a second thread peeks at the main thread's stack
# and counts the innermost method defined in this file (the top-level
# methods below are private methods of Object, so their labels start with
# "Object#"). Real profilers (stackprof, vernier) sample from native code,
# far more often and with less overhead. This one can only look when the main
# thread gives up the GVL, which for pure Ruby code is about every 100 ms.
def current_method(target)
  target.backtrace_locations&.find { |location| location.label.include?("Object#") }&.base_label
end

def sampling_loop(target, counts, interval)
  loop do
    name = current_method(target)
    counts[name] += 1 if name
    sleep interval
  end
end

def sample(target = Thread.current, interval: 0.001)
  counts = Hash.new(0)
  sampler = Thread.new { sampling_loop(target, counts, interval) }
  yield
  sampler.kill
  counts
end

def check_digit_slow(digits)
  sum = 0
  digits.each_char.with_index do |char, index|
    sum += char.to_i * (index.even? ? 1 : 3)
  end
  (10 - (sum % 10)) % 10
end

def checksum_all(isbns)
  isbns.each { |isbn| check_digit_slow(isbn) }
end

def parse_all(isbns)
  isbns.each { |isbn| isbn.delete("-") }
end

isbns = Array.new(1_000_000) { |i| format("978%09d", i) }
counts = sample do
  checksum_all(isbns) # the hot spot
  parse_all(isbns)
end

puts counts.max_by { |_line, hits| hits }.first # prints 29

The program prints the name of the method where most samples landed:

check_digit_slow

It printed the same name in repeated runs on this machine, but the sample counts themselves (a dozen or two) vary from run to run. The toy can only look when the main thread releases the GVL (at the end of its time slice, about every 100 ms for pure Ruby code, see Threads and the GVL), which is why it needs a workload of seconds. Real sampling profilers take their samples from a native timer and are accurate at much shorter intervals. Related built-in tracing facilities (TracePoint, Coverage, caller) are in Reflection, Introspection and Tracing.

Source: Thread#backtrace_locations and Thread::Backtrace::Location

Pitfalls of micro-benchmarks

A micro-benchmark measures a tiny piece of code in isolation and is easy to get wrong. The usual traps:

Trap What goes wrong and how to avoid it

Trap	What goes wrong and how to avoid it
No warm-up	The first iterations pay for method caches, lazy `require`, autoloading and, with YJIT, compilation. Use `bmbm`, or benchmark-ips' warm-up phase, and discard the first runs.
Garbage collection noise	A collection that happens to fall inside one report slows it for reasons unrelated to its code. Call `GC.start` before each measurement, run longer, or compare allocation counts.
Constant folding and dead work	A block whose result is unused may be cheaper than real code; use the result (assign it, add it to a checksum) so the work is not optional.
Unrepresentative data	A hash of 10 keys behaves differently from one of 10 million (cache misses, resizing, GC). Benchmark at the size that occurs in production.
One sample	A single run is a point, not a measurement. Repeat, look at the spread, and distrust differences below it.
Different conditions	Compare the two versions in the same process, in an interleaved order, with the same Ruby, JIT setting and machine state (battery saving, other load).
The wrong clock	`Time.now` can jump; use a monotonic clock for durations.
Optimising what is not hot	Making a 1% function 10x faster gains 0.9%. Profile first.
Ignoring memory	A faster version that allocates ten times more can lose in a real program, because the garbage collector pays the difference later.
YJIT warm-up	With YJIT the first iterations are interpreted and compiled code is produced later; benchmark long enough and report steady state, and say whether the JIT was on. See YJIT, ZJIT and the VM.

No warm-up

The first iterations pay for method caches, lazy require, autoloading and, with YJIT, compilation. Use bmbm, or benchmark-ips' warm-up phase, and discard the first runs.

Garbage collection noise

A collection that happens to fall inside one report slows it for reasons unrelated to its code. Call GC.start before each measurement, run longer, or compare allocation counts.

Constant folding and dead work

A block whose result is unused may be cheaper than real code; use the result (assign it, add it to a checksum) so the work is not optional.

Unrepresentative data

A hash of 10 keys behaves differently from one of 10 million (cache misses, resizing, GC). Benchmark at the size that occurs in production.

One sample

A single run is a point, not a measurement. Repeat, look at the spread, and distrust differences below it.

Different conditions

Compare the two versions in the same process, in an interleaved order, with the same Ruby, JIT setting and machine state (battery saving, other load).

The wrong clock

Time.now can jump; use a monotonic clock for durations.

Optimising what is not hot

Making a 1% function 10x faster gains 0.9%. Profile first.

Ignoring memory

A faster version that allocates ten times more can lose in a real program, because the garbage collector pays the difference later.

YJIT warm-up

With YJIT the first iterations are interpreted and compiled code is produced later; benchmark long enough and report steady state, and say whether the JIT was on. See YJIT, ZJIT and the VM.

Threads change the arithmetic too. Running CPU-bound Ruby code on several threads does not speed it up under the GVL, and a benchmark that spawns threads can show a loss; I/O-bound work behaves the opposite way. Measure with the concurrency model you will deploy (threads, processes, Ractors, or a fiber scheduler).

Performance tests

Performance expectations belong in the test suite only if the tests stay reliable. Asserting on wall-clock time ("this must take under 50 ms") makes the suite flaky: a busy continuous-integration machine, a cold cache or a garbage collection at the wrong moment turns a green build red, and a faster machine hides real regressions. Prefer assertions on quantities that do not depend on the machine:

Allocation budgets: the number of objects a hot method allocates (GC.stat(:total_allocated_objects) before and after).
Algorithmic bounds: the same operation on inputs of two sizes allocates (or performs) the same amount, so it is not proportional to the size.
Relative comparisons: one implementation allocates less than another.

Time-based checks, if they are needed at all, belong in a separate, opt-in task (a Rake task or a tagged group that runs on a dedicated machine, and not in the default test run), with a tolerance wide enough to survive noise, and trends are better tracked over many runs than failed on one. Tagging tests as performance tests so that the default run skips them is a reasonable convention; what is not reasonable is to make the default build depend on a stopwatch.

The example is a Minitest file with two stable checks: concatenation allocates more than Array#join, and a set lookup allocates nothing no matter how large the set is. See Testing with Minitest for the rest of the framework.

# frozen_string_literal: true

require "minitest/autorun"
require "objspace"

# A performance test asserts on something stable (allocations, complexity),
# never on wall-clock time, which varies between machines and runs.
class PerformanceTest < Minitest::Test
  def allocated
    GC.start
    before = GC.stat(:total_allocated_objects)
    yield
    GC.stat(:total_allocated_objects) - before
  end

  def test_join_allocates_less_than_concatenation
    parts = %w[Dune Frank Herbert]
    parts.join(" ") # warm up
    joined = allocated { 100.times { parts.join(" ") } }
    summed = allocated { 100.times { parts[0] + " " + parts[1] + " " + parts[2] } } # rubocop:disable Style/StringConcatenation

    assert_operator joined, :<, summed
  end

  def test_set_lookup_allocates_nothing_per_call
    large = Array.new(100_000) { |i| "isbn-#{i}" }.to_set
    large.include?("isbn-99") # warm up

    assert_operator allocated { 1_000.times { large.include?("isbn-99") } }, :<, 10
  end
end

Output of ruby -w perf_test.rb (the elapsed time varies):

2 runs, 4 assertions, 0 failures, 0 errors, 0 skips

Source: GC.stat and Minitest

Where the books are outdated

The two books that informed this page cover performance briefly: one has a chapter on benchmarking, profiling and tagged performance tests, the other uses allocation counting and Data versus Struct measurements to support design decisions. Where their advice no longer matches Ruby 4.0:

Older statement Current practice

Older statement	Current practice
`Benchmark` is "part of the standard library", loaded with a bare `require`.	It is a bundled gem since Ruby 4.0 (a default gem before). A plain `ruby script.rb` still finds it, but a Bundler project must add `gem "benchmark"`. The documentation moved to the gem’s own repository.
Benchmarks are run on one version and a plain interpreter, with the JIT either ignored or mentioned only as a build option.	YJIT can be switched on at runtime with `RubyVM::YJIT.enable`, and ZJIT (new in 4.0) is experimental; a benchmark must state which one was active. See YJIT, ZJIT and the VM.
Wall-clock assertions in the default test run.	Flaky by nature; assert allocation counts and algorithmic bounds, and move timing to an opt-in task (see above).
Performance conclusions drawn from `Struct` versus `Hash` or `Data` timings of an older Ruby.	`Class#new` and keyword-argument calls got faster in 4.0, so measure on the version you deploy, and count allocations as well as time.
Rely on `OpenStruct` or `ostruct` for convenient records.	`OpenStruct` is a bundled gem since 4.0 and slow; use `Struct` or `Data`.
`Time.now` as a stopwatch.	Use `Process.clock_gettime(Process::CLOCK_MONOTONIC)` or `Benchmark.realtime`.

Benchmark is "part of the standard library", loaded with a bare require.

It is a bundled gem since Ruby 4.0 (a default gem before). A plain ruby script.rb still finds it, but a Bundler project must add gem "benchmark". The documentation moved to the gem’s own repository.

Benchmarks are run on one version and a plain interpreter, with the JIT either ignored or mentioned only as a build option.

YJIT can be switched on at runtime with RubyVM::YJIT.enable, and ZJIT (new in 4.0) is experimental; a benchmark must state which one was active. See YJIT, ZJIT and the VM.

Wall-clock assertions in the default test run.

Flaky by nature; assert allocation counts and algorithmic bounds, and move timing to an opt-in task (see above).

Performance conclusions drawn from Struct versus Hash or Data timings of an older Ruby.

Class#new and keyword-argument calls got faster in 4.0, so measure on the version you deploy, and count allocations as well as time.

Rely on OpenStruct or ostruct for convenient records.

OpenStruct is a bundled gem since 4.0 and slow; use Struct or Data.

Time.now as a stopwatch.

Use Process.clock_gettime(Process::CLOCK_MONOTONIC) or Benchmark.realtime.

Migration notes for older code are collected in What’s New and Migrating Older Code.

Version notes

Version	Change relevant to this page
3.2	`Data` is added to core (the record type compared above). Checked: `Data` is undefined on Ruby 3.1.7 and defined on 3.2.11.
3.3	`RubyVM::YJIT.enable` switches YJIT on at runtime (present on 3.3.12, absent on 3.2.11).
3.4	Prism is the default parser; `it` is the implicit block parameter; strings from plain literals are chilled. `Benchmark` is still a default gem.
4.0	`Benchmark` becomes a bundled gem (benchmark 0.5.0). `Class#new` is faster, especially with keyword arguments, and the improvement is also in YJIT and ZJIT. GC heaps of different size pools grow independently. ZJIT is introduced as an experimental method-based JIT: where available it is enabled with `--zjit` or `RubyVM::ZJIT.enable`, and the NEWS advises against deploying it in production. The `ruby` binary used for this page was built without ZJIT, so nothing about ZJIT performance was measured.

References

Ruby 3.4 documentation: Benchmark (last core-library page) and ruby/benchmark (bundled gem since 4.0)
Process.clock_gettime, GC and ObjectSpace
Thread::Backtrace::Location, TracePoint and Warning (:performance category)
YJIT and ZJIT guides, and Ruby 4.0.0 NEWS
benchmark-ips and benchmark-memory (community gems; not run for this page)
stackprof, vernier, ruby-prof and memory_profiler (community profilers; not run for this page)
Firefox Profiler (viewer for vernier profiles)