Standard Library Tour
|
This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production. The reference material consulted while preparing these pages is listed in the section bibliography. |
This page is written against Ruby 4.0 (4.0.7, checked on 2026-10-18). Ruby ships with a large collection of libraries that you load with require instead of getting automatically the way you get String or Array. Most of them are small, old and stable, and a handful (Logger, ERB, Forwardable, SecureRandom, Digest, Timeout, Zlib) are used in almost every real program. This page does two jobs. First it explains how the library is organized today: a large part of it is packaged as gems (since Ruby 2.6, with more libraries moving each release since 3.1), split into default gems that are always loadable and bundled gems that a project has to list in its Gemfile, and Ruby 4.0 moved two of the best known libraries, logger and benchmark, from the first group to the second. Then it takes seventeen libraries one by one, each with a short, runnable example based on the running shelf gem (a small library catalogue, see the shelf gem source), the methods and options worth knowing, and the pitfalls.
The topics that have a page of their own are not repeated here: JSON, YAML, CSV and Marshal are in Serialization: JSON, YAML, CSV and Marshal, Net::HTTP, URI and sockets in Networking and HTTP, files, Tempfile and Pathname in IO, Files and Directories, Open3 and signals in Processes, Signals and Shell, OptionParser in Command-Line Programs, and Date in Time and Dates. How a gem is found, activated and locked is the subject of RubyGems and Bundler, and require itself is explained in Code Loading: require and autoload. The security consequences of some of the libraries below (Timeout, Digest, SecureRandom, Zlib, ERB) are collected in Security.
Every example was syntax-checked with ruby -wc, run with ruby -w on Ruby 4.0.7 (the one exception, Benchmark.bm, is explained in its section) and linted with RuboCop 1.92 using the configuration of the shelf gem (double-quoted strings, NewCops: enable, Style/Documentation off). The output shown is the real output; p prints the inspect form. Examples that use the shelf gem are run with ruby -Ilib, from the gem directory. Randomness, clocks and the machine make some values different on every run, so those examples print the shape of the value (its size, its class) and say so.
How the standard library is organized
The library has four layers, and which layer a name belongs to decides what you have to write to use it.
| Layer | What it is | What you write |
|---|---|---|
Core |
Built into the interpreter: |
Nothing. |
Libraries and extensions |
Parts of the interpreter distribution that are not gems: |
|
Default gems |
Libraries packaged as gems that ship inside Ruby and cannot be uninstalled: |
|
Bundled gems |
Libraries that are installed together with Ruby but can be uninstalled and are declared as ordinary dependencies: |
|
The authoritative list is the Ruby Standard Library page of the 4.0 documentation. Two details are easy to miss. The bundled gems are not documented there (nor in the docs.ruby-lang.org class pages): their documentation lives with each gem, for example ruby.github.io/logger, which is why several links on this page point at GitHub or at a gem’s own site. And a gem can be listed on that page with a link to its GitHub repository because the code is maintained outside the Ruby repository, even when it is a default gem.
The first example asks RubyGems which of the libraries toured below are default gems and which are bundled gems on this machine. Gem::Specification#default_gem? is the flag RubyGems keeps; a bundled gem is simply a gem that is installed but not flagged.
# frozen_string_literal: true
# require name => gem name, for the libraries toured on this page.
LIBRARIES = {
"logger" => "logger", "erb" => "erb", "forwardable" => "forwardable", "delegate" => "delegate",
"singleton" => "singleton", "observer" => "observer", "securerandom" => "securerandom",
"digest" => "digest", "openssl" => "openssl", "benchmark" => "benchmark", "pp" => "pp",
"shellwords" => "shellwords", "timeout" => "timeout", "weakref" => "weakref", "etc" => "etc",
"zlib" => "zlib", "strscan" => "strscan"
}.freeze
LIBRARIES.each do |feature, name|
spec = Gem::Specification.find_by_name(name)
kind = spec.default_gem? ? "default" : "bundled"
puts "#{feature.ljust(12)} #{kind.ljust(8)} #{spec.version}"
end
Output of ruby -w e00_kinds.rb on Ruby 4.0.7:
logger bundled 1.7.0
erb default 6.0.7
forwardable default 1.4.0
delegate default 0.6.1
singleton default 0.3.0
observer bundled 0.1.2
securerandom default 0.4.1
digest default 3.2.1
openssl default 4.0.2
benchmark bundled 0.5.0
pp default 0.6.3
shellwords default 0.2.2
timeout default 0.6.0
weakref default 0.1.4
etc default 1.4.6
zlib default 3.2.3
strscan default 3.1.6
Fourteen of the seventeen are default gems; logger and benchmark joined the bundled gems in 4.0, and observer in 3.4. A bundled gem is just a regular gem with a version of its own (logger 1.7.0); you can see its files with gem contents logger.
Source: Gem::Specification
Using a bundled gem in a project
An application lists the bundled gems it uses in its Gemfile; a library lists them as dependencies in its gemspec. Default gems need no entry, although you can add one to pin a different version than the one Ruby ships.
# frozen_string_literal: true
source "https://rubygems.org"
gemspec
# Bundled gems that Ruby 4.0 no longer lets a bundle load for free.
gem "benchmark"
gem "logger"
gem "observer"
# frozen_string_literal: true
Gem::Specification.new do |spec|
spec.name = "shelf"
spec.version = "0.1.0"
spec.summary = "A small library catalogue"
spec.authors = ["A. Writer"]
spec.required_ruby_version = ">= 4.0"
spec.files = Dir["lib/**/*.rb"]
spec.metadata["rubygems_mfa_required"] = "true"
# A library declares what it requires at run time; default gems need no entry.
spec.add_dependency "logger", "~> 1.7"
spec.add_dependency "observer", "~> 0.1"
end
Source: Bundler: Gemfile and RubyGems: add_dependency
Run bundle install after editing the Gemfile (the Bundler workflow is in RubyGems and Bundler). The failure mode to recognize is a script that ran on 3.3 and stops on 3.4 or 4.0: under Bundler, require "observer" (3.4 and later) or require "logger" (4.0 and later) raises LoadError: cannot load such file, because the gem is installed but hidden by the bundle. In the Ruby release before a move, RubyGems and Bundler print a warning when a bundle requires a gem that was about to become bundled (introduced in 3.3, Feature #19351). Ruby 4.0.7 keeps the table of announced moves in lib/ruby/4.0.0/bundled_gems.rb; it still carries one gem that has not moved yet, tsort, announced for Ruby 4.1.
The moves, release by release
| Ruby | Change in the standard library |
|---|---|
3.1 |
|
3.2 |
|
3.3 |
|
3.4 |
|
4.0 |
|
4.1 (not released on 2026-10-18; 4.1.0-preview1 exists) |
|
Source: NEWS for 3.1, NEWS for 3.3, NEWS for 3.4 and NEWS for 4.0
A map of the rest of the library
Libraries that matter but are not toured below, and where they are covered.
| Need | Library | Page |
|---|---|---|
Read and write JSON, YAML, CSV; binary object dumps |
|
|
HTTP, sockets, DNS, URIs |
|
|
Files, directories, temporary files, in-memory IO |
|
|
Running other programs |
|
|
Command-line options |
|
|
Dates and times |
|
|
Threads and synchronization |
|
|
Sets and graphs |
|
|
Tests |
|
|
Profiling and GC statistics |
|
|
Debugging and consoles |
|
Logger
Logger writes timestamped, levelled messages to an IO object or a file, with optional rotation. It is a bundled gem since Ruby 4.0 (logger 1.7.0; a default gem up to 3.4), so a bundle that uses it needs gem "logger". Prefer it to puts for anything that runs unattended: it adds the severity, can be silenced by level, and can be pointed at a file, $stderr or nothing without changing the call sites.
# frozen_string_literal: true
require "logger"
require "shelf"
# One logger per application, written to $stdout (use a file path or an
# IO object; $stderr is the other common choice).
log = Logger.new($stdout, progname: "shelf", level: Logger::INFO)
log.formatter = proc do |severity, _time, progname, message|
"#{severity.ljust(5)} [#{progname}] #{message}\n"
end
catalog = Shelf::Catalog.new
book = Shelf::Book.new(Shelf::Isbn.new(digits: "978-0-306-40615-7"), "Rubies", Shelf::Author.new("A. Writer"))
log.debug("not shown: below the INFO level")
log.info("adding #{book}")
catalog.add(book)
log.warn { "catalogue has #{catalog.count} book(s)" }
begin
Shelf::Isbn.new(digits: "123")
rescue ArgumentError => e
log.error(e)
end
log.add(Logger::FATAL, "giving up", "other")
log.level = :debug
log.debug("now shown")
p log.level
# The default formatter: severity initial, time, process id, severity, progname, message.
stamp = Logger::Formatter.new.call("INFO", Time.utc(2026, 1, 2, 3, 4, 5), "shelf", "hello")
puts stamp.sub(/#\d+/, "#PID")
# Temporary level change (logger 1.6.0, Ruby 3.3); logging to a file with rotation.
log.with_level(:error) do
log.warn("hidden")
log.error("shown")
end
rotating = Logger.new("shelf.log", 3, 1_048_576) # keep 3 files of 1 MiB each
p rotating.class
rotating.close
File.delete("shelf.log")
quiet = Logger.new(nil) # a null logger for tests
p quiet.info("nobody hears this")
Output of ruby -w -Ilib e01_logger.rb:
INFO [shelf] adding Rubies by A. Writer
WARN [shelf] catalogue has 1 book(s)
ERROR [shelf] invalid ISBN: 123
FATAL [other] giving up
DEBUG [shelf] now shown
0
I, [2026-01-02T03:04:05.000000 #PID] INFO -- shelf: hello
ERROR [shelf] shown
Logger
true
Source: Logger
What to know:
-
Levels.
Logger::DEBUG(0),INFO,WARN,ERROR,FATALandUNKNOWN(5). A message is written only when its severity is at least the logger’slevel. Thelevel:keyword andlevel=accept a constant, a symbol (:warn) or a string ("warn");levelreturns the integer. The default isDEBUG. -
Four ways to log.
log.info("text")(the argument is the message),log.info("progname") { "text" }(the block is evaluated only if the level is enabled, which is how to avoid building expensive messages),log.add(severity, message, progname)andlog << "raw text", which writes without any formatting and ignores the level. -
Exceptions as messages. An
Exceptionpassed toerroris formatted by the default formatter asmessage (ClassName)followed by its backtrace, if it has one. A custom formatter, as above, receives the object itself and decides. -
The default format is
I, [2026-01-02T03:04:05.000000 #4242] INFO — shelf: hello: severity initial, time, process id, severity, progname and message.datetime_format:changes only the time. A formatter is any object withcall(severity, time, progname, message);Logger::Formatteris the default. -
Destinations. An IO (
$stdout,$stderr), a path (the file is created, appended to, and starts with a header line unlessskip_header: true), ornilfor a logger that discards everything, handy in tests. With a path,Logger.new(path, 3, 1_048_576)keeps three files of at most 1 MiB, andLogger.new(path, "daily")(or"weekly","monthly") rotates by date. -
Threads. Writes to the log device are serialized by a monitor, so one logger can be shared by threads.
with_level(:error) { ... }(logger 1.6.0, Ruby 3.3) changes the level for the duration of the block, for the current fiber only (so other threads keep theirs). -
Buffering.
Logger.new($stdout)is as buffered as$stdoutis; when output goes to a pipe or a file, call$stdout.sync = trueor lines can appear late and out of order with$stderr. -
Do not log secrets. Passwords, tokens and full request bodies end up in files that many people can read; log identifiers instead.
-
Structured logging (JSON lines) is a custom formatter that returns
JSON.generate(...) + "\n". Application frameworks add tagging and request ids; none of this is in the standard library.
ERB
ERB (Embedded Ruby) turns a template with <% %> tags into a string: the Ruby between the tags is executed and the text around it is copied. It is the template engine behind generated configuration files, reports, e-mails and many static-site tools. It is a default gem (erb 6.0.7 in Ruby 4.0).
# frozen_string_literal: true
require "erb"
require "shelf"
TEMPLATE = <<~ERB
Catalogue (<%= books.size %> books)
<%- books.each_with_index do |book, i| -%>
<%= i + 1 %>. <%= ERB::Util.html_escape(book.title) %> -- <%= book.author.name %>
<%- end -%>
ERB
author = Shelf::Author.new("Ann <Admin> & Co")
books = [
Shelf::Book.new(Shelf::Isbn.new(digits: "978-0-306-40615-7"), "Rubies & Gems", author),
Shelf::Book.new(Shelf::Isbn.new(digits: "9780306406157"), "Plain", author)
]
# trim_mode "-" lets <%- and -%> remove the whole line around a tag.
erb = ERB.new(TEMPLATE, trim_mode: "-")
puts erb.result_with_hash(books: books)
# html_escape is also available as ERB::Util.h; url_encode escapes a URL component.
puts ERB::Util.h("<b>\"quoted\" & 'single'</b>")
puts ERB::Util.url_encode("a title/with spaces?")
Output of ruby -w e02_erb.rb:
Catalogue (2 books)
1. Rubies & Gems -- Ann <Admin> & Co
2. Plain -- Ann <Admin> & Co
<b>"quoted" & 'single'</b>
a%20title%2Fwith%20spaces%3F
What to know:
-
Tags.
<%= expression %>inserts the value of the expression converted withto_s;<% code %>runs code and inserts nothing;<%# comment %>is ignored;<%%produces a literal<%. -
Running a template.
result(binding)evaluates the template in the given binding (so local variables are visible),result_with_hash(hash)evaluates it with exactly those names as locals (the safer default, shown above),runprints the result,srcreturns the compiled Ruby, anddef_method(klass, "name(args)")compiles the template once into a method, which is the fast way to render the same template many times. -
Trim modes are chosen with
trim_mode:."-"removes the whole line around a tag written<%-…-%>(used above),"<>"omits the newline of lines that start with<%and end with%>,">"omits the newline after every%>, and"%"makes a line that starts with%a line of Ruby. They can be combined ("%<>"). Without one, the blank lines left by control tags stay in the output. -
No escaping.
<%= %>does not escape anything. For HTML, useERB::Util.html_escape(aliash;ERB::Escape.html_escapeis the fast alias, 3.2) on every untrusted value,ERB::Util.url_encodefor URL components. Frameworks add automatic escaping on top; plainERBhas none. -
A template is code. Never build an
ERBfrom untrusted input, and never render a template with a binding that exposes sensitive objects:<%= File.read("/etc/passwd") %>is valid ERB. See Security. -
Ruby 4.0. The positional arguments of
ERB.new(ERB.new(str, nil, "-"), with a safe level and the trim mode as the second and third arguments) were removed in erb 6.0, which ships with Ruby 4.0;ERB.new(str, nil, "-")now raisesArgumentError: wrong number of arguments (given 3, expected 1). UseERB.new(str, trim_mode: "-"). Older tutorials, and the books this reference was cross-checked against, still show the positional form. -
Pitfall. Under
ruby -w, a template that contains a tag such as<% 1 %>warnspossibly useless use of a literal in void context, because the compiled code is checked like any other Ruby.
Forwardable
Forwardable defines methods that forward to another object, so that a class can expose part of an interface of an object it holds instead of inheriting from it or writing one-line wrapper methods. It is a default gem (forwardable 1.4.0 in Ruby 4.0), loaded with require "forwardable", and it is the lightweight implementation of the composition-over-inheritance rule.
# frozen_string_literal: true
require "forwardable"
require "shelf"
module Reading
# A reading list that exposes part of Array's interface and nothing else.
class List
extend Forwardable
include Enumerable
def_delegators :@books, :size, :empty?, :<<, :each
def_delegator :@books, :first, :next_up
def_delegator "self.class", :name, :kind
def initialize = @books = []
end
# Delegation at module level: Reading.loan_days reads Shelf.config.loan_days.
module Settings
extend SingleForwardable
def_delegators :"Shelf.config", :loan_days, :loan_days=
end
end
list = Reading::List.new
list << "Rubies" << "Gems"
p list.size
p list.next_up
p list.kind
p list.map(&:upcase)
p list.respond_to?(:push)
begin
list.push("x")
rescue NoMethodError => e
puts e.message
end
Reading::Settings.loan_days = 21
p Shelf.config.loan_days
Output of ruby -w -Ilib e03_forwardable.rb:
2
"Rubies"
"Reading::List"
["RUBIES", "GEMS"]
false
undefined method 'push' for an instance of Reading::List
21
Source: Forwardable and SingleForwardable
What to know:
-
The API.
def_delegator(accessor, method, ali = method)defines one forwarding method,def_delegators(accessor, *methods)defines several with their own names, anddelegate [:a, :b] => :@targetis the hash form. The long namesdef_instance_delegator,def_instance_delegatorsandinstance_delegateare the same methods.SingleForwardabledoes the same for one object or a module (extend SingleForwardable), asSettingsdoes above. -
The accessor is an instance variable name (
:@books), a method name (:books) or a string holding a Ruby expression ("self.class","Shelf.config"). Expressions are evaluated at every call. -
Only what you list is forwarded. The example shows that
pushis missing: the point of forwarding over inheritance is a smaller interface.include Enumerableplus a forwardedeachis the cheapest way to make a wrapper iterable. -
Return value and errors. The forwarded method returns what the target returned, and an error raised by the target keeps its message. Keyword arguments and blocks are passed through.
-
Wrong accessor. A typo in the accessor name is only found when the method is called (
NoMethodErrororNameErrorfor the accessor), not when the class is loaded. -
Choosing. Use
Forwardablewhen you want a fixed, explicit list of delegated methods. UseSimpleDelegator(next section) when you want to forward everything and override a few methods.
delegate: SimpleDelegator and DelegateClass
The delegate library wraps an object and forwards every method call it does not define itself, which is the classic Decorator implementation (the pattern is discussed in SOLID and Design Patterns). It is a default gem (delegate 0.6.1 in Ruby 4.0).
# frozen_string_literal: true
require "delegate"
require "shelf"
module Decorators
# Decorator: adds behaviour, forwards everything else to the wrapped Book.
class ShoutingBook < SimpleDelegator
def title = super.upcase
def to_s = "#{title} by #{author.name}"
end
# DelegateClass builds a class whose public interface is that of Shelf::Book.
class LoggedBook < DelegateClass(Shelf::Book)
attr_reader :calls
def initialize(book)
super
@calls = []
end
def title
@calls << :title
super
end
end
end
book = Shelf::Book.new(Shelf::Isbn.new(digits: "9780306406157"), "Rubies", Shelf::Author.new("A. Writer"))
shout = Decorators::ShoutingBook.new(book)
puts shout
p shout.__getobj__.equal?(book)
p shout == book
p shout.is_a?(Shelf::Book)
p shout.respond_to?(:isbn)
logged = Decorators::LoggedBook.new(book)
logged.title
logged.title
p logged.calls
p logged.is_a?(Shelf::Book)
begin
shout.missing
rescue NoMethodError => e
puts e.message
end
shout.__setobj__(Shelf::Book.new(book.isbn, "Gems", book.author))
puts shout
Output of ruby -w -Ilib e04_delegate.rb:
RUBIES by A. Writer
true
true
false
true
[:title, :title]
false
undefined method 'missing' for an instance of Decorators::ShoutingBook
GEMS by A. Writer
Source: SimpleDelegator and Delegator
What to know:
-
SimpleDelegator.new(obj)wraps any object;__getobj__returns it and__setobj__replaces it.superinside an overriding method calls the wrapped object’s method, as intitleabove. -
DelegateClass(SomeClass)returns a class whose instance methods are copied fromSomeClassat definition time, so the wrapper responds to exactly the public interface of that class and is slightly faster thanmethod_missing. Inherit from it and callsuperininitializeto set the wrapped object. -
Identity is not preserved.
Delegatorinherits fromBasicObject, soshout.is_a?(Shelf::Book)isfalseandShelf::Book === shoutdoes not match acasebranch, whileshout == bookistrue(the delegator forwards==). Do not usecase/whenon the class, orkind_of?, to examine a decorated object; use duck typing or pattern matching on its interface. -
Pitfalls. Private methods of the wrapped object are not forwarded (
NoMethodError). The missing-method message names the wrapper, as the output shows (undefined method 'missing' for an instance of Decorators::ShoutingBook). Wrapping changesequal?: the wrapper and the wrapped object are two objects, so identity-based collections (compare_by_identityhashes,Setof identities) treat them as different.
Singleton
Singleton turns a class into one that has exactly one instance, available as Klass.instance. It is a default gem (singleton 0.3.0 in Ruby 4.0).
# frozen_string_literal: true
require "singleton"
require "shelf"
class Registry
include Singleton
attr_reader :catalogs
def initialize = @catalogs = {}
end
a = Registry.instance
b = Registry.instance
p a.equal?(b)
a.catalogs[:main] = Shelf::Catalog.new
p b.catalogs.keys
begin
Registry.new
rescue NoMethodError => e
puts e.message
end
begin
a.dup
rescue TypeError => e
puts e.message
end
begin
a.clone
rescue TypeError => e
puts e.message
end
Output of ruby -w e05_singleton.rb:
true
[:main]
private method 'new' called for class Registry
can't dup instance of singleton Registry
can't clone instance of singleton Registry
Source: Singleton
What to know:
-
include Singletonmakesnewandallocateprivate, makesdupandcloneraiseTypeError, and definesinstance, which creates the object on first use. Creation is guarded by a mutex, so concurrent first calls produce one object. In a Ractor program the instance is per Ractor, see Ractors. -
A subclass of a singleton class has its own instance (
A.instance.equal?(B.instance)isfalse). -
A
Marshal.loadofMarshal.dump(Registry.instance)returns the same instance, not a copy;allocateis private likenew. -
This is not the same as a singleton class (the metaclass of an object, see Singleton Classes and Class-Level Behavior); the name is shared by accident.
-
Prefer simpler tools. A global that tests cannot replace is hard to test. In Ruby you rarely need the pattern: a module with
extend self, a constant that holds an object, or a memoized accessor such asShelf.configin the running gem (@config ||= ...) gives the same single point of access with less ceremony and can be reset in a test. Reach forSingletonwhen you truly need to forbid a second instance.
observer
observer implements the Observer pattern: an Observable object keeps a list of observers and calls each of them when it changes. It is a bundled gem since Ruby 3.4 (observer 0.1.2), so a bundle needs gem "observer". Books on design patterns in Ruby typically present it as part of the standard library, which was true when they were written (it was a default gem) and is not any more.
# frozen_string_literal: true
require "observer"
require "shelf"
# The Observer pattern with the bundled "observer" gem.
module Desk
class Notifier
def initialize(name) = @name = name
# Called by notify_observers with whatever arguments were passed.
def update(book, state) = puts "#{@name}: #{book.title} is now #{state}"
end
class WatchedLoan < Shelf::Loan
include Observable
def lend
@state = :lent
changed
notify_observers(book, state)
end
end
end
book = Shelf::Book.new(Shelf::Isbn.new(digits: "9780306406157"), "Rubies", Shelf::Author.new("A. Writer"))
loan = Desk::WatchedLoan.new(book)
loan.add_observer(Desk::Notifier.new("desk"))
loan.add_observer(proc { |b, s| puts "audit: #{b.title}/#{s}" }, :call)
p loan.count_observers
loan.lend
loan.lend
loan.changed
p loan.changed?
loan.delete_observers
p loan.count_observers
begin
loan.add_observer(Object.new)
rescue NoMethodError => e
puts e.message
end
Output of ruby -w -Ilib e06_observer.rb:
2
desk: Rubies is now lent
audit: Rubies/lent
desk: Rubies is now lent
audit: Rubies/lent
true
0
observer does not respond to `update'
Source: ruby/observer on GitHub
What to know:
-
include Observableaddsadd_observer(observer, func = :update),delete_observer,delete_observers,count_observers,changed(state = true),changed?andnotify_observers(*args). -
notify_observersdoes nothing unlesschangedwas called first, and it clears the changed flag afterwards. Forgettingchangedis the usual bug: observers never run, and no error says so. -
The observer is called with whatever arguments
notify_observersreceived, through the method named byfunc;add_observerraisesNoMethodErrorat once if the object does not respond to it (the message in the output keeps the old backtick quoting because it is written in the gem, not generated by the interpreter). -
The observable keeps a strong reference to its observers, so a long-lived subject that is never told
delete_observerleaks them. -
Most Ruby programs use a block or a callable instead (
loan.on_lend { |book| ... }), which needs no library. Chooseobserverfor the interface it standardizes, not out of habit.
SecureRandom
SecureRandom generates random values from the operating system’s cryptographically secure source (getrandom(2), /dev/urandom or the platform equivalent). It is a default gem (securerandom 0.4.1 in Ruby 4.0) and a front end to Random::Formatter, which since Ruby 3.1 you can also load without it (require "random/formatter", then Random.hex(8)).
# frozen_string_literal: true
require "securerandom"
# Values are random, so the example prints their shape, not the value.
token = SecureRandom.hex(16)
p token.size
p token.match?(/\A\h{32}\z/)
uuid = SecureRandom.uuid
p uuid.size
p uuid[14] # version nibble of a UUID v4
p SecureRandom.uuid_v7[14] # v7: time-ordered, good as a database key
p SecureRandom.alphanumeric(12).size
p SecureRandom.base64(9).size
p SecureRandom.urlsafe_base64(9).size
p SecureRandom.random_bytes(8).encoding
p SecureRandom.random_number(100).between?(0, 99)
p SecureRandom.random_number.class # no argument: Float in 0.0...1.0
p SecureRandom.random_number(1.0..2.0).class
p SecureRandom.bytes(4).bytesize
# Random is NOT for secrets: a fixed seed repeats the sequence.
a = Random.new(42)
b = Random.new(42)
p a.rand(1000) == b.rand(1000)
Output of ruby -w e07_securerandom.rb (the values are random, so only their shape is printed):
32
true
36
"4"
"7"
12
12
12
#<Encoding:BINARY (ASCII-8BIT)>
true
Float
Float
4
true
Source: SecureRandom and Random::Formatter
What to know:
-
Use it for anything an attacker must not guess: session identifiers, password-reset and API tokens, salts, nonces, keys. Use
RandomorKernel#randonly for simulations, shuffling and tests:Randomis a deterministic Mersenne Twister whose whole future follows from the seed, as the last lines of the example show. -
The methods.
hex(n)returns2nhexadecimal digits fornrandom bytes,base64(n),urlsafe_base64(n, padding = false),alphanumeric(n, chars: ...),random_bytes(n)andbytes(n)(binary strings),random_number(aFloat, anIntegerbelown, or a number in a range, and also calledrand),uuid(version 4) anduuid_v7(time-ordered, Ruby 3.3, good for database keys because new values sort after old ones). -
Length. The argument is the number of random bytes, not the number of characters:
hex(16)is 32 characters,base64(9)is 12. The default of 16 bytes (128 bits) is adequate for tokens; shorter values can be guessed. -
Comparing a secret to user input must be constant time, see
OpenSSL.fixed_length_secure_comparein the Digest section. -
No seeding. There is no
srandforSecureRandom; a "reproducible" value from it is a bug. Tests inject the generator or stub the call.
Digest
Digest computes message digests (hashes): SHA-256 and the other SHA-2 sizes, SHA-1, MD5 and RIPEMD-160, implemented in C. It is a default gem (digest 3.2.1 in Ruby 4.0). It is for integrity and identification (checksums, cache keys, content-addressed storage, ETags), not for passwords.
# frozen_string_literal: true
require "digest"
require "openssl"
p Digest::SHA256.hexdigest("abc")
p Digest::MD5.hexdigest("abc") # fine for checksums, never for security
p Digest::SHA256.base64digest("abc")
# Incremental hashing: feed data piece by piece.
sha = Digest::SHA256.new
sha << "a"
sha.update("bc")
p sha.hexdigest == Digest::SHA256.hexdigest("abc")
p sha.digest_length
p sha.block_length
# Digest::SHA256.file reads a file in chunks without loading it.
require "tempfile"
Tempfile.create("shelf") do |f|
f.write("abc")
f.flush
p Digest::SHA256.file(f.path).hexdigest == Digest::SHA256.hexdigest("abc")
end
# A keyed hash (HMAC) comes from OpenSSL, not from Digest.
key = "placeholder-key"
mac = OpenSSL::HMAC.hexdigest("SHA256", key, "payload")
p mac.size
expected = OpenSSL::HMAC.hexdigest("SHA256", key, "payload")
p OpenSSL.fixed_length_secure_compare(mac, expected)
p OpenSSL::OPENSSL_LIBRARY_VERSION.start_with?("OpenSSL")
# Passwords are not hashed with Digest: use a slow key-derivation function.
# The iteration count is tiny only to keep this example fast.
derived = OpenSSL::KDF.pbkdf2_hmac(
"placeholder-password", salt: "placeholder-salt", iterations: 1_000, length: 16, hash: "sha256"
)
p derived.bytesize
Output of ruby -w e08_digest.rb:
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
"900150983cd24fb0d6963f7d28e17f72"
"ungWv48Bz+pBQUDeXa4iI7ADYaOWF3qctBD/YfIAFa0="
true
32
64
true
64
true
true
16
Source: Digest, Digest::Class and OpenSSL::HMAC
What to know:
-
Classes:
Digest::SHA256(use this one),Digest::SHA384,Digest::SHA512,Digest::SHA2.new(bitlen),Digest::SHA1,Digest::MD5,Digest::RMD160. There is noDigest::SHA3; useOpenSSL::Digest.new("SHA3-256"). -
Three ways to compute: the class methods
hexdigest,digest(raw bytes) andbase64digest; the file shortcutDigest::SHA256.file(path), which reads in chunks; and an instance fed with<<orupdatefor streaming, finished withhexdigest(an instance can bereset). Two digests compare with==. -
MD5 and SHA-1 are broken for collision resistance; they remain acceptable for non-adversarial uses such as de-duplication or matching a published checksum.
-
Passwords need a slow, salted key-derivation function, not a digest:
OpenSSL::KDF.pbkdf2_hmacandscrypt(shown above with a deliberately tiny count) or a community gem such asbcryptorargon2. A fast digest of a password can be brute-forced at billions of guesses per second. -
Message authentication (is this payload from someone who knows the key?) is an HMAC, not
Digest::SHA256.hexdigest(key + message)(which is vulnerable to length extension). Compare the received and the expected MAC withOpenSSL.fixed_length_secure_compare(it accepts only strings of equal length and raisesArgumentError: inputs must be of equal lengthotherwise) so that the comparison time does not leak how many leading characters matched.
OpenSSL
OpenSSL is the binding to the system’s OpenSSL (or LibreSSL) library, a default gem (openssl 4.0.2 in Ruby 4.0), and the base of every TLS connection Net::HTTP or a TCPSocket upgraded with OpenSSL::SSL::SSLSocket makes. The version of the library it was built against is OpenSSL::OPENSSL_LIBRARY_VERSION. Beyond the digests and HMACs above it offers symmetric ciphers (OpenSSL::Cipher, with authenticated modes such as AES-256-GCM, where the key, a fresh nonce and the authentication tag all have to be handled correctly), public-key cryptography and certificates (OpenSSL::PKey, OpenSSL::X509), key derivation (OpenSSL::KDF) and TLS contexts (OpenSSL::SSL::SSLContext). Treat it as a toolbox for protocol implementers, not a place to invent a scheme: prefer a high-level library or a service for encryption at rest, never turn off certificate verification (OpenSSL::SSL::VERIFY_NONE) to make an error disappear, and keep keys out of the source tree. The detailed security guidance is in Security; the API is at OpenSSL in the 4.0 documentation.
Benchmark
Benchmark measures how long a piece of code takes: wall-clock (real) time and CPU time (user and system). It is a bundled gem since Ruby 4.0 (benchmark 0.5.0; a default gem up to 3.4), so a bundle needs gem "benchmark".
# frozen_string_literal: true
require "benchmark"
ids = (1..2_000).to_a
set = ids.to_set
probes = Array.new(2_000) { |i| i * 2 }
elapsed = Benchmark.realtime { probes.each { |i| ids.include?(i) } }
p elapsed.class
tms = Benchmark.measure("set lookups") { probes.each { |i| set.include?(i) } }
p tms.class
p tms.label
p tms.real >= 0.0
p tms.format("%n: %.3r").class
p Benchmark::CAPTION
p Benchmark::FORMAT
p Benchmark::VERSION
Output of ruby -w e09_benchmark.rb:
Float
Benchmark::Tms
"set lookups"
true
String
" user system total real\n"
"%10.6u %10.6y %10.6t %10.6r\n"
"0.5.0"
Source: ruby/benchmark on GitHub
The report methods bm and bmbm print a table:
# frozen_string_literal: true
require "benchmark"
ids = (1..2_000).to_a
set = ids.to_set
probes = Array.new(2_000) { |i| i * 2 }
# bm prints a table: user, system, user+system and real time in seconds.
Benchmark.bm(7) do |x|
x.report("array:") { probes.each { |i| ids.include?(i) } }
x.report("set:") { probes.each { |i| set.include?(i) } }
end
# bmbm runs a rehearsal first so that GC and caches do not favour the second job.
Benchmark.bmbm(7) do |x|
x.report("array:") { probes.each { |i| ids.include?(i) } }
x.report("set:") { probes.each { |i| set.include?(i) } }
end
Source: ruby/benchmark on GitHub
Output of ruby e09b_bm.rb (a run without -w; the numbers differ on every run and machine, the shape and the ranking do not):
user system total real
array: 0.006820 0.000009 0.006829 ( 0.006863)
set: 0.000061 0.000003 0.000064 ( 0.000063)
Rehearsal -------------------------------------------
array: 0.006564 0.000000 0.006564 ( 0.006568)
set: 0.000064 0.000000 0.000064 ( 0.000063)
---------------------------------- total: 0.006628sec
user system total real
array: 0.006488 0.000028 0.006516 ( 0.006529)
set: 0.000061 0.000001 0.000062 ( 0.000061)
Source: ruby/benchmark on GitHub
What to know:
-
The methods.
Benchmark.realtime { }returns the elapsed seconds as aFloatmeasured with the monotonic clock;Benchmark.measure(label) { }returns aBenchmark::Tmswithutime,stime,cutime,cstime,real,totalandlabel, andformatwith the%n,%u,%y,%tand%rdirectives;Benchmark.bm(label_width)andBenchmark.benchmark(caption, label_width, format, *labels)print the table;Benchmark.bmbmruns every job twice, a "rehearsal" first, so that warm-up and garbage collection do not favour the second job. -
A warning from the gem. Under
ruby -w, benchmark 0.5.0 printswarning: too many arguments for format stringfrombenchmark.rbeach timebmprints a row. It comes from the gem, not from your code, and is harmless; run the reports without-w. -
Measure the right thing. Run each job long enough that it takes at least several milliseconds, outside the code you want to measure create the data, and compare more than one run. A single
realtimeof a microsecond-scale job measures the noise. -
Prefer a counter to a clock when you can. "How many comparisons did it make?" does not vary with load. The comparisons above are 2,000
Array#include?calls against 2,000Set#include?calls over a 2,000-element collection: theSetwins by roughly two orders of magnitude, but the ratio changes with the sizes. -
For iterations per second with error bars (
benchmark-ips), memory benchmarks and profilers, see Performance, Benchmarking and Profiling.
PP
PP is the pretty printer behind Kernel#pp: it prints a nested object over several lines when it does not fit on one, with indentation that shows its structure. It is a default gem (pp 0.6.3 in Ruby 4.0), and Kernel#pp is available without a require. The helper PrettyPrint is its layout engine.
# frozen_string_literal: true
require "pp"
require "shelf"
isbn = Shelf::Isbn.new(digits: "9780306406157")
book = Shelf::Book.new(isbn, "Rubies", Shelf::Author.new("A. Writer"))
data = { book:, tags: %w[ruby gems catalogue shelf library] }
p data # one long line
pp data # wraps at 80 columns, returns its argument
puts data.pretty_inspect.lines.size
PP.pp(data, $stdout, 40) # explicit width
puts PP.singleline_pp(data, +"")
class Stack
def initialize(*items) = @items = items
def pretty_print(printer)
printer.group(1, "#<Stack", ">") do
printer.breakable
printer.seplist(@items) { |item| printer.pp(item) }
end
end
end
PP.pp(Stack.new(*1..12), $stdout, 24)
Output of ruby -w -Ilib e10_pp.rb:
{book: #<data Shelf::Book isbn=#<data Shelf::Isbn digits="9780306406157">, title="Rubies", author=#<data Shelf::Author name="A. Writer">>, tags: ["ruby", "gems", "catalogue", "shelf", "library"]}
{book:
#<data Shelf::Book
isbn=#<data Shelf::Isbn digits="9780306406157">,
title="Rubies",
author=#<data Shelf::Author name="A. Writer">>,
tags: ["ruby", "gems", "catalogue", "shelf", "library"]}
6
{book:
#<data Shelf::Book
isbn=
#<data Shelf::Isbn
digits="9780306406157">,
title="Rubies",
author=
#<data Shelf::Author
name="A. Writer">>,
tags:
["ruby",
"gems",
"catalogue",
"shelf",
"library"]}
{book: #<data Shelf::Book isbn=#<data Shelf::Isbn digits="9780306406157">, title="Rubies", author=#<data Shelf::Author name="A. Writer">>, tags: ["ruby", "gems", "catalogue", "shelf", "library"]}
#<Stack
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12>
Source: PP, PrettyPrint and Kernel#pp
What to know:
-
Entry points.
pp objwrites to$stdoutand returnsobj(several arguments return an array, none returnsnil);obj.pretty_inspectreturns the string;PP.pp(obj, out, width)writes to any object with<<using the given width;PP.singleline_pp(obj, out)prints on one line. -
Width.
ppwraps at the terminal width when$stdoutis a terminal (since Ruby 3.1, viaIO#winsize) and at 80 columns otherwise, which is why a log file and a terminal show different line breaks for the same object. -
Output format. Since 3.4 hashes print as
{key: value}and{"str" => 1}, andDataandStructinstances as#<data Shelf::Book ...>and#<struct ...>, exactly asinspectdoes, but broken over lines. -
Customizing. Define
pretty_print(printer)on a class and build the layout withprinter.group(indent, open, close) { ... },printer.breakable(a space, or a newline when the group does not fit),printer.seplist(items) { ... },printer.textandprinter.pp(child). To restrict which instance variables are shown, overridepretty_print_instance_variables; for objects that may contain themselves,pretty_print_cycle. (Ruby 4.0 also lets a class limit whatinspectshows with a privateinstance_variables_to_inspectmethod.) -
Use it for debugging output and for tests that print structures, not for serialization.
Shellwords
Shellwords splits a command-line string into words with the rules of the Bourne shell, and quotes words so that a shell reads them back as one word each. It is a default gem (shellwords 0.2.2 in Ruby 4.0) and adds String#shellsplit, String#shellescape and Array#shelljoin.
# frozen_string_literal: true
require "shellwords"
p Shellwords.split(%q(git commit -m "first draft" 'single quoted' back\ slash))
p "ls -l 'My Books'".shellsplit
p Shellwords.escape("My Books; rm -rf /")
p "it's".shellescape
p ["cp", "My Books/a b.txt", "/tmp/x"].shelljoin
p Shellwords.escape("")
begin
Shellwords.split('echo "unterminated')
rescue ArgumentError => e
puts e.message
end
Output of ruby -w e11_shellwords.rb:
["git", "commit", "-m", "first draft", "single quoted", "back slash"]
["ls", "-l", "My Books"]
"My\\ Books\\;\\ rm\\ -rf\\ /"
"it\\'s"
"cp My\\ Books/a\\ b.txt /tmp/x"
"''"
Unmatched quote at 5: ..."
Source: Shellwords
What to know:
-
Parsing.
Shellwords.split(aliasesshellsplitandshellwords) understands single quotes, double quotes and backslashes, and raisesArgumentErrorwith a message such asUnmatched quote at 5: ...for an unbalanced quote. It does not expand variables, globs or$(...). -
Quoting.
Shellwords.escape(aliasshellescape) escapes shell metacharacters with backslashes and turns the empty string into'';Shellwords.join(aliasshelljoin) does it for an array. This is for POSIX shells; it is no use forcmd.exeor PowerShell. -
It is a patch, not a defence. The robust way to run a program is to avoid the shell altogether:
system("cp", source, target)andOpen3.capture2e("cp", source, target)pass an argument array and no shell is involved, so there is nothing to escape. Reach forShellwordswhen you must build a command string for a shell you do not control (ansshremote command, a shell script you generate), see Processes, Signals and Shell and Security.
Timeout
Timeout.timeout(seconds) runs a block and raises Timeout::Error in the calling thread if the block takes longer. It is a default gem (timeout 0.6.0 in Ruby 4.0).
# frozen_string_literal: true
require "timeout"
begin
Timeout.timeout(0.05) { sleep 1 }
rescue Timeout::Error => e
p e.class
p e.message
end
# Your own exception class, and a custom message.
class SlowStore < StandardError; end
begin
Timeout.timeout(0.05, SlowStore, "store did not answer") { sleep 1 }
rescue SlowStore => e
p e.message
end
p Timeout.timeout(1) { :done }
p Timeout.timeout(nil) { :no_limit }
p Timeout.timeout(0) { :zero_means_no_limit }
p Timeout::Error.ancestors.take(3)
p Timeout::ExitException.ancestors.take(2)
begin
Timeout.timeout(-1) { :x }
rescue ArgumentError => e
puts e.message
end
Output of ruby -w e12_timeout.rb:
Timeout::Error
"execution expired"
"store did not answer"
:done
:no_limit
:zero_means_no_limit
[Timeout::Error, RuntimeError, StandardError]
[Timeout::ExitException, Exception]
Timeout sec must be a non-negative number
Source: Timeout
What to know:
-
Signature.
Timeout.timeout(sec, klass = nil, message = nil) { |sec| ... }. With aklassthat exception is raised instead (so callers can tell your timeout from another), with amessageit replaces the defaultexecution expired.niland0mean "no limit"; negative values raiseArgumentErrorsince Ruby 3.4 (timeout 0.4.3), and silently meant "no limit" before that. -
Timeout::Errorinherits fromRuntimeError, so a barerescue => earound the call swallows it. The exception the library uses internally to unwind the block,Timeout::ExitException, derives fromException, so a barerescue => einside the block does not catch it; the caller oftimeoutseesTimeout::Error. -
It is dangerous. The exception is raised in the middle of whatever the block was executing, including
ensurecode, a database transaction, a half-written file, or code holding aMutex. Nothing guarantees that the interrupted code gets to restore its invariants. This is why the Ruby community’s advice has become "do not useTimeoutaround code you do not control". -
Prefer timeouts that the operation itself supports:
Net::HTTP#open_timeout,#read_timeoutand#write_timeout,Socket.tcp(..., connect_timeout:),IO#timeout=(Ruby 3.2, raisesIO::TimeoutError),Queue#pop(timeout:)(Ruby 3.2) and, for loops, an explicit deadline computed fromProcess.clock_gettime(Process::CLOCK_MONOTONIC). In Ruby 4.0,TCPSocket.newconsistently raisesIO::TimeoutErrorwhen its timeout expires. The details and the other caveats are in Threads and the GVL. -
Fibers. Under a fiber scheduler,
Timeout.timeoutuses the scheduler’stimeout_afterhook (Ruby 3.1), so it interrupts a blocked fiber instead of a thread; see Fibers and the Fiber Scheduler.
WeakRef
WeakRef is a reference to an object that does not keep it alive: when no other reference exists, the garbage collector may free the object, and the WeakRef then reports it is gone. It is a default gem (weakref 0.1.4 in Ruby 4.0) and a Delegator, so it forwards methods to its referent.
# frozen_string_literal: true
require "weakref"
book = +"Rubies"
ref = WeakRef.new(book)
p ref.weakref_alive?
p ref.upcase # forwards to the referent
p ref.class # WeakRef delegates, so it looks like a String
p ref.__getobj__.equal?(book)
p WeakRef::RefError.ancestors.take(2)
# ObjectSpace::WeakMap keeps keys weakly; WeakKeyMap compares by eql?
map = ObjectSpace::WeakKeyMap.new
key = +"isbn-1"
map[key] = :cached
p map[+"isbn-1"]
p map.key?(key)
p map.getkey(+"isbn-1").equal?(key)
Output of ruby -w e13_weakref.rb:
true
"RUBIES"
WeakRef
true
[WeakRef::RefError, StandardError]
:cached
true
true
Source: WeakRef and ObjectSpace::WeakKeyMap
What to know:
-
WeakRef.new(obj)wrapsobj;weakref_alive?tells whether it is still there. Once the object has been collected, any forwarded call and__getobj__raiseWeakRef::RefError(aStandardError), with the messageInvalid Reference - probably recycled. -
You cannot demonstrate collection reliably. When an object is freed depends on the garbage collector, and the example above only shows the object alive while a strong reference (
book) exists. Never write code or tests that expect a particular moment of collection; see Garbage Collection and Memory. -
Typical use is a cache that must not prevent its entries from being freed. The simplest correct pattern is to check
weakref_alive?and rebuild the value when it is false, accepting that it can die between the check and the use (rescueWeakRef::RefError). -
Better tools exist.
ObjectSpace::WeakMap(keys and values held weakly, compared by identity) andObjectSpace::WeakKeyMap(Ruby 3.3; keys held weakly and compared witheql?andhash, values held strongly, as in the example, so equal strings find the same entry) are implemented in C and are what caches use today.
Etc
Etc gives access to information about the system: users and groups from /etc/passwd and /etc/group, the machine and operating system, the number of processors, and some configuration values. It is a C extension and a default gem (etc 1.4.6 in Ruby 4.0).
# frozen_string_literal: true
require "etc"
p Etc.nprocessors.positive?
p Etc.uname.keys
p Etc.uname[:sysname].class
p Etc.getlogin.class
p Etc.getpwuid.name.class
p Etc.getpwuid.class
p Etc.getpwnam("root").uid
p Etc.getgrgid(0).name.class
p Etc.sysconfdir.class
p Etc.systmpdir.start_with?("/")
p Etc.sysconf(Etc::SC_PAGESIZE).positive?
Output of ruby -w e14_etc.rb (machine-dependent values are shown as their class or as a comparison):
true
[:sysname, :nodename, :release, :version, :machine]
String
String
String
Etc::Passwd
0
String
String
true
true
Source: Etc
What to know:
-
Users and groups.
Etc.getpwuid(uid = Process.uid),Etc.getpwnam(name),Etc.getgrgidandEtc.getgrnamreturnEtc::PasswdandEtc::Groupstructs (name,uid,gid,dir,shell, …); an unknown name raisesArgumentError(can't find user for ...).Etc.passwdandEtc.groupiterate over every entry.Etc.getlogincan returnnilwhen the process has no controlling terminal (a service or container), soEtc.getpwuid.nameis the reliable way to ask "who am I". -
The machine.
Etc.unamereturns a hash with:sysname,:nodename,:release,:versionand:machine.Etc.nprocessorsreturns the number of processors available to the process and is the usual default for a thread or worker pool size; it does not necessarily reflect a CPU quota imposed by a container, so treat it as an upper bound. -
Directories and limits.
Etc.sysconfdir(where the Ruby installation keeps its configuration) andEtc.systmpdir(the system temporary directory, whichDir.tmpdirbuilds on),Etc.sysconf(Etc::SC_PAGESIZE)and the otherSC_*constants,Etc.confstrand theCS_*constants. Many of these do not exist on Windows, where only a subset ofEtcis available. -
For the home directory,
Dir.homeandFile.expand_path("~")are enough;Etcis for when you need another user’s entry.
Zlib
Zlib compresses and decompresses data with the DEFLATE algorithm, in the raw, zlib and gzip formats, over a binding to the zlib C library. It is a default gem (zlib 3.2.3 in Ruby 4.0).
# frozen_string_literal: true
require "zlib"
require "stringio"
text = "shelf " * 200
packed = Zlib.deflate(text)
p text.bytesize
p packed.bytesize
p Zlib.inflate(packed) == text
p Zlib.gunzip(Zlib.gzip(text)) == text
p Zlib.crc32("abc")
p Zlib.adler32("abc")
# gzip in memory (use Zlib::GzipWriter.open(path) for files).
io = StringIO.new
io.set_encoding(Encoding::BINARY)
gz = Zlib::GzipWriter.new(io)
gz.mtime = Time.at(0)
gz.write(text)
gz.close
p io.string.bytesize < text.bytesize
p io.string.byteslice(0, 2).bytes # gzip magic number
Zlib::GzipReader.wrap(StringIO.new(io.string)) { |r| p r.read == text }
begin
Zlib.inflate("not zlib data")
rescue Zlib::Error => e
p e.class
puts e.message
end
p Zlib::BEST_COMPRESSION
# Never inflate untrusted data in one call: read in chunks and enforce a limit.
def bounded_gunzip(data, limit:)
out = +""
Zlib::GzipReader.wrap(StringIO.new(data)) do |reader|
while (chunk = reader.read(4096))
out << chunk
raise ArgumentError, "more than #{limit} bytes" if out.bytesize > limit
end
end
out
end
p bounded_gunzip(Zlib.gzip(text), limit: 10_000).bytesize
begin
bounded_gunzip(Zlib.gzip(text), limit: 100)
rescue ArgumentError => e
puts e.message
end
Output of ruby -w e15_zlib.rb:
1200
24
true
true
891568578
38600999
true
[31, 139]
true
Zlib::DataError
incorrect header check
9
1200
more than 100 bytes
Source: Zlib and Zlib::GzipWriter
What to know:
-
Three formats, one library.
Zlib.deflateandZlib.inflateuse the zlib format (a header and an Adler-32 checksum),Zlib.gzipandZlib.gunzipthe gzip file format (a header with a name and modification time, and a CRC-32 trailer), andZlib::Deflate.new(level, -Zlib::MAX_WBITS)raw DEFLATE. Usinginflateon gzip data is aZlib::DataError(incorrect header check). -
Streaming.
Zlib::GzipWriter.open(path)andZlib::GzipReader.open(path)behave like files and take a block;GzipReader.wrap(io)/GzipWriter.wrap(io)work on any IO (aStringIO, a socket),each_lineandread(length)stream.Zlib::DeflateandZlib::Inflateare the low-level streaming objects; setlevel:(Zlib::BEST_SPEED1 toZlib::BEST_COMPRESSION9) to trade time for size. -
Errors all inherit from
Zlib::Error:Zlib::DataError(corrupt input),Zlib::BufError,Zlib::StreamError,Zlib::MemError, andZlib::GzipFile::Error(and its subclassesCRCError,LengthErrorandNoFooter) for gzip files. -
Checksums.
Zlib.crc32andZlib.adler32take an optional running value so that a checksum can be computed over chunks. They detect accidents, not tampering. -
A decompression bomb is a few kilobytes that inflate to gigabytes. Never call
Zlib.inflateorGzipReader#readwithout a length on untrusted data. Thebounded_gunziphelper at the end of the example reads in chunks and stops when the output passes a limit. The same advice applies to every archive format. -
ZIP and tar archives are not in the standard library (
Gem::Package::TarReaderaside); the communityrubyzipgem is the usual choice.
StringScanner
StringScanner (from the strscan extension, a default gem, strscan 3.1.6 in Ruby 4.0) scans a string from left to right with regular expressions, keeping a scan pointer. It is the right tool for hand-written tokenizers and parsers: it never copies the string, and the regexps are anchored at the pointer, so they cannot accidentally match somewhere later.
# frozen_string_literal: true
require "strscan"
# A tiny tokenizer for catalogue queries such as: title:ruby year:>2010 "gem book"
module Query
Token = Data.define(:type, :value)
# The order matters: the first rule that matches at the scan pointer wins.
RULES = { field: /(\w+):/, phrase: /"([^"]*)"/, number: /[<>]?\d+/, word: /\w+/ }.freeze
def self.tokenize(source)
scanner = StringScanner.new(source)
tokens = []
tokens << next_token(scanner) until skip_blanks(scanner).eos?
tokens
end
def self.skip_blanks(scanner) = scanner.tap { |s| s.skip(/\s+/) }
def self.next_token(scanner)
RULES.each do |type, pattern|
return Token.new(type, scanner[1] || scanner.matched) if scanner.scan(pattern)
end
raise ArgumentError, "unexpected #{scanner.getch.inspect} at #{scanner.pos - 1}"
end
end
Query.tokenize(%(title:ruby year:>2010 "gem book")).each { |t| p t }
begin
Query.tokenize("title:ruby !")
rescue ArgumentError => e
puts e.message
end
s = StringScanner.new("2010-05-17")
p s.scan_integer # strscan 3.1.1, Ruby 3.4
p s.skip("-") # a String pattern is matched literally; returns its length
p s.check(/\d+/) # look ahead without moving
p s.peek(2)
p s.pos
p s.rest
p s.scan_until(/-/)
p s.pre_match
p s.eos?
Output of ruby -w e16_strscan.rb:
#<data Query::Token type=:field, value="title">
#<data Query::Token type=:word, value="ruby">
#<data Query::Token type=:field, value="year">
#<data Query::Token type=:number, value=">2010">
#<data Query::Token type=:phrase, value="gem book">
unexpected "!" at 11
2010
1
"05"
"05"
5
"05-17"
"05-"
"2010-05"
false
Source: StringScanner
What to know:
-
Advance and look ahead.
scan(re)matches at the pointer and advances;scan_until(re)searches ahead and advances past the match;skipandskip_untilreturn the length instead of the text;check,check_untilandmatch?do the same without moving;getchtakes one character,peek(n)shows the nextnbytes,scan_integer(Ruby 3.4) reads an integer (base: 16for hexadecimal). -
Inspect the state.
matched,pre_match,post_match,[](capture groups by number or name,scanner[:name]),captures,rest,eos?,pos(a byte position;charposcounts characters),pos=andunscan(undo the last match). -
Anchors. A pattern is matched at the pointer;
\Ameans "at the pointer" and^"at the start of a line", unless the scanner was created withStringScanner.new(string, fixed_anchor: true), in which case\Ameans the start of the string. -
Order the rules from the most to the least specific, as in the example’s
RULEStable, and always provide a final failure branch that raises with the position (scanner.pos), otherwise an unexpected character loops forever or is skipped silently. -
Compared to
String#scan, which returns every match of a pattern, a scanner can switch patterns depending on what it has just read. For the regular-expression syntax itself see Regular Expressions.
Quick reference
| Library | Require | Kind in 4.0 | Reach for it to |
|---|---|---|---|
Logger |
|
bundled (since 4.0) |
levelled, timestamped log lines to a file or stream |
ERB |
|
default |
render text from a template |
Forwardable |
|
default |
expose a chosen part of a held object’s interface |
delegate |
|
default |
wrap an object and forward everything (decorator) |
Singleton |
|
default |
forbid a second instance of a class |
observer |
|
bundled (since 3.4) |
notify subscribers when an object changes |
SecureRandom |
|
default |
tokens, ids, salts, any value an attacker must not guess |
Digest |
|
default |
checksums and content hashes |
OpenSSL |
|
default |
HMAC, ciphers, certificates, TLS contexts, key derivation |
Benchmark |
|
bundled (since 4.0) |
quick timing comparisons |
PP |
|
default |
readable multi-line dumps of nested data |
Shellwords |
|
default |
split and quote words for a POSIX shell |
Timeout |
|
default |
last-resort deadline for code you control |
WeakRef |
|
default |
a reference that does not keep its object alive |
Etc |
|
default |
users, groups, processor count, system information |
Zlib |
|
default |
compress and decompress, gzip, checksums |
StringScanner |
|
default |
tokenizers and hand-written parsers |
Where older material is outdated
Material written for Ruby 3.3 and earlier, including the books this reference was cross-checked against, says the following about the libraries on this page. Each row is a statement that was true and no longer is.
| What older material says | What is true in Ruby 4.0 |
|---|---|
"`Benchmark` is part of the standard library" and |
|
"`Logger` is part of the standard library" |
|
The Observer pattern is built on the standard |
|
|
removed in erb 6.0 (Ruby 4.0); use |
|
|
|
both are bundled gems since 4.0. |
|
|
|
the width comes from the terminal since 3.1. |
|
the pipe form was removed in 4.0; use |
|
|
A single |
never; use a key-derivation function. |
Pitfalls
-
The require that worked and then did not. A
LoadErrorforlogger,benchmark,observer,csv,base64,bigdecimalorostructafter an upgrade, only underbundle exec, means the library became a bundled gem. Add it to theGemfile(an application) or the gemspec (a library). -
Using
Timeoutas a safety net. It can corrupt state; use the operation’s own timeout. -
Treating
Digestas a password hash,randas a token source, or==as a MAC comparison. -
Calling
notify_observerswithoutchanged. -
Relying on
is_a?with a delegator. -
Assuming
WeakRefis deterministic. -
Inflating untrusted compressed data in one call.
-
Building an
ERBtemplate from user input, or printing untrusted values with<%= %>and no escaping. -
Logging secrets, or building an expensive message that a block would have avoided building.
Version notes
| Version | Note |
|---|---|
3.1 |
|
3.2 |
|
3.3 |
|
3.4 |
|
4.0 |
|
4.1 (not released) |
|
References
-
Ruby Standard Library (4.0): the list of core extensions, default gems and bundled gems
-
NEWS for Ruby 4.0, 3.4, 3.3, 3.2 and 3.1
-
Logger, ERB, Forwardable, SimpleDelegator, Singleton and observer
-
SecureRandom, Digest, OpenSSL and benchmark
-
PP, Shellwords, Timeout, WeakRef, Etc, Zlib and StringScanner