Testing

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

This page is written against Pyramid 2.1 (the 2.1-branch documentation and the 2.1 release installed from PyPI) on Python 3.13, checked on 2026-10-07, with WebTest 3.0.7, pytest 9.1 and pytest-cov 7.1. Every example was executed: the project files and the tests of the Bookshelf Shelves scenario run together under pytest (20 tests, all passing, 98 percent coverage with branch measurement), and the stand-alone snippets run as plain Python. Pyramid ships no test runner. It ships a small module of test doubles, pyramid.testing, a rule for pushing and popping its thread locals around a test, and an official recommendation to drive functional tests with WebTest. Fixtures, parametrization, mocks and coverage come from pytest, which is also what the cookiecutter starter project uses.

The other frameworks of this site cover the same ground with their own tools: Flask has a test client (Flask: Testing), Tornado ships AsyncHTTPTestCase (Tornado: Testing) and FastAPI uses TestClient (FastAPI: Testing). Pyramid’s test doubles exist because a Pyramid view is a plain function of a request, and because so much of the framework (route generation, permissions, renderers, events) is looked up in a registry that a unit test must populate by hand.

The Three Kinds of Test

The official chapter distinguishes three levels. The difference is how much of the real application a test runs:

Level What is real What is faked

Unit

The function under test: a view, a subscriber, a helper.

Everything it touches. The request is a DummyRequest, resources are DummyResource, and the security policy and the renderers are test doubles registered in an empty registry with the testing_* helpers of the configurator.

Integration

The code under test and its integration with the framework: a real registry populated by config.include(...) of your own includeme functions, real route generation, real request extensions.

External services and, usually, the HTTP layer (the view is still called directly).

Functional

The whole WSGI application built by main(...), driven over a fake HTTP connection by WebTest: routing, tweens, views, renderers, sessions and cookies.

Only what is outside the process: databases, remote APIs, mail, the clock.

The official guidance is to write all three. Unit tests reach every code path cheaply, functional tests prove that the parts fit together as they will in production. The usual shape is many unit tests, fewer integration tests and a thin layer of functional tests.

flowchart LR U["Unit test: view called directly with DummyRequest, registry built with testing helpers"] I["Integration test: registry built by config.include, real routes and request extensions"] F["Functional test: WebTest TestApp calls the full WSGI app made by main"] U -->|"more of the real stack, slower, fewer tests"| I I -->|"more of the real stack, slower, fewer tests"| F
Figure 1. Three levels of test and how much of the application each one runs

Source: Pyramid: Unit, Integration, and Functional Testing

The Application Under Test

The examples need something to test. The code below is a small slice of Bookshelf Shelves: an in-memory shelf store, a session-based security policy with one ACL, a client for an external ISBN catalogue and a handful of JSON views. It is deliberately plain, so that the tests, not the application, are the subject. The security policy is the one described in Security Policy and Authentication, the ACL is the one described in Authorization: ACLs and Permissions, and the includeme functions are the hooks of Configuration and the Configurator. The package is named bookshelf_shelves, like the project generated in Cookiecutter Projects and pserve.

# norun  (project file bookshelf_shelves/__init__.py)
from pyramid.config import Configurator
from pyramid.session import SignedCookieSessionFactory


def main(global_config, **settings):
    with Configurator(settings=settings) as config:
        config.set_session_factory(SignedCookieSessionFactory(settings["session.secret"]))
        config.include(".security")
        config.include(".routes")
        config.include(".store")
        config.include(".isbn")
        config.scan(".views")
        return config.make_wsgi_app()

Source: Pyramid: Creating a Pyramid Project

# norun  (project file bookshelf_shelves/store.py)
class ShelfStore:
    """An in-memory stand-in for the real data layer."""

    def __init__(self):
        self._shelves = {}

    def all(self):
        return [self._shelves[slug] for slug in sorted(self._shelves)]

    def get(self, slug):
        return self._shelves.get(slug)

    def add(self, slug, title, owner):
        shelf = {"slug": slug, "title": title, "owner": owner}
        self._shelves[slug] = shelf
        return shelf


def includeme(config):
    config.registry["shelf_store"] = ShelfStore()
    config.add_request_method(lambda request: request.registry["shelf_store"], "shelves", reify=True)

Source: API: Configurator.add_request_method

# norun  (project file bookshelf_shelves/security.py)
from pyramid.authentication import SessionAuthenticationHelper
from pyramid.authorization import ACLHelper, Allow, Authenticated, Everyone


class SessionSecurityPolicy:
    def __init__(self):
        self.helper = SessionAuthenticationHelper()

    def identity(self, request):
        return self.helper.authenticated_userid(request)

    def authenticated_userid(self, request):
        return self.identity(request)

    def permits(self, request, context, permission):
        principals = [Everyone]
        userid = self.identity(request)
        if userid is not None:
            principals += [Authenticated, f"user:{userid}"]
        return ACLHelper().permits(context, principals, permission)

    def remember(self, request, userid, **kw):
        return self.helper.remember(request, userid, **kw)

    def forget(self, request, **kw):
        return self.helper.forget(request, **kw)


class Root:
    __acl__ = [(Allow, Authenticated, "create")]

    def __init__(self, request):
        self.request = request


def includeme(config):
    config.set_security_policy(SessionSecurityPolicy())
    config.set_root_factory(Root)

Source: Pyramid: Security

# norun  (project file bookshelf_shelves/routes.py)
def includeme(config):
    config.add_route("shelves", "/shelves")
    config.add_route("shelf", "/shelves/{slug}")
    config.add_route("isbn", "/isbn/{isbn}")
    config.add_route("login", "/login")

Source: Pyramid: URL Dispatch

# norun  (project file bookshelf_shelves/isbn.py)
import json
import urllib.error
import urllib.parse
import urllib.request


class HttpIsbnClient:
    """Looks a title up in an external catalogue service."""

    def __init__(self, base_url, timeout=3.0):
        self.base_url = base_url.rstrip("/")
        self.timeout = timeout

    def title(self, isbn):
        url = f"{self.base_url}/{urllib.parse.quote(isbn, safe='')}"
        try:
            with urllib.request.urlopen(url, timeout=self.timeout) as response:
                return json.load(response)["title"]
        except urllib.error.HTTPError as exc:
            if exc.code == 404:
                return None
            raise


def includeme(config):
    client = HttpIsbnClient(config.registry.settings["isbn.base_url"])
    config.registry["isbn_client"] = client
    config.add_request_method(lambda request: request.registry["isbn_client"], "isbn_client", reify=True)

Source: Pyramid: Extending Pyramid Configuration

# norun  (project file bookshelf_shelves/views/shelves.py)
import hmac
import urllib.error

from pyramid.httpexceptions import (
    HTTPBadGateway, HTTPBadRequest, HTTPConflict, HTTPNotFound, HTTPUnauthorized,
)
from pyramid.security import remember
from pyramid.view import view_config


@view_config(route_name="shelves", request_method="GET", renderer="json")
def list_shelves(request):
    return request.shelves.all()


@view_config(route_name="shelves", request_method="POST", renderer="json", permission="create")
def create_shelf(request):
    data = request.json_body
    slug, title = data.get("slug", ""), data.get("title", "")
    if not slug.isidentifier() or not title:
        raise HTTPBadRequest(json_body={"error": "slug and title are required"})
    if request.shelves.get(slug) is not None:
        raise HTTPConflict(json_body={"error": "slug already used"})
    shelf = request.shelves.add(slug, title, owner=request.authenticated_userid)
    request.response.status = 201
    request.response.headers["Location"] = request.route_url("shelf", slug=slug)
    return shelf


@view_config(route_name="shelf", request_method="GET", renderer="json")
def show_shelf(request):
    shelf = request.shelves.get(request.matchdict["slug"])
    if shelf is None:
        raise HTTPNotFound()
    return {**shelf, "can_add": bool(request.has_permission("create"))}


@view_config(route_name="isbn", request_method="GET", renderer="json")
def isbn_title(request):
    isbn = request.matchdict["isbn"]
    try:
        title = request.isbn_client.title(isbn)
    except (urllib.error.URLError, TimeoutError):
        raise HTTPBadGateway(json_body={"error": "catalogue unavailable"})
    if title is None:
        raise HTTPNotFound()
    return {"isbn": isbn, "title": title}


@view_config(route_name="login", request_method="POST", renderer="json")
def login(request):
    settings = request.registry.settings
    data = request.json_body
    known = hmac.compare_digest(str(data.get("username", "")), settings["demo.username"])
    right = hmac.compare_digest(str(data.get("password", "")), settings["demo.password"])
    if not (known and right):
        raise HTTPUnauthorized(json_body={"error": "bad credentials"})
    request.response.headerlist.extend(remember(request, data["username"]))
    return {"user": data["username"]}

Source: Pyramid: View Configuration

Three design choices matter for testing, and they are the ones to copy:

  • Collaborators are looked up through the request or the registry. The views never import a global store or build an HTTP client. They read request.shelves and request.isbn_client, which includeme registers with add_request_method from objects kept in the registry. A unit test hands the view a fake through a DummyRequest keyword argument, and a functional test swaps the registry entry.

  • Configuration is split into small includeme functions. An integration test can include just the routes, or just the store, without building the whole application.

  • Secrets come from settings. The session secret and the demo password are read from settings and are never written in testing.ini: the fixtures below generate them with the secrets module for each test run. A real application compares password hashes against a user table, as in SQLAlchemy, Transactions and Migrations; the plain comparison here only keeps the example short.

Setting Up pytest and WebTest

The cookiecutter starter declares the testing tools as an optional dependency group, so a project is prepared for tests with one command. The tests_require list of setup.py shown in parts of the official chapter has been replaced by pyproject.toml (see What Changed and Where Older Material Is Outdated):

python -m venv env
env/bin/pip install -e ".[testing]"
env/bin/pytest

Source: Pyramid: Creating Functional Tests

The extra installs webtest, pytest and pytest-cov. The rest of the configuration is a few lines of pyproject.toml. These are the values used for this page; the generated project has similar ones.

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.coverage.run]
source = ["bookshelf_shelves"]
branch = true

Source: Pyramid: Creating a Pyramid Project

The test settings live in testing.ini, a copy of production.ini without the server section, in which the debug helpers are off. Nothing sensitive goes in it:

[app:main]
use = egg:bookshelf_shelves

pyramid.reload_templates = false
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en

demo.username = ada
isbn.base_url = http://127.0.0.1:9

Source: Pyramid: Environment Variables and .ini File Settings

The isbn.base_url points at the discard port of the loopback interface, so a test that forgets to replace the catalogue fails fast instead of reaching a real service. The .ini format and the settings are explained in Settings, Environment and .ini Files.

Unit Tests with pyramid.testing

Thread Locals: setUp, tearDown and testConfig

Pyramid keeps two thread-local values, the current request and the current registry, and a surprising number of APIs read them: render_to_response, route_url without an explicit request, get_current_request and every method of the configurator you use in a test. The rule of the official chapter is simple. If the code under test, or the test itself, uses the Configurator or any get_current_* function, push a fresh registry in the test setup and pop it in the teardown. pyramid.testing.setUp() does the first and pyramid.testing.tearDown() the second. setUp returns an autocommitting Configurator, so every config.add_route(...) takes effect immediately and no commit() is needed.

import unittest
from pyramid import testing
from pyramid.threadlocal import get_current_registry, get_current_request


class ThreadLocalTest(unittest.TestCase):
    def setUp(self):
        self.request = testing.DummyRequest()
        self.config = testing.setUp(request=self.request)

    def tearDown(self):
        testing.tearDown()

    def test_thread_locals_are_pushed(self):
        self.assertIs(get_current_request(), self.request)
        self.assertIs(get_current_registry(), self.config.registry)


suite = unittest.defaultTestLoader.loadTestsFromTestCase(ThreadLocalTest)
result = unittest.TextTestRunner(verbosity=0).run(suite)
assert result.wasSuccessful()
print(get_current_request())    # None again: tearDown popped the stack

Source: Pyramid: Test Set Up and Tear Down

The unittest style above is the one of the official chapter, and it still works with pytest, which collects unittest.TestCase classes. The idiomatic pytest form is a fixture. The cookiecutter’s dummy_config fixture wraps testConfig, a context manager that calls setUp on entry and tearDown on exit, even when the test fails:

# norun  (the dummy_config fixture of tests/conftest.py, shown in full below)
@pytest.fixture
def dummy_config(dummy_request):
    with testConfig(request=dummy_request) as config:
        yield config

Source: API: pyramid.testing.testConfig

The arguments of setUp and testConfig are registry, request, hook_zca, autocommit and settings; setUp also takes package:

Argument Effect

registry

Use this registry instead of a new empty one.

request

The object returned by get_current_request() during the test. Without it that function returns None.

hook_zca

When true (the default) and zope.component is importable, the global ZCA API (getSiteManager, getAdapter) is pointed at the test registry. tearDown(unhook_zca=True) undoes it.

autocommit

True by default, so configuration calls take effect at once. Pass false to test code that needs the deferred, conflict-detecting behaviour of a normal Configurator.

settings

A dictionary used as the settings of the configurator, so that request.registry.settings has the values the code under test reads.

package

The package used to resolve relative includes such as config.include(".routes"). It defaults to the caller’s package.

cleanUp is an alias of setUp. The register* helpers of the BFG era (registerDummySecurityPolicy and friends) were removed: use the Configurator.testing_* methods instead (see What Changed and Where Older Material Is Outdated).

DummyRequest and How Faithful It Is

DummyRequest imitates a request with the minimum of attributes: params, GET, POST, headers, cookies, environ, path, matchdict, context, session (a DummySession), registry and response, plus the URL methods (route_url, route_path, static_url), the security methods (has_permission, authenticated_userid, identity), localizer and add_response_callback. Any extra keyword argument becomes an attribute, which is how DummyRequest(shelves=store, json_body={...}) supplied the collaborators in the views above. Passing post= fills POST and sets the method to POST.

It is not a real request, and the official API page says so: use pyramid.request.Request when the code under test needs the whole object. The following snippet shows three of the differences that matter most, and the instance-level override of a request property that is useful for mocking:

from pyramid import testing
from pyramid.request import Request, apply_request_extensions
from pyramid.threadlocal import get_current_request

print(get_current_request())                                 # None: nothing was pushed

dummy = testing.DummyRequest(params={"q": "dune"})
print(type(dummy.GET).__name__, hasattr(dummy, "json_body"))
real = Request.blank("/shelves?q=dune")
print(type(real.GET).__name__)

with testing.testConfig() as config:
    config.add_request_method(lambda request: "the real backend", "backend", property=True)
    request = Request.blank("/")
    request.registry = config.registry
    apply_request_extensions(request)
    print(request.backend)
    request.backend = "a fake"            # since Pyramid 2.0 a property can be overridden on the instance
    print(request.backend)
    del request.backend
    print(request.backend)

Source: API: pyramid.testing.DummyRequest

The output is:

None
dict False
GetDict
the real backend
a fake
the real backend

The differences to remember:

Difference Consequence

GET and POST are plain dictionaries, not WebOb multi-dicts.

request.params.getall("tag") and request.POST.getone("title") fail. Replace the attribute with a MultiDict, or use a real Request.blank(...).

No request extensions and no thread locals.

request.shelves and any other add_request_method value is missing unless you pass it as a keyword. A request made by prepare(registry=app.registry), or by Request.blank plus apply_request_extensions, has them.

json_body and body are not parsed.

Pass json_body= or body= yourself, as the unit tests below do.

route_url and route_path need the route to exist.

Register it first with config.add_route(...) on the setUp configurator. Without any registry the call raises ComponentLookupError.

session is a DummySession, so it exists even when the application has no session factory.

On a real request, request.session without config.set_session_factory(...) raises AttributeError. A unit test cannot catch a forgotten factory; a functional test does.

Since Pyramid 2.0 a request property added with add_request_method(..., property=True), or with request.set_property, can be overridden on the instance and restored with del, as the last lines of the snippet show. The 2.0 release notes name mocking request properties in tests as the reason.

Source: What’s New in Pyramid 2.0

Dummy Resources, Sessions, Renderers, Events and Security

The Configurator returned by setUp has four testing_* methods. Each registers a stand-in for a real component and returns an object to assert against. Next to them are the dummy classes:

Method or class Use

testing_securitypolicy(userid=None, identity=None, permissive=True, remember_result=None, forget_result=None)

Registers a fake security policy. permissive=False makes every has_permission call and every permission= check fail. userid is what request.authenticated_userid returns. remember_result and forget_result are the header lists that remember and forget return.

testing_add_renderer(path, renderer=None)

Registers a fake renderer for a template or renderer path and returns a DummyTemplateRenderer. Its assert_(**kw) checks that the view passed those keys and values to the renderer.

testing_add_subscriber(event_iface=None)

Registers a subscriber that appends matching events to the returned list. With no argument it collects every event.

testing_resources(resources)

Takes a dictionary from path to resource, so that find_resource finds them without a real tree.

DummyResource(__name__=None, __parent__=None, __provides__=None, **kw)

A resource with the mapping protocol: root["dune"] = child sets the child’s name and parent. Keywords become attributes, __provides__ attaches an interface for context= view lookups, and clone() copies the resource.

DummySession

A dictionary with the session API: flash, pop_flash, peek_flash, changed, invalidate, get_csrf_token and new_csrf_token.

Source: API: Configurator.testing_securitypolicy

All of them in one view and one test:

from pyramid import testing
from pyramid.renderers import render_to_response
from pyramid.security import forget, remember
from pyramid.traversal import find_resource


class ShelfCreated:
    def __init__(self, shelf):
        self.shelf = shelf


def book_page(request):
    book = request.context["dune"]
    request.session.flash(f"Opened {book.title}")
    request.registry.notify(ShelfCreated(book))
    return render_to_response("templates/book.pt", {"book": book}, request=request)


with testing.testConfig() as config:
    root = testing.DummyResource()
    root["dune"] = testing.DummyResource(title="Dune")      # keyword arguments become attributes
    renderer = config.testing_add_renderer("templates/book.pt")
    events = config.testing_add_subscriber(ShelfCreated)

    request = testing.DummyRequest()
    request.context = root
    book_page(request)

    renderer.assert_(book=root["dune"])                      # what the template would have received
    assert [event.shelf.title for event in events] == ["Dune"]
    assert request.session.peek_flash() == ["Opened Dune"]
    assert root["dune"].__name__ == "dune" and root["dune"].__parent__ is root

    config.testing_resources({"/classics": testing.DummyResource(title="Classics")})
    assert find_resource(root, "/classics").title == "Classics"

    config.testing_securitypolicy(userid="ada", remember_result=[("Set-Cookie", "demo=1")], forget_result=[("Set-Cookie", "demo=; Max-Age=0")])
    assert remember(request, "ada") == [("Set-Cookie", "demo=1")]
    assert forget(request) == [("Set-Cookie", "demo=; Max-Age=0")]
    assert request.authenticated_userid == "ada"
print("ok")

Source: API: pyramid.testing

The DummySession returns a constant token in the installed 2.1 source. That is convenient and also a trap: a test that passes with the dummy token proves nothing about token handling. Test CSRF through WebTest, as shown below.

Unit-Testing the Bookshelf Views

With these tools the views of the scenario are tested without HTTP and without building the application. Notice what each test supplies: a store, a route for route_url, and a security policy for authenticated_userid and has_permission.

# norun  (project file tests/test_unit_views.py)
import pytest
from pyramid.httpexceptions import HTTPBadRequest, HTTPConflict, HTTPNotFound
from pyramid.testing import DummyRequest, DummyResource

from bookshelf_shelves.store import ShelfStore
from bookshelf_shelves.views.shelves import create_shelf, list_shelves, show_shelf


@pytest.fixture
def store():
    store = ShelfStore()
    store.add("classics", "Classics", "ada")
    return store


def test_list_shelves(store):
    request = DummyRequest(shelves=store)
    assert [shelf["slug"] for shelf in list_shelves(request)] == ["classics"]


def test_create_shelf(dummy_config, store):
    dummy_config.add_route("shelf", "/shelves/{slug}")
    dummy_config.testing_securitypolicy(userid="ada")
    request = DummyRequest(shelves=store, json_body={"slug": "scifi", "title": "Science fiction"})

    shelf = create_shelf(request)

    assert shelf == {"slug": "scifi", "title": "Science fiction", "owner": "ada"}
    assert request.response.status_int == 201
    assert request.response.headers["Location"] == "http://example.com/shelves/scifi"


@pytest.mark.parametrize(
    "body, expected",
    [
        ({"slug": "not valid", "title": "X"}, HTTPBadRequest),
        ({"slug": "classics", "title": "X"}, HTTPConflict),
    ],
)
def test_create_shelf_rejects(dummy_config, store, body, expected):
    request = DummyRequest(shelves=store, json_body=body)
    with pytest.raises(expected):
        create_shelf(request)


def test_show_shelf_not_found(store):
    request = DummyRequest(shelves=store)
    request.matchdict = {"slug": "missing"}
    with pytest.raises(HTTPNotFound):
        show_shelf(request)


@pytest.mark.parametrize("permissive", [True, False])
def test_show_shelf_can_add(dummy_config, store, permissive):
    dummy_config.testing_securitypolicy(userid="grace", permissive=permissive)
    request = DummyRequest(shelves=store)
    request.matchdict = {"slug": "classics"}
    request.context = DummyResource()

    assert show_shelf(request)["can_add"] is permissive

Source: Pyramid: Using the Configurator and pyramid.testing APIs in Unit Tests

Calling a view that raises an HTTP exception is a normal pytest.raises, because HTTPNotFound and its siblings are Python exceptions as well as responses. How the router turns them into responses, and how to test the exception views that shape them, is in Errors and Exception Views. The unit tests above do not exercise the permission="create" of the create_shelf view: the router checks the permission before it calls the view, so only an integration or functional test sees it.

Integration Tests

A Pyramid integration test builds a registry the way the application does, by including configuration, and checks how the pieces behave together. Two useful forms need no HTTP. The first includes one module on the testConfig configurator, so that only that part of the configuration is active. The second builds the whole application with main(...). make_wsgi_app() ends in commit(), so conflicting directives in your includes raise there (see the two-phase configuration in Configuration and the Configurator), and the introspector lets the test assert on what was configured:

# norun  (project file tests/test_integration.py)
from pyramid.testing import DummyRequest

from bookshelf_shelves.views.shelves import list_shelves


def test_routes_module_registers_the_route_table(dummy_config):
    dummy_config.include("bookshelf_shelves.routes")
    assert DummyRequest().route_path("shelf", slug="classics") == "/shelves/classics"


def test_the_whole_application_is_configured_without_conflicts(app):
    # main() ends in make_wsgi_app(), which commits: conflicting directives would raise here
    routes = app.registry.introspector.get_category("routes")
    assert {item["introspectable"]["name"] for item in routes} == {"shelves", "shelf", "isbn", "login"}


def test_request_extensions_are_applied_to_a_real_request(app, app_request):
    assert app_request.shelves is app.registry["shelf_store"]
    assert list_shelves(app_request) == []

Source: Pyramid: Creating Integration Tests

The third test uses the app_request fixture (defined with the others below): a real request made by pyramid.scripting.prepare(registry=app.registry). prepare pushes the thread locals, applies the request extensions and returns a dictionary with root, closer, registry, request and root_factory. Used as a context manager it calls closer for you. It is the best tool when a unit test needs a request on which request.shelves, request.route_url and request.has_permission behave as in production.

Source: API: pyramid.scripting

The official chapter prefers unit tests with testing_* registrations over integration tests where both would do: they are faster and easier to reason about. Reach for integration tests for what a unit test cannot see: that the includes compose, that a route name used in a view exists, that predicates do not collide.

Functional Tests with WebTest

The TestApp

webtest.TestApp(app) wraps any WSGI application, including the one returned by main(...), and exposes it as an HTTP client that never opens a socket. The members used most are:

Call What it does

testapp.get(url, params=..., headers=..., status=...)

Performs a request. status= asserts the status (an integer, a list of integers, or a pattern such as "3*" or "*") and raises AppError on a mismatch. Without it WebTest accepts only 2xx and 3xx and raises for 4xx and 5xx unless expect_errors=True.

post, put, patch, delete, options, head

The other methods. post(url, {"title": "x"}) sends a form body.

post_json, put_json, patch_json, delete_json

Send a JSON body with the right Content-Type. res.json parses a JSON response.

res.json, res.text, res.body, res.headers, res.html

The response in several forms. res.html is a BeautifulSoup tree.

res.follow(), res.maybe_follow()

Follow the Location of a redirect. maybe_follow follows as many redirects as there are.

res.forms, res.form

The HTML forms of the page. form["name"] = value sets a field and form.submit() posts it with the hidden fields and the cookies.

res.click(...), res.goto(href)

Follow a link by its text or href, or go to an arbitrary URL.

testapp.cookies, testapp.set_cookie(name, value), testapp.reset()

The cookie jar is kept between requests, like a browser’s. reset empties it.

webtest.TestApp(app, extra_environ={...}, lint=True)

extra_environ adds to the WSGI environ of every request. lint (true by default) validates the application’s behaviour against the WSGI specification, which catches malformed headers early.

Source: WebTest: TestApp

Fixtures: The Cookiecutter conftest.py

The starter project’s tests/conftest.py defines, in order, a --ini command-line option (so that the suite can run against another file), the ini_file, app_settings and app fixtures, testapp, and the app_request, dummy_request and dummy_config fixtures of the previous sections. The version below is the one used for this page. It differs from the generated file in three ways that are worth copying. The secrets are generated per run instead of being written in testing.ini, the app has function scope so that every test starts with an empty store, and there are a login helper and a loopback fake of the catalogue.

# norun  (project file tests/conftest.py)
import json
import os
import secrets
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer

import pytest
import webtest
from pyramid.paster import get_appsettings
from pyramid.scripting import prepare
from pyramid.testing import DummyRequest, testConfig

from bookshelf_shelves import main


def pytest_addoption(parser):
    parser.addoption("--ini", action="store", metavar="INI_FILE")


@pytest.fixture(scope="session")
def ini_file(request):
    return os.path.abspath(request.config.option.ini or "testing.ini")


@pytest.fixture(scope="session")
def app_settings(ini_file):
    settings = get_appsettings(ini_file)
    # secrets live in the fixture, never in testing.ini: a fresh value per test run
    settings["session.secret"] = secrets.token_hex(32)
    settings["demo.password"] = secrets.token_urlsafe(16)
    return settings


@pytest.fixture
def app(app_settings):
    # function scope: a new registry, and so a new empty store, for every test
    return main({}, **app_settings)


@pytest.fixture
def testapp(app):
    return webtest.TestApp(app, extra_environ={"HTTP_HOST": "example.com"})


@pytest.fixture
def login(testapp, app_settings):
    def do_login():
        credentials = {"username": app_settings["demo.username"], "password": app_settings["demo.password"]}
        testapp.post_json("/login", credentials, status=200)

    return do_login


@pytest.fixture
def app_request(app):
    """A real request, with request extensions and thread locals."""
    with prepare(registry=app.registry) as env:
        request = env["request"]
        request.host = "example.com"
        yield request


@pytest.fixture
def dummy_request():
    """A lightweight request: no request extensions, no thread locals."""
    request = DummyRequest()
    request.host = "example.com"
    return request


@pytest.fixture
def dummy_config(dummy_request):
    """A configurator that pushes the thread locals and is popped afterwards."""
    with testConfig(request=dummy_request) as config:
        yield config


@pytest.fixture
def fake_catalogue():
    """A real HTTP server on an unused loopback port that plays the external catalogue."""
    titles = {"9780000000001": "Dune"}

    class Handler(BaseHTTPRequestHandler):
        def do_GET(self):
            title = titles.get(self.path.lstrip("/"))
            self.send_response(200 if title else 404)
            self.send_header("Content-Type", "application/json")
            self.end_headers()
            self.wfile.write(json.dumps({"title": title}).encode())

        def log_message(self, *args):
            pass

    server = HTTPServer(("127.0.0.1", 0), Handler)
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()
    yield f"http://127.0.0.1:{server.server_port}"
    server.shutdown()
    server.server_close()
    thread.join()

Source: API: pyramid.paster

get_appsettings(config_uri, name=None, options=None) returns the settings of the app section of an .ini file as a dictionary. Its options argument supplies the variables for %(name)s interpolation in the file, so an alternative to the fixture above is to write demo.password = %(demo_password)s in testing.ini and pass options={"demo_password": ...}. The HTTP_HOST of extra_environ makes the host stable, which matters for route_url output and for the origin checks of CSRF Protection.

Login, Cookies and JSON

The TestApp remembers the Set-Cookie of the login response and sends it back, so a session-based application is tested the way a browser would use it:

# norun  (project file tests/test_functional.py)
import urllib.error

import pytest
import webtest

from bookshelf_shelves import main


def test_the_list_starts_empty(testapp):
    assert testapp.get("/shelves", status=200).json == []


def test_anonymous_users_cannot_create(testapp):
    testapp.post_json("/shelves", {"slug": "scifi", "title": "Science fiction"}, status=403)


def test_bad_credentials(testapp):
    res = testapp.post_json("/login", {"username": "ada", "password": "wrong"}, status=401)
    assert res.json == {"error": "bad credentials"}


def test_login_create_and_read(testapp, login):
    login()                                        # the TestApp keeps the session cookie
    res = testapp.post_json("/shelves", {"slug": "scifi", "title": "Science fiction"}, status=201)
    assert res.headers["Location"] == "http://example.com/shelves/scifi"
    assert testapp.get("/shelves/scifi").json["can_add"] is True

    testapp.reset()                                # forget the cookie: back to anonymous
    assert testapp.get("/shelves/scifi").json["can_add"] is False


def test_validation_and_conflicts(testapp, login):
    login()
    res = testapp.post_json("/shelves", {"slug": "no spaces", "title": "X"}, status=400)
    assert res.json == {"error": "slug and title are required"}
    testapp.post_json("/shelves", {"slug": "scifi", "title": "A"}, status=201)
    testapp.post_json("/shelves", {"slug": "scifi", "title": "B"}, status=409)


def test_unknown_shelf_is_a_404(testapp):
    testapp.get("/shelves/missing", status=404)


def test_each_test_gets_a_fresh_store(testapp):
    assert testapp.get("/shelves").json == []      # the shelves of the other tests are gone


class FakeCatalogue:
    def __init__(self, titles=None, error=None):
        self.titles, self.error = titles or {}, error

    def title(self, isbn):
        if self.error:
            raise self.error
        return self.titles.get(isbn)


def test_catalogue_replaced_in_the_registry(testapp, app, monkeypatch):
    monkeypatch.setitem(app.registry, "isbn_client", FakeCatalogue({"1": "Dune"}))
    assert testapp.get("/isbn/1").json == {"isbn": "1", "title": "Dune"}
    testapp.get("/isbn/2", status=404)


def test_catalogue_outage_is_a_502(testapp, app, monkeypatch):
    fake = FakeCatalogue(error=urllib.error.URLError("connection refused"))
    monkeypatch.setitem(app.registry, "isbn_client", fake)
    assert testapp.get("/isbn/1", status=502).json == {"error": "catalogue unavailable"}


def test_real_client_against_a_loopback_server(app_settings, fake_catalogue):
    app = main({}, **{**app_settings, "isbn.base_url": fake_catalogue})
    testapp = webtest.TestApp(app)
    assert testapp.get("/isbn/9780000000001").json["title"] == "Dune"
    testapp.get("/isbn/unknown", status=404)

Source: WebTest: API

Two idioms are visible. Asserting the status in the call (status=403) is the one to prefer, because the failure message then shows the response body. testapp.reset() returns the same client to the anonymous state without building a new application.

Forms, Redirects and CSRF Tokens

Applications that render HTML forms are tested through the form, not around it. The hidden CSRF field is posted automatically when the test submits the form it fetched, and a request that skips the form is rejected. The snippet is a self-contained miniature of the Bookshelf Shelves "add a shelf" page with require_csrf=True, a flash message and a redirect:

import html
import secrets

import webtest
from pyramid.config import Configurator
from pyramid.csrf import get_csrf_token
from pyramid.httpexceptions import HTTPSeeOther
from pyramid.response import Response
from pyramid.session import SignedCookieSessionFactory

SHELVES = []


def page(request):
    flashes = "".join(f"<p class='flash'>{html.escape(m)}</p>" for m in request.session.pop_flash())
    items = "".join(f"<li>{html.escape(title)}</li>" for title in SHELVES)
    form = (
        '<form method="post" action="/shelves/new">'
        f'<input type="hidden" name="csrf_token" value="{get_csrf_token(request)}">'
        '<input type="text" name="title"><button type="submit">Add shelf</button></form>'
    )
    return Response(f"<html><body>{flashes}<ul>{items}</ul>{form}</body></html>")


def add(request):
    title = request.POST.get("title", "").strip()
    if title:
        SHELVES.append(title)
        request.session.flash(f"Shelf {title} added")
    return HTTPSeeOther(request.route_path("shelves"))


with Configurator() as config:
    config.set_session_factory(SignedCookieSessionFactory(secrets.token_hex(32)))
    config.set_default_csrf_options(require_csrf=True)
    config.add_route("shelves", "/shelves")
    config.add_route("add", "/shelves/new")
    config.add_view(page, route_name="shelves")
    config.add_view(add, route_name="add", request_method="POST")
    app = config.make_wsgi_app()

testapp = webtest.TestApp(app)
res = testapp.get("/shelves")
form = res.forms[0]                       # or res.form when the page has a single form
form["title"] = "Classics & more"
res = form.submit().follow()              # 303 -> GET /shelves
assert res.status_int == 200
assert "Shelf Classics &amp; more added" in res.text and "<li>Classics &amp; more</li>" in res.text
assert "session" in testapp.cookies

# a POST without the token never reaches the view
testapp.post("/shelves/new", {"title": "Forged"}, status=400)
assert SHELVES == ["Classics & more"]
print("ok", res.html.find("p", class_="flash").text)

Source: Pyramid: Preventing Cross-Site Request Forgery Attacks

The same test could not be written with a DummyRequest: nothing there runs the CSRF view deriver. Checking that a protected view is really protected is a functional test. The mechanism and its options are in CSRF Protection, and the session and flash APIs used here are in Sessions and Flash Messages.

Secure Cookies Need HTTPS in the Test

A session factory built with secure=True sets the Secure flag, and the TestApp’s cookie jar honours it: over plain http the cookie is never sent back. The next snippet shows the symptom and the cure, which is to set the URL scheme in the WSGI environ:

import secrets, webtest
from pyramid.config import Configurator
from pyramid.session import SignedCookieSessionFactory

def hit(request):
    request.session["n"] = request.session.get("n", 0) + 1
    return {"n": request.session["n"]}

with Configurator() as config:
    config.set_session_factory(SignedCookieSessionFactory(secrets.token_hex(32), secure=True))
    config.add_route("hit", "/hit")
    config.add_view(hit, route_name="hit", renderer="json")
    app = config.make_wsgi_app()

http = webtest.TestApp(app)
print([http.get("/hit").json["n"] for _ in range(2)])
https = webtest.TestApp(app, extra_environ={"wsgi.url_scheme": "https"})
print([https.get("/hit").json["n"] for _ in range(2)])

Source: WebTest: TestApp

It prints:

[1, 1]
[1, 2]

The first client never gets its session back (the counter restarts at one on every request) and the second behaves like a browser on HTTPS. The same environ key makes request.scheme and request.url report https, which the Origin and Referer checks of the CSRF origin test also need.

Isolating Tests

Tests that share state fail in an order-dependent way. The sources of shared state in a Pyramid application, and the remedy for each, are:

Shared state Remedy

The application registry and anything kept in it: the shelf store of this scenario, a cache, a connection pool.

Build app per test (function scope), as the conftest.py above does. It is cheap when startup does not connect to anything. With a session-scoped app (the cookiecutter default) reset the stores explicitly in a fixture.

The thread locals.

setUp and tearDown, or testConfig, so that a test cannot leak a registry into the next one.

Cookies in a TestApp.

Create the TestApp per test, as the testapp fixture does, or call testapp.reset().

A relational database.

Run each test inside a transaction that is rolled back. The cookiecutter’s sqlalchemy backend does it with a doomed transaction.TransactionManager, a session joined to it and a TestApp whose environ says tm.active. It is described in the section Testing with Transaction Rollback of SQLAlchemy, Transactions and Migrations.

A ZODB database.

Point the application at an in-memory storage with a memory:// URI in testing.ini, as in ZODB and Other Data Stores.

Module-level globals and the environment.

monkeypatch.setenv, monkeypatch.setattr and monkeypatch.setitem, which undo themselves at the end of the test.

Source: pytest: How to use fixtures

The test test_each_test_gets_a_fresh_store in the functional tests above is the canary: it passes only while the other tests leave nothing behind.

Faking External Services

An application that calls a remote API needs tests that never leave the machine. Pyramid does not prescribe a mechanism. Three work well, from the cheapest to the most realistic:

  • Replace the collaborator in the registry. monkeypatch.setitem(app.registry, "isbn_client", fake) swaps the object that request.isbn_client reads. The fake needs only the methods the view calls, and it can raise the exceptions the real client raises. This is what test_catalogue_replaced_in_the_registry and test_catalogue_outage_is_a_502 do, and it is why the client is read from the registry and not imported.

  • Override the request property. On a request built with prepare or Request.blank, assigning request.isbn_client = fake replaces the value for that instance (see the 2.0 note above). It suits integration tests.

  • Run a real HTTP server on a loopback port. The fake_catalogue fixture starts http.server.HTTPServer on port 0 (the operating system picks a free one) in a thread, and the test points isbn.base_url at it. It exercises the real client code, its URL quoting and its handling of status codes, and it is the only way to catch a bug inside the client itself.

The mixture is deliberate: most tests use the cheap fake and one or two use the loopback server. Never let a test call a real third-party service. It is slow and flaky, and it may spend money or leak data. Pyramid itself is synchronous, so the tests of its views remain synchronous even when a view starts threads.

Source: pytest: How to monkeypatch and mock modules and environments

Running Tests and Measuring Coverage

The commands used most in daily work, from the project root:

env/bin/pytest                                  # everything under testpaths
env/bin/pytest tests/test_unit_views.py -x      # one file, stop at the first failure
env/bin/pytest -k "login and not unit"          # select by name
env/bin/pytest --lf                             # only what failed last time
env/bin/pytest --ini testing.ini tests          # the option of conftest.py: give the path explicitly
env/bin/pytest --cov --cov-report=term-missing  # coverage with the missing line numbers
env/bin/pytest --cov --cov-fail-under=90        # fail below a threshold, for CI

Source: pytest-cov documentation

The scenario’s suite reports:

Name                                  Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------
bookshelf_shelves/__init__.py            11      0      0      0   100%
bookshelf_shelves/isbn.py                21      1      2      1    91%   22
bookshelf_shelves/routes.py               5      0      0      0   100%
bookshelf_shelves/security.py            26      1      2      0    96%   26
bookshelf_shelves/store.py               14      0      0      0   100%
bookshelf_shelves/views/__init__.py       0      0      0      0   100%
bookshelf_shelves/views/shelves.py       46      0     10      0   100%
---------------------------------------------------------------------------------
TOTAL                                   123      2     14      1    98%
20 passed

Two lines are missing, and they are the useful ones: the branch of the catalogue client that re-raises an HTTP error other than 404 (isbn.py) and the forget method of the policy (security.py). Coverage tells you what no test ran, not that what ran is correct. Treat the number as a floor for a CI gate, keep branch = true, and read the missing lines.

The --ini option needs the explicit path (tests) in the command above. pytest reads the pytest_addoption hook of tests/conftest.py only after it has found the file, and with testpaths alone it rejects the unknown option with "unrecognized arguments". Putting the hook in a conftest.py at the project root avoids the need.

Source: coverage.py documentation

Testing Specific Features

Feature How to test it

Authorization

Unit: testing_securitypolicy(userid=..., permissive=...). Functional: log in as different users and assert 403 versus 2xx. Test the real ACLs with a functional test, or by calling ACLHelper().permits(context, principals, permission) on your resource. The details are in Authorization: ACLs and Permissions.

Authentication

Functional: post the login, then check the cookie and a protected route. Also check that a wrong password does not set a session. Never assert only on the unit-test dummy.

Exception views and error pages

Functional: testapp.get("/missing", status=404) and check the body of the 404 view. For API errors compare res.json. See Errors and Exception Views.

CSRF

Functional only: submit the form (token present) and post without it (status=400), as above. See CSRF Protection.

Sessions and flash messages

Unit: DummySession (request.session.peek_flash()). Functional: follow the redirect and check the page, as above. See Sessions and Flash Messages.

Renderers and templates

Unit: testing_add_renderer and assert_ for the data the view returns. Functional: assert on res.text or res.json. See Renderers and JSON and Templates.

Static assets

Functional: testapp.get("/static/theme.css", status=200) and check the Content-Type and cache headers. See Static Assets.

Routes and URL generation

Integration: include the routes and assert request.route_path(...), as in the integration test above. See URL Dispatch.

Traversal

Unit with DummyResource trees, or testing_resources for path lookups. See Traversal and Resources.

Events, tweens and request hooks

Unit: testing_add_subscriber for the events your code sends. Functional: tweens and subscribers run only in a real application, so test their effect through a request (a header added, a transaction rolled back). The section has a later page on them.

Command-line scripts

Call the function with a real request made by prepare, or run the script in a subprocess against testing.ini. See Command Line & p-scripts.

Source: Pyramid: Unit, Integration, and Functional Testing

Pitfalls

  • Forgetting tearDown. A registry that is not popped leaks into the next test, which then passes or fails depending on the order. Use testConfig or a fixture, so that the teardown is not optional.

  • Testing a view only with DummyRequest. It has no CSRF check, no tweens, no permission check, no request extensions, a session that always exists and dict parameters. Each of those can hide a bug from a green unit suite. Keep at least one functional test per view.

  • testing_securitypolicy hides the real policy. A permissive dummy proves that the view works for someone who is allowed. It says nothing about who is allowed. Test the ACL itself separately.

  • Autocommit hides conflicts. The setUp configurator commits at once, so a test that includes two modules which register the same route passes, while the application fails at startup. The integration test that builds the app with main(...) is what catches it.

  • A session-scoped application with mutable state. The cookiecutter’s app fixture is session-scoped, so a store kept in the registry is shared by every test. Reset it, or use function scope.

  • Hardcoded secrets in testing.ini or in fixtures. They are copied into other projects, committed for good and flagged by scanners. Generate them with secrets.token_hex() per run, or read them from the environment.

  • Real network access. A test that calls a real service fails on a train and passes on the build server, or the other way round. Replace the client, and use http://127.0.0.1:9 as the default base URL of the test settings.

  • Debug helpers in the test settings. pyramid_debugtoolbar and the pyramid.debug_* settings change responses (extra HTML, extra logging). Keep them out of testing.ini so that the tests see what production serves.

  • Time and randomness. Tests that compare timestamps or random identifiers are flaky. Inject the clock and the generator through the registry, like the store, and replace them in tests.

  • Asserting on the wrong layer. status= in the call, res.json for APIs and res.text for HTML are cheap and precise. Avoid asserting on whole HTML pages: assert on a stable fragment, or on an element found through res.html.

  • Slow suites. Building the application per test is fine until startup does real work. Then build once per session and isolate the state instead.

What Changed and Where Older Material Is Outdated

  • The Pyramid change history records the move from nose and coverage to pytest and pytest-cov, and the project and its documentation use pytest today. Instructions that say python setup.py test or nosetests no longer apply.

  • The 2.1 starter project uses pyproject.toml for packaging and for the pytest and coverage configuration, and the 2.1 release notes say so. The official testing chapter still shows a setup.py with tests_require and an extras_require entry. The idea (a testing extra) is unchanged and the file is not, so do not look for a setup.py in a new project.

  • The register* helpers of pyramid.testing (registerDummySecurityPolicy and the others) were removed in favour of Configurator.testing_securitypolicy, testing_add_renderer, testing_add_subscriber and testing_resources.

  • The 2.1 signature of testing_securitypolicy is userid, identity, permissive, remember_result and forget_result, in line with the single security policy of Pyramid 2.0. Tutorials for the 1.x authentication policies pass other arguments, such as group identifiers, and register a dummy authentication policy instead.

  • Pyramid 2.0 made request properties added with add_request_method(..., property=True) or request.set_property overridable per instance, which simplifies mocking them.

  • DummyRequest has gained attributes over the years (domain, script_name, charset, a session, response, request callbacks and the URL, security and localization methods). A workaround in an old test for the absence of one of them can go.

  • The official chapter says that unittest is the suggested mechanism, and the 2.1 starter project is pytest-first with fixtures. Both work, and this page uses pytest.

  • Material that builds the TestApp once per module, or with a config: URI such as testing.ini#main, still works. The per-test fixtures shown here are easier to isolate.

Source: What’s New in Pyramid 2.1

References