Playwright Test Agents

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

Most of the AI features in this section let an agent drive a browser while it works: Playwright MCP & CLI gives a coding agent a browser to explore and verify with, and How AI Agents Drive Browsers explains the mechanics. Playwright Test Agents solve a narrower problem: they produce and maintain an ordinary Playwright test suite. The output of the process is not a chat answer or a one-off exploration but files in your repository (a Markdown test plan and .spec.ts tests) that run in CI like any other test.

This page is written for Playwright 1.64.1 (Node.js) and was checked on 25 October 2026. Test Agents were introduced in Playwright 1.56. Every command, file and output below was produced against the Bookshelf application with Playwright 1.64.1 and its Chromium. Two things were not run: the agents inside a real Claude Code, Codex, Copilot or OpenCode session (see the next section for exactly what was run and how it was driven), and the Copy prompt button inside UI Mode and the trace viewer, which need a window (the HTML report was run, headless).

How the examples on this page were produced

Test Agents are not a new execution engine. They are three agent definition files, one MCP server, and a convention for where plans and tests live. The model that reads the definitions and chooses the tool calls belongs to the client you pick (Claude Code, Codex, Copilot in VS Code, OpenCode). Playwright supplies everything around the model: the instructions, the tool list and the playwright run-test-mcp-server process that executes the tools.

For this page, the installation and the server are real, and the model loop was driven by the assistant that wrote the page, not by a Claude Code subagent. A headless claude -p session could not authenticate in the sandbox where the page was written, so no Claude Code session produced the transcripts. Instead, a small stdio MCP client started the real npx playwright run-test-mcp-server and the authoring assistant issued the test planner, generator and healer tool calls one at a time, reading each result before choosing the next one. That is the observe-act loop an agent runs, with the same tools and the same results, but it is not a recording of a Claude Code subagent following the generated .claude/agents/*.md files, and the choices (which scenarios to plan, which assertions to write) are the authoring assistant’s. Treat the transcripts as evidence of what the tools do and return, and treat the quality of the test planning and generation as one sample, not a benchmark.

Where this page says an agent "decides", read it as "an LLM following the agent definition decides".

The three agents

Agent Job Reads Writes

planner

Explores the running application and designs scenarios, including edge cases and validation errors.

A seed test (to reach a ready page), optionally a request such as "plan guest checkout" or a product requirements document.

A Markdown plan, for example specs/catalogue-search.md, through planner_save_plan.

generator

Turns one scenario of the test plan into a test. It performs each step in a live browser first, so locators and assertions are checked against the real page, and then writes the file.

The plan, the seed test and a log of the live steps (generator_read_log).

One .spec.ts file per scenario, through generator_write_test.

healer

Runs the suite, debugs failures, patches the test, reruns, and repeats. If it is confident the test is right and the application is broken, it marks the test test.fixme() with a comment.

Failing test output, the page snapshot at the failure and the console and network logs.

Edits to existing test files.

The official page describes them as agents that can run independently, in sequence or chained in an agentic loop. In sequence they produce coverage for an application: plan, then generate each scenario, then run and heal.

flowchart LR S["Seed test: tests/seed.spec.ts"] --> P["Planner: explore the app"] P --> PL["specs: Markdown plan"] PL --> G["Generator: perform steps live, write one test per scenario"] G --> T["tests: generated spec files"] T --> R{"Suite passes?"} R -- "Yes" --> RV["Human review and commit"] R -- "No" --> H["Healer: test_debug, snapshot, patch, rerun"] H --> R H -. "still failing, feature broken" .-> FX["test.fixme plus comment"] FX --> RV

Two properties of the loop matter for everything that follows. First, the loop closes on a passing test, not on a correct test: the healer’s stop condition is "the run is green", so a test that was patched to stop failing needs a human to confirm that it still asserts the right thing. Second, the generator can only assert what it observed, so the quality of the assertions depends on the test plan’s "expect" lines and on the verification tools the agent chooses. Both points come up in the review section.

Installing the agents with init-agents

init-agents writes the agent definitions for one client into the repository. It must run inside a project that has @playwright/test and a playwright.config.ts, because it picks a project from the configuration to use for the seed test.

npx playwright init-agents --help

Source: Playwright Test Agents

Usage: npx playwright init-agents [options]

Initialize repository agents

Options:
  --loop <loop>        Agentic loop provider (choices: "claude", "codex",
                       "copilot", "opencode", "vscode", "vscode-legacy")
  -c, --config <file>  Configuration file to find a project to use for seed test
  --project <project>  Project to use for seed test
  --prompts            Whether to include prompts in the agent initialization
  -h, --help           display help for command

Output of the command above, run with Playwright 1.64.1. The official page and the 1.56 release notes list only vscode, claude, opencode and (on the page) codex; 1.64.1 accepts six values.

--loop value Client Files it writes (real run)

claude

Claude Code

.claude/agents/playwright-test-{planner,generator,healer}.md and .mcp.json

codex

OpenAI Codex

.codex/agents/playwright_test_{planner,generator,healer}.toml; no MCP file is written

copilot

GitHub Copilot (including the cloud agent)

.github/agents/playwright-test-*.agent.md, .vscode/mcp.json and .github/workflows/copilot-setup-steps.yml, plus a printed MCP snippet to paste into the repository’s Copilot settings

opencode

OpenCode

.opencode/prompts/playwright-test-*.md and opencode.json (agents with mode: subagent and a per-agent tool allow-list)

vscode

VS Code agent mode (custom agents)

The same .github/agents/*.agent.md, .vscode/mcp.json and copilot-setup-steps.yml as copilot

vscode-legacy

VS Code chat modes (older format)

.github/chatmodes/*.chatmode.md (the file names start with a theatre-masks emoji) and .vscode/mcp.json

Every loop also writes specs/README.md (the directory for plans) and tests/seed.spec.ts (the seed test). The copilot and vscode loops produced identical files in the real run. The official page recommends VS Code 1.105 or later for the vscode loop.

The command for this page’s project:

npx playwright init-agents --loop=claude

Source: Playwright Test Agents

 🎭 Using project "chromium" as a primary project
 📝 specs/README.md - directory for test plans
 🌱 tests/seed.spec.ts - default environment seed file
 🤖 .claude/agents/playwright-test-generator.md - agent definition
 🤖 .claude/agents/playwright-test-healer.md - agent definition
 🤖 .claude/agents/playwright-test-planner.md - agent definition
 🔧 .mcp.json - mcp configuration
 ✅ Done.

Source: Playwright Test Agents

The --prompts option

--prompts additionally writes reusable prompt templates. With --loop=claude it created four files in .claude/prompts/: playwright-test-plan.md, playwright-test-generate.md, playwright-test-heal.md and playwright-test-coverage.md. The first three are one-line requests aimed at a single agent (for example "Run all my tests and fix the failing ones." for the healer). The coverage template chains the whole loop: it asks the test planner subagent for a plan, then calls the generator subagent for each test case in the test plan, one after another and not in parallel, then calls the healer subagent with "Run all tests and fix the failing ones one after another."

npx playwright init-agents --loop=claude --prompts

Source: Playwright Test Agents

Choosing the project

The seed test belongs to one Playwright project, because the project decides the browser, baseURL and any dependencies (for example a setup project that signs in). init-agents picked chromium automatically in the real run. With several projects, pass --project <name>; with a configuration outside the default location, pass -c <file>. Choose the project that has the fixtures your tests need, not the one that is merely first in the file.

The seed test

The seed test is the starting state of every agent session. The planner and the generator run it first (through planner_setup_page and generator_setup_page), and the browser stays open at the end of the test, so the agent begins exactly where the seed stopped, with all the fixtures, hooks, global setup and storageState that the test would have had. The official page also says the seed is a model for the generated tests: they copy its imports and structure.

init-agents writes a placeholder:

import { test, expect } from '@playwright/test';

test.describe('Test group', () => {
  test('seed', async ({ page }) => {
    // generate code here.
  });
});

Source: Playwright Test Agents

Replace the body with whatever gets a page to the interesting starting point. The Bookshelf seed used in the run below navigates to the catalogue and checks the heading, so that the server is known to be up before an agent starts:

import { test, expect } from '@playwright/test';

test.describe('Bookshelf seed', () => {
  test('seed', async ({ page }) => {
    await page.goto('/books');
    await expect(page.getByRole('heading', { name: 'Catalogue' })).toBeVisible();
  });
});

Source: Playwright Test Agents

If the application needs a signed-in user, make the seed import your fixtures (for example import { test, expect } from './fixtures') so that the agents inherit the authenticated context. The official example does exactly that. Never put a real password in the seed; read it from an environment variable such as BOOKSHELF_READER_PASSWORD, as in Playwright: Authentication & Isolation.

The agent definition files

Each definition has three parts: front matter that names the agent and lists the tools it may call, a role description, and a procedure. They are plain Markdown, so they can be read, reviewed and changed like any other file. The generated planner for Claude Code starts like this (the tool list is shortened here; the real file grants 21 MCP tools):

---
name: playwright-test-planner
description: Use this agent when you need to create comprehensive test plan for a web application or website
tools: Glob, Grep, Read, LS, mcp__playwright-test__browser_click, mcp__playwright-test__browser_navigate, mcp__playwright-test__browser_snapshot, mcp__playwright-test__browser_type, mcp__playwright-test__planner_setup_page, mcp__playwright-test__planner_save_plan
model: sonnet
color: green
---

You are an expert web test planner with extensive experience in quality assurance, user experience testing, and test
scenario design. ...

You will:

1. **Navigate and Explore**
   - Invoke the `planner_setup_page` tool once to set up page before using any other tools
   - Explore the browser snapshot
   - Do not take screenshots unless absolutely necessary
   ...
5. **Create Documentation**

   Submit your test plan using `planner_save_plan` tool.

Source: Playwright Test Agents

The three Claude Code files differ in their tool allow-lists, which is where most of the design lives:

Agent Colour / model Tools beyond file reading

planner

green, sonnet

planner_setup_page, planner_save_plan, and browser tools to explore (click, type, navigate, snapshot, screenshot, console and network reads, browser_run_code_unsafe, browser_evaluate)

generator

blue, sonnet

generator_setup_page, generator_read_log, generator_write_test, browser action tools, and the assertion tools browser_verify_element_visible, browser_verify_list_visible, browser_verify_text_visible, browser_verify_value

healer

red, sonnet

Edit, MultiEdit, Write (the only agent that may edit files), test_list, test_run, test_debug, browser_snapshot, browser_generate_locator, console and network reads

The generator’s procedure forces the order "perform each step in a real browser, read the generated log, then immediately write the test", and its instructions require a single test per file, a describe named after the test plan’s suite, a test title equal to the scenario name, and a comment with the step text before each step. The healer’s procedure is a loop (run, debug, investigate, fix, restart, iterate) with an explicit escape: "If the error persists and you have high level of confidence that the test is correct, mark this test as test.fixme() so that it is skipped during the execution", with a comment before the failing step. It also forbids waiting for networkidle, and tells the agent not to ask the user questions.

The model: sonnet line is a default written by Playwright. Change it in the file if you want another model, or remove it so that the subagent inherits the session’s model.

The MCP server configuration

Claude Code reads .mcp.json at the project root. init-agents registers one server, named playwright-test:

{
  "mcpServers": {
    "playwright-test": {
      "command": "npx",
      "args": [
        "playwright",
        "run-test-mcp-server"
      ]
    }
  }
}

Source: Playwright Test Agents

This is a different server from @playwright/mcp, which Playwright MCP & CLI covers. run-test-mcp-server ships inside the playwright package, so it always matches the installed test runner version, and it adds test-runner tools to the same browser tools. npx playwright run-test-mcp-server --help in 1.64.1 lists --headless (the browser is headed by default), -c, --config <file>, --host, and --port for the SSE transport. The server reported itself as Playwright Test Runner version 1.64.1 during the handshake.

Listing its tools returns 92 entries: the three planner tools (planner_setup_page, planner_save_plan, planner_submit_plan), the three generator tools, the three runner tools (test_list, test_run, test_debug) and the rest browser_* tools, which include the capability-gated groups (cookies, storage, routing, tracing, video, WebMCP) described in the Playwright MCP page. The agent files are what narrow this down: the allow-list in each file means that, for example, the test planner can never edit a file and the healer cannot save a new test.

One detail found while driving the tools: every browser_* tool in this server requires an intent string, a short description of why the call is made. The generator uses the test plan step text as the intent, and the log it writes later uses those intents as step titles. A call without it is rejected with a schema error:

>>> browser_type {"element":"Search books","target":"e13","text":"zzz","submit":true}
[
  {
    "expected": "string",
    "code": "invalid_type",
    "path": [
      "intent"
    ],
    "message": "Invalid input: expected string, received undefined"
  }
]

Source: Playwright Test Agents

Other clients

The other loops write the same three roles in each client’s format. The codex loop writes TOML files with a sandbox_mode = "workspace-write" line and the instructions in developer_instructions; it writes no MCP file, so register the server in Codex yourself (npx playwright run-test-mcp-server as a stdio command). The opencode loop writes opencode.json containing the MCP server ("type": "local"), disables the generic playwright* tools globally, and lists per-agent tool maps. The vscode and copilot loops write .vscode/mcp.json with "type": "stdio" and print a snippet (with "tools": ["*"]) to paste into GitHub, Settings, Copilot, Coding agent, MCP configuration. They also write .github/workflows/copilot-setup-steps.yml, which installs dependencies and browsers before the Copilot cloud agent starts; it was generated with older action majors than the ones used elsewhere in this section, so compare it with Running in CI/CD before committing it. Copilot, Codex, OpenCode and VS Code sessions were not run here; their files were generated and read, nothing more.

The planner

The planner is the exploratory step. Its first call must be planner_setup_page, which runs the seed test and returns the page snapshot:

>>> planner_setup_page {"project":"chromium","seedFile":"tests/seed.spec.ts"}

Running 1 test using 1 worker
### Paused at end of test. ready for interaction

### Page state
- Page URL: http://localhost:3971/books
- Page Title: Catalogue - Bookshelf
- Page Snapshot:
```yaml
- generic [active] [ref=e1]:
  ...
  - main [ref=e9]:
    - heading "Catalogue" [level=1] [ref=e10]
    - search [ref=e11]:
      - generic [ref=e12]: Search books
      - searchbox "Search books" [ref=e13]
      - button "Search" [ref=e14] [cursor=pointer]
    - paragraph [ref=e15]: 24 books found
    - list [ref=e16]:
    ...
```

Source: Playwright Test Agents

From there the test planner uses ordinary browser tools to look around, and finally saves the test plan. The call below is the real argument; planner_save_plan renders it as Markdown. The plan covers two scenarios; only the second was carried through generation in the run:

>>> planner_save_plan {"name":"Bookshelf catalogue and search","fileName":"specs/catalogue-search.md","overview":"Bookshelf is a small book catalogue with search, pagination and book detail pages. This plan covers the anonymous search and pagination flows.","suites":[{"name":"Catalogue search","seedFile":"tests/seed.spec.ts","tests":[{"name":"Search narrows the catalogue","file":"tests/catalogue-search/search-narrows-results.spec.ts","steps":[{"perform":"Type \"compiler\" in the \"Search books\" box and press Enter","expect":["The URL contains q=compiler","The page shows \"1 book found\"","The list contains \"The Quiet Compiler\""]}]},{"name":"Search with no matches","file":"tests/catalogue-search/search-no-matches.spec.ts","steps":[{"perform":"Search for \"zzzz\"","expect":["The page shows \"0 books found\"","The book list is empty"]}]}]}]}
Test plan saved to specs/catalogue-search.md

Source: Playwright Test Agents

The resulting specs/catalogue-search.md is the artifact that people review and edit:

# Bookshelf catalogue and search

## Application Overview

Bookshelf is a small book catalogue with search, pagination and book detail pages. This plan covers the anonymous search and pagination flows.

## Test Scenarios

### 1. Catalogue search

**Seed:** `tests/seed.spec.ts`

#### 1.1. Search narrows the catalogue

**File:** `tests/catalogue-search/search-narrows-results.spec.ts`

**Steps:**
  1. Type "compiler" in the "Search books" box and press Enter
    - expect: The URL contains q=compiler
    - expect: The page shows "1 book found"
    - expect: The list contains "The Quiet Compiler"

#### 1.2. Search with no matches

**File:** `tests/catalogue-search/search-no-matches.spec.ts`

**Steps:**
  1. Search for "zzzz"
    - expect: The page shows "0 books found"
    - expect: The book list is empty

Source: Playwright Test Agents

The format is deliberate: scenario numbers (1.1), a Seed line, a File line that fixes where the test will be written, Steps and expect lines. The generator is told to copy the suite name and test name verbatim, so editing a title in the test plan and regenerating changes the test title. A plan is cheap to review in a pull request, and it is the right place to correct an agent: if an expectation is vague, fix the test plan before generating. planner_submit_plan also exists in the server’s tool list, but the generated planner definition only grants planner_save_plan.

An important property of this plan is what it does not contain. The planner explored the anonymous catalogue only; it did not sign in, post a review or exercise the admin pages, and a plan "assumes blank or fresh state" according to its instructions. Coverage depends on the request: ask for the flows you care about, and add a PRD or a list of acceptance criteria if there is one.

The generator

The generator takes one scenario. In Claude Code you would invoke the subagent with a block like the one the generated file documents (suite name, test name, file, seed file, body); the --prompts coverage template shows the exact shape. The tool calls from the run, for the test plan item "Search with no matches", were:

>>> generator_setup_page {"project":"chromium","seedFile":"tests/seed.spec.ts","plan":"Search for \"zzzz\" in the Bookshelf catalogue. Expect \"0 books found\" and an empty list."}
... (same page snapshot as the test planner)

>>> browser_type {"element":"Search books","target":"e13","text":"zzzz","submit":true,"intent":"Search for zzzz"}
### Ran Playwright code
```js
await page.getByTestId('search-input').fill('zzzz');
await page.getByTestId('search-input').press('Enter');
```
### Page
- Page URL: http://localhost:3971/books?q=zzzz

>>> browser_verify_text_visible {"text":"0 books found","intent":"Expect zero results"}
### Result
Done
### Ran Playwright code
```js
await expect(page.getByTestId('result-count')).toBeVisible();
```

Source: Playwright Test Agents

Two features of these results are what make the generator more reliable than an agent that guesses a test from the HTML. The browser tool returns the Playwright code that is equivalent to the action it just performed, and the locator in that code (getByTestId('search-input')) was resolved against the live page, using the project’s testIdAttribute. The verification tool returns the equivalent expect call, and a failed check returns an error instead of "Done".

The agent then calls generator_read_log, which returns the test plan, the seed file source, the recorded steps and a "Best practices" list ("NEVER! use page.waitForTimeout()", "NEVER! use page.waitForLoadState()", "NEVER! use page.evaluate()", and similar), and immediately writes the file with generator_write_test. The test below is what was written, with one change to the assertion explained in the next subsection:

// spec: specs/catalogue-search.md
// seed: tests/seed.spec.ts
import { test, expect } from '@playwright/test';

test.describe('Catalogue search', () => {
  test('Search with no matches', async ({ page }) => {
    await page.goto('/books');
    await page.getByTestId('search-input').fill('zzzz');
    await page.getByTestId('search-input').press('Enter');
    await expect(page.getByTestId('result-count')).toHaveText('0 books found');
    await expect(page.getByTestId('book-title')).toHaveCount(0);
  });
});

Source: Playwright Test Agents

The two header comments (spec: and seed:) are the link back to the test plan, and they are what a reviewer follows from a test to its scenario. The first test run passed:

>>> test_run {"locations":["tests/catalogue-search"],"projects":["chromium"]}

Running 1 test using 1 worker
  ✓  1 [id=7acad78fd264a0fdd2b0-ba94b9bdd70c1aa55527] [project=chromium] › tests/catalogue-search/search-no-matches.spec.ts:6:7 › Catalogue search › Search with no matches (149ms)
  1 passed (612ms)

Source: Playwright Test Agents

A weak assertion produced by the verification tool

The Ran Playwright code line for browser_verify_text_visible above is expect(page.getByTestId('result-count')).toBeVisible(). The check the tool performed live was on the text "0 books found", but the code it emitted into the log, and which the generator would copy into the test, only asserts that the element is visible. The element is visible when the catalogue shows "24 books found" too. The test written for this page replaces it with toHaveText('0 books found'), which is the expectation in the test plan.

To measure the difference, two tests were run against a catalogue page where the search step was skipped on purpose (the page still shows all 24 books):

import { test, expect } from '@playwright/test';

// The search is skipped on purpose: the page still shows all 24 books.
test('weak generated assertion passes on an unfiltered page', async ({ page }) => {
  await page.goto('/books');
  await expect(page.getByTestId('result-count')).toBeVisible();
});

test('strengthened assertion catches the same defect', async ({ page }) => {
  await page.goto('/books');
  await expect(page.getByTestId('result-count')).toHaveText('0 books found', { timeout: 2000 });
});

Source: Playwright: LocatorAssertions

  ✓  1 [chromium] › tests/weak-vs-strong.spec.ts:4:5 › weak generated assertion passes on an unfiltered page (63ms)
  ✘  2 [chromium] › tests/weak-vs-strong.spec.ts:9:5 › strengthened assertion catches the same defect (2.0s)

    Error: expect(locator).toHaveText(expected) failed

    Locator:  getByTestId('result-count')
    Expected: "0 books found"
    Received: "24 books found"
    Timeout:  2000ms

Source: Playwright: LocatorAssertions

This is the single most important review point for generated tests: a green generated test can assert much less than the test plan promised. The remedy is mechanical. After generation, compare each expect: line of the test plan with the assertions in the test, and strengthen the weak ones by hand (toHaveText, toHaveCount, toHaveURL, toHaveValue). The Playwright: Assertions page lists the matchers, and Test Design & Page Objects explains why the assertions should describe user-visible outcomes.

The healer

The healer is run when tests fail. Its input is usually just "run all my tests and fix the failing ones". It calls test_run, and for each failure calls test_debug, which reruns the test under the debugger and pauses at the failing step with the page still open.

To produce a realistic failure, the application drift was simulated by changing the test, not the application: the locator in the passing test was edited from search-input to search-box, as if a developer had renamed the data-testid and the test had not followed. The run failed with Playwright’s usual timeout:

>>> test_run {"locations":["tests/catalogue-search"],"projects":["chromium"]}

Running 1 test using 1 worker
  ✘  1 [id=7acad78fd264a0fdd2b0-ba94b9bdd70c1aa55527] [project=chromium] › tests/catalogue-search/search-no-matches.spec.ts:6:7 › Catalogue search › Search with no matches (30.0s)
  1) ... › Catalogue search › Search with no matches

    Test timeout of 30000ms exceeded.

    Error: locator.fill: Test timeout of 30000ms exceeded.
    Call log:
      - waiting for getByTestId('search-box')

       6 |   test('Search with no matches', async ({ page }) => {
       7 |     await page.goto('/books');
    >  8 |     await page.getByTestId('search-box').fill('zzzz');
         |                                          ^
       9 |     await page.getByTestId('search-box').press('Enter');
    ...
    Error Context: test-results/catalogue-search-search-no-733bf-arch-Search-with-no-matches-chromium/error-context.md

  1 failed

Source: Playwright Test Agents

test_debug then returned a paused session with the error, the page URL and a fresh accessibility snapshot taken at the moment of failure, and the instruction "Try recovering from the error prior to continuing". The snapshot contains searchbox "Search books" [ref=e13], so the element exists; only its test id changed. Debug runs use a shorter action timeout (5000 ms in the log) than the 30 s of the normal run, which makes a stale locator fail quickly in the debug session. The healer then asks the browser for a locator for that element:

>>> test_debug {"test":{"id":"7acad78fd264a0fdd2b0-ba94b9bdd70c1aa55527","title":"Search with no matches"}}

Running 1 test using 1 worker
### Paused on error:
TimeoutError: locator.fill: Timeout 5000ms exceeded.
Call log:
  - waiting for getByTestId('search-box')

### Page state
- Page URL: http://localhost:3971/books
- Page Title: Catalogue - Bookshelf
- Page Snapshot:
  ...
    - search [ref=e11]:
      - generic [ref=e12]: Search books
      - searchbox "Search books" [ref=e13]
  ...
### Task
Try recovering from the error prior to continuing

>>> browser_generate_locator {"element":"Search books box","target":"e13"}
### Result
getByTestId('search-input')

Source: Playwright Test Agents

The healer’s edit is then an ordinary file edit (search-box back to search-input in two places) and a final test_run, which passed:

>>> test_run {"locations":["tests/catalogue-search"],"projects":["chromium"]}

Running 1 test using 1 worker
  ✓  1 [id=7acad78fd264a0fdd2b0-ba94b9bdd70c1aa55527] [project=chromium] › tests/catalogue-search/search-no-matches.spec.ts:6:7 › Catalogue search › Search with no matches (84ms)
  1 passed (545ms)

Source: Playwright Test Agents

The file edit was made by the authoring assistant here, because the Edit tool belongs to the client and not to the MCP server; in a Claude Code session the healer subagent has Edit, MultiEdit and Write in its allow-list and makes that change itself.

This case is the easy one: the element moved, the behaviour did not. The healer’s guardrails exist for the other cases. When the application really is broken, patching the test to pass would hide a bug, so the definition tells the healer to skip the test with test.fixme() and a comment instead of rewriting the expectation. In practice a model does not always draw that line correctly. Review every healer diff with this question: did the locator change, or did the expected behaviour change? A diff that touches an expect value deserves the same scrutiny as a change to production code.

Running the agents in a client

With Claude Code, the generated .claude/agents/*.md files become subagents and .mcp.json registers the server (Claude Code asks for approval of project-scoped MCP servers the first time). You then ask for the work in natural language, naming the subagent, or paste the contents of a --prompts template. Subagent usage is covered in the Claude Code documentation, and Agent Mode and Headless & CI Automation explain the general model. Typical requests:

Use the playwright-test-planner subagent to create a plan for searching the
Bookshelf catalogue. Seed: tests/seed.spec.ts. Plan: specs/catalogue-search.md.

Use the playwright-test-generator subagent for scenario 1.2 of
specs/catalogue-search.md.

Use the playwright-test-healer subagent to run all tests and fix the failing ones.

Source: Playwright Test Agents

These prompts were not run in a Claude Code session, for the reason given at the start of the page. A non-interactive form such as claude -p with --mcp-config .mcp.json is documented in the Claude Code documentation and was not executed either.

Running agents in CI is a different decision from running the generated tests in CI. The generated tests are plain Playwright tests and belong in the normal pipeline (Running in CI/CD). The agents themselves have the power to edit files and to run arbitrary browser code (browser_evaluate, browser_run_code_unsafe), so run them where a person can review the resulting diff, on a branch, with the browser pointed at a test environment and not at production. The surface they read (page text, console output, network responses) can contain instructions written by someone else; Securing Browser Agents and Prompt Injection cover the risk, and Human in the Loop covers approval policies.

Regenerating after upgrades

The agent files are tied to the Playwright version because they list tool names and instructions that belong to the run-test-mcp-server of that version. The official page says to regenerate them whenever Playwright is updated so that they pick up new tools and instructions. The 1.56 release notes say the command generates "the latest agent definitions". Concretely, after npm install @playwright/test@latest run the same init-agents command again.

To see what a rerun does to files you may have edited, the command was run a second time after appending a line to the generated playwright-test-generator.md and to tests/seed.spec.ts. The result:

$ echo "# local edit: always use getByTestId" >> .claude/agents/playwright-test-generator.md
$ echo "// my seed edit" >> tests/seed.spec.ts
$ npx playwright init-agents --loop=claude
 ...
 ✅ Done.
$ diff -r .claude ../claude-before
$ tail -1 tests/seed.spec.ts
// my seed edit

Source: Playwright Test Agents

The rerun restored the agent file (no difference from the pristine copy: the local edit was overwritten) and left the seed test alone. So init-agents treats the agent definitions as generated output and the seed and plans as yours. Practical consequences:

  • Commit the agent files, so git diff after a regeneration shows exactly what the new version changed: tool names, new instructions, different front matter. Review that diff as you would a dependency upgrade.

  • Do not customise the generated files in place. Put team-specific rules in a place the client reads separately (for Claude Code, the project’s CLAUDE.md; for others, their instruction files), or keep a small post-processing step that reapplies your edits after each regeneration.

  • A seed that you changed is safe, but compare it with a fresh template when upgrading across several versions; the official examples evolve.

  • Pin the Playwright version in package.json and upgrade deliberately. npx playwright --version and the release notes tell you whether the agent tool list changed. Between 1.56 and 1.64 the --loop choices grew from three to six values and the server’s tool list gained capability-gated groups.

Copy prompt in the HTML report, UI Mode and the trace viewer

Playwright 1.51 added a "Copy prompt" button to errors in the HTML report, the trace viewer and UI Mode. It copies a pre-filled prompt for any LLM chat that contains the error message and useful context for fixing the failure. It is not part of Test Agents, which need a configured client, and it is the lightest-weight way to use an LLM with a failing test: you decide when to send the prompt and where to paste it. The 1.56 release notes added an option to turn the button off.

The HTML report was run to see the real content. A failing test was produced, the report generated with the html reporter, and the report was opened in headless Chromium, which found one "Copy prompt" button on the failed test. Its clipboard text, trimmed:

# Instructions

- Following Playwright test failed.
- Explain why, be concise, respect Playwright best practices.
- Provide a snippet of code with the fix, if possible.

# Test info

- Name: weak-vs-strong.spec.ts >> strengthened assertion catches the same defect
- Location: tests/weak-vs-strong.spec.ts:9:5

# Error details

Error: expect(locator).toHaveText(expected) failed

Locator:  getByTestId('result-count')
Expected: "0 books found"
Received: "24 books found"
Timeout:  2000ms
...

```yaml
- paragraph: 24 books found
```

# Test source

  9  | test('strengthened assertion catches the same defect', async ({ page }) => {
  10 |   await page.goto('/books');
> 11 |   await expect(page.getByTestId('result-count')).toHaveText('0 books found', { timeout: 2000 });

Source: Playwright: Release notes

The prompt contains the test name and location, the error with call log, the accessibility snapshot of the page when the test failed (in the yaml block) and the numbered test source around the failing line. That is enough for a chat model to suggest a fix without access to your repository. Because it contains source code, error messages and page text, check that none of it is confidential before pasting it into an external service.

To hide the button, for example on a report published to a wide audience, set the reporter option or the environment variable:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [['html', { noCopyPrompt: true }]],
});

Source: Playwright: Reporters

PLAYWRIGHT_HTML_NO_COPY_PROMPT=1 npx playwright test

Source: Playwright: Reporters

With PLAYWRIGHT_HTML_NO_COPY_PROMPT=1 the same report was opened again and contained no "Copy prompt" button. The option is documented as accepting true, 1, false and 0 and defaults to false.

The button in UI Mode and in the trace viewer (opened with npx playwright show-trace, or from the report) was not run, because those are interactive windows and the machine that produced these examples had no display to drive. The release notes describe it as the same button on errors in both places. See Playwright: Debugging, Tracing & Codegen for UI Mode, the trace viewer and the reporter options.

Reviewing generated tests

Generated tests are a first draft written by a system that rewards passing. Review them the way you review code from a new colleague who has never seen the codebase.

Check What to look for

Assertions match the test plan

Every expect: line should have an assertion that would fail if the behaviour were wrong. toBeVisible() on a container, or on an element that is always rendered, is the weak pattern shown above. Prefer toHaveText, toHaveCount, toHaveURL, toHaveValue.

Locators

Roles, labels and test ids first. Reject chains of .locator('div > div:nth-child(3)'), text locators on copy that marketing will change, and .first() used to silence strict-mode errors. See Playwright: Locators, Actions & Auto-waiting.

Waits

There should be none. The generator instructions forbid waitForTimeout, waitForLoadState and waitForNavigation, but a healer patch for a timing failure can reintroduce a sleep. See Flaky Tests & Reliability.

Independence

A test should create its own data (through the API where possible) and not depend on the previous test. The planner assumes a fresh state; the application may not provide one. See Test Design & Page Objects.

Secrets and data

No credentials in files, no real personal data. Read secrets from the environment.

Duplication

Ten generated tests that each repeat the same four sign-in steps should use a fixture or a page object, or the seed. Agents write what the test plan says; they do not refactor across files unless asked.

Healer diffs

Check whether a patch changes a locator or an expected value, and whether a test.fixme() hides a regression that should be filed.

Stability

Run the new tests repeatedly (--repeat-each=10) and across projects, then keep them in the normal suite with retries and traces configured as usual.

Make the agent files part of this review too. A change to a generated definition that, for example, adds browser_run_code_unsafe to the generator’s tools is a change in what an unattended session can do.

Other ways to get the same outcome

Test Agents are specific to Playwright. For Cypress, the AI features (cy.prompt, Studio AI, Cloud MCP) and for Selenium (the AI Agents guide) are described in AI Features in Cypress & Selenium. Without any agent definitions, a coding assistant with Playwright MCP can explore an application and write tests on request, which Playwright MCP & CLI demonstrates, and Script-less Testing with AI Agents discusses when to keep an agent session as an exploratory tool and when to turn it into a regression test. The generic test-with-AI workflow is in Testing & Debugging with AI.

What changed recently

  • Playwright 1.51: the Copy prompt button on errors in the HTML report, the trace viewer and UI Mode.

  • Playwright 1.56: Test Agents, with npx playwright init-agents --loop=vscode|claude|opencode, and an html reporter option to disable the Copy prompt button (noCopyPrompt).

  • Playwright 1.59: agent-oriented command-line debugging (npx playwright test --debug=cli, attaching with playwright-cli) and trace analysis (npx playwright trace), which a healer-style loop can use instead of MCP.

  • Playwright 1.64.1 (the version used here): init-agents accepts claude, codex, copilot, opencode, vscode and vscode-legacy, and has --project, --config and --prompts. The official Test Agents page at the check date still lists the four older commands in its table.

  • The official Test Agents page shows a Claude Code example with --loop=vscode; the working command for Claude Code is --loop=claude, as used here.

See What Changed & Upgrading for the full list.

References