Script-less Testing with AI Agents

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

The reference material consulted while preparing these pages is listed in the section bibliography.

A script-less test has no test code. Instead of a script that says which locator to click and which assertion to make, the tester gives an AI agent a goal and acceptance criteria ("a signed-in reader can add a review; a review shorter than ten characters is rejected"), and the agent drives a real browser through a browser tool server, observes the page, decides the next action, and finally reports pass or fail together with evidence. The tools that give an agent a browser are described in How AI Agents Drive Browsers, Playwright MCP & CLI and Chrome DevTools for Agents; this page is about what to do with them when the product under test is a web page.

The page targets Playwright 1.64, Cypress 16.2 and Selenium 4.51 for the scripted side of every comparison, and was checked on 25 October 2026. The worked examples ran against the Bookshelf demo application described in Example Application & Test Layout, served on localhost:3971, with @playwright/mcp 0.0.83, @playwright/cli 0.1.22 and chrome-devtools-mcp 1.12.0, in headless Chromium on macOS.

One fact about how those runs were driven matters for reading the transcripts. A headless claude -p --mcp-config session could not authenticate on the machine that produced this page, so no Claude Code session generated the transcripts. A small stdio MCP client started the real servers, and an AI model issued the tool calls one at a time, reading each result before choosing the next call. That is the same observe-act loop an agent runs, and the tool calls and outputs shown below are real and unedited apart from the trimming and sanitising described in each example. The model that chose the calls was not Claude Code, and nothing on this page claims a Claude Code run. Examples that need a Copilot plan, VS Code chat, a Cypress Cloud account or a connected Chrome extension were documented from the official pages and were not run; each says so.

The idea: goals instead of steps

A scripted test encodes a procedure; a script-less test encodes an intent. Compare the two for the same feature:

Scripted:     click #rating, select "4", type the text, click Submit,
              expect the POST to return 201 and the text to be visible.
Script-less:  A signed-in reader can add a review to a book. The review is
              saved, and bad input is rejected with a visible message.

The agent supplies the procedure at run time, from the accessibility snapshot of the page. Three consequences follow, and the rest of the page returns to them:

Cheap to write, expensive to run. Writing the intent takes a minute; every run spends model tokens and seconds on each step instead of milliseconds.
Adaptive and non-repeatable. The agent copes with a renamed button or a moved field, but two runs can take different paths and word the verdict differently.
Only as good as the evidence. A verdict is trustworthy when it comes with something a human or a script can re-check: a response status, a snapshot, a screenshot, a console line.

Script-less testing complements a scripted suite. It is strongest for new features that have no tests yet, for exploratory sessions and charters, for design and accessibility reviews, and for diagnosing a failure. It is weakest as the regression gate of a release; see Limits.

The loop

flowchart LR G["Goal and acceptance criteria"] --> E["Explore: snapshot the page"] E --> A["Act: click, type, navigate"] A --> O["Observe: new snapshot, network, console"] O --> J{"Judge: criterion met?"} J -- "not yet" --> E J -- "yes or no" --> V["Evidence: snapshots, screenshots, logs"] V --> R["Report pass or fail per criterion"]

The agent-loop theory (perception, reasoning, tool use, stop conditions) is in Agent Loop & Architectures, and good browser tools follow Tool Design. The MCP plumbing is in MCP Architecture and Connecting Hosts.

Setting up Claude Code with a browser server

The documented way to give Claude Code a browser is to register an MCP server. These commands come from the Claude Code and Playwright MCP documentation and were not executed here, for the authentication reason given in the introduction:

# Playwright MCP, headless, throw-away profile
claude mcp add playwright -- npx @playwright/mcp@latest --headless --isolated

# Playwright MCP with the extra verify and locator tools
claude mcp add playwright-testing -- npx @playwright/mcp@latest --headless --isolated --caps testing

# Chrome DevTools MCP, without usage statistics and CrUX lookups
claude mcp add chrome-devtools -- npx chrome-devtools-mcp@latest --headless --isolated --no-usage-statistics --no-performance-crux

Source: Claude Code: Connect Claude Code to tools via MCP

For a real, logged-in browser instead of an isolated one, Claude Code also has a Chrome integration (claude --chrome or /chrome) that drives the user’s own Chrome through an extension, with per-site permissions. No Chrome extension was connected to the session that produced this page, so that path is documented, not run; its trade-offs are covered in Agents That Control the Browser and its risks in Securing Browser Agents.

Source: Claude Code: Use Claude Code with Chrome

Example 1: verify the add-review feature from acceptance criteria

Tools: Claude Code (or any MCP host) with Playwright MCP. Ran: yes, against Bookshelf on localhost:3971, @playwright/mcp 0.0.83 with --headless --isolated.

The prompt gives the goal, the setup and three acceptance criteria. It does not name a single locator.

Open http://localhost:3971/signin and sign in as the reader (the password is in
BOOKSHELF_READER_PASSWORD). Then open the book "The Quiet Compiler".

Verify these acceptance criteria on the "Add a review" form and report PASS or
FAIL for each, with the evidence you used (network request, visible text, console):

1. A rating of 4 and the text "Clear and practical, with a great index." is saved
   and the new review appears in the list.
2. A review shorter than 10 characters is rejected with a visible message and
   no request is sent to the server.
3. If the server fails to save, the user sees an error, the optimistic entry
   disappears, and the failure is visible in the console.

Do not change the application. Stop after the three checks.

The agent signs in. The first two attempts at browser_fill_form and browser_click were rejected because the tool schema in 0.0.83 names the element argument target, not ref; the agent corrected the call from the validation error, which is the self-repair a script would need a human for:

>>> browser_fill_form {"fields":[{"name":"Username","type":"textbox","ref":"e12","value":"reader"}, ...]}
### Error
Invalid arguments for tool "browser_fill_form":
✖ Invalid input: expected string, received undefined
  → at fields[0].target

>>> browser_fill_form {"fields":[{"name":"Username","type":"textbox","target":"e12","value":"reader"},
                                 {"name":"Password","type":"textbox","target":"e13","value":"$BOOKSHELF_READER_PASSWORD"}]}
### Ran Playwright code
await page.getByTestId('username').fill('reader');
await page.getByTestId('password').fill('$BOOKSHELF_READER_PASSWORD');

>>> browser_click {"element":"Sign in button","target":"e14"}
### Ran Playwright code
await page.getByTestId('signin-submit').click();
### Page
- Page URL: http://localhost:3971/account

In the transcripts the literal demo password was replaced by the variable name; the page never prints a password value.

Criterion 1, the happy path. Note that the server tool echoes the Playwright code it ran, which is the key to Turning a session into a regression test:

>>> browser_navigate {"url":"http://localhost:3971/books/1"}
>>> browser_select_option {"element":"Rating","target":"f2e31","values":["4"]}
    await page.getByTestId('review-rating-input').selectOption('4');
>>> browser_type {"element":"Your review","target":"f2e32","text":"Clear and practical, with a great index."}
    await page.getByTestId('review-text-input').fill('Clear and practical, with a great index.');
>>> browser_click {"element":"Submit review","target":"f2e33"}
    await page.getByTestId('review-submit').click();
>>> browser_find {"text":"Clear and practical"}
        - listitem [ref=f2e35]:
          - strong [ref=f2e36]: reader
          - text: ★★★★
          - paragraph [ref=f2e37]: Clear and practical, with a great index.
>>> browser_network_requests {}
5. [POST] http://localhost:3971/api/reviews => [201] Created

Criterion 2, the negative path:

>>> browser_select_option {"element":"Rating","target":"f2e31","values":["3"]}
>>> browser_type {"element":"Your review","target":"f2e32","text":"Too short"}
>>> browser_click {"element":"Submit review","target":"f2e33"}
>>> browser_find {"text":"Too short"}
        - textbox "Your review" [ref=f2e32]: Too short
        - alert [ref=f2e38]: Write at least 10 characters
>>> browser_network_requests {}
5. [POST] http://localhost:3971/api/reviews => [201] Created

The request list still ends at request 5, the earlier valid review, so no second POST was sent. Criterion 3 uses the application’s built-in failure trigger (review text containing the word FAIL makes the server answer 500):

>>> browser_type {"element":"Your review","target":"f2e32","text":"This text will FAIL on save."}
>>> browser_click {"element":"Submit review","target":"f2e33"}
### Page
- Console: 1 errors, 0 warnings
>>> browser_find {"text":"FAIL"}
        - textbox "Your review" [ref=f2e32]: This text will FAIL on save.
        - alert [ref=f2e38]: Could not save the review
>>> browser_network_requests {}
5. [POST] http://localhost:3971/api/reviews => [201] Created
6. [POST] http://localhost:3971/api/reviews => [500] Internal Server Error
>>> browser_console_messages {}
[ERROR] Failed to load resource: the server responded with a status of 500 (Internal Server Error) @ http://localhost:3971/api/reviews:0

The report an agent writes from this evidence has one line per criterion: 1 PASS (POST 201 and the text in the list), 2 PASS (alert "Write at least 10 characters", no new POST), 3 PASS (POST 500, alert "Could not save the review", the text was not added to the list, one console error naming the 500). The quality of each line is the evidence behind it. The conclusions of criterion 3 rest on the absence of the entry in the snapshot, which is why the agent should re-read the page after the wait and not rely on the first snapshot after the click. The transcript shows the agent did that: it waited one second, then called browser_find again before concluding.

Source: Playwright MCP: Introduction

Example 2: compare the page with a design and fix the difference

Tools: Claude Code with Playwright MCP (a screenshot comparison prompt). Ran: partly. The measuring and screenshot steps ran; the fix step is described, not run, because the page already met the design.

The Claude Code best practices describe "verify UI changes visually" as a pattern: paste a design, ask Claude to implement it, take a screenshot of the result, compare, list differences and fix them. For a web page the design can also be written as measurable criteria, which gives the agent something exact to check instead of a judgement by eye:

Design target for http://localhost:3971/books:
- at 1280x800 the book grid shows at least 4 columns;
- at 375x812 (a phone) it shows 2 columns and the page has no horizontal scroll;
- the page title (h1) is 32px.

Take a screenshot at each size. Measure the grid with the browser, do not guess
from the picture. List every difference from the design. If there are
differences, fix them in the stylesheet and re-measure. Report what you changed.

The agent resized the window, took a screenshot and measured the computed styles:

>>> browser_resize {"width":1280,"height":800}
>>> browser_navigate {"url":"http://localhost:3971/books"}
>>> browser_take_screenshot {"type":"png","filename":"catalogue-desktop.png"}
>>> browser_evaluate {"function":"() => { const g=document.querySelector(\"[data-testid=book-list]\")||document.querySelector(\"main ul\"); const cs=getComputedStyle(g); return {tag:g.tagName, display:cs.display, columns:cs.gridTemplateColumns, cardCount:g.children.length, font:getComputedStyle(document.querySelector(\"h1\")).fontSize}; }"}
{
  "tag": "UL",
  "display": "grid",
  "columns": "185.594px 185.594px 185.609px 185.594px 185.609px",
  "cardCount": 6,
  "font": "32px"
}
>>> browser_resize {"width":375,"height":812}
>>> browser_evaluate {"function":"() => { ...; return {columns:cs.gridTemplateColumns, overflowX: document.documentElement.scrollWidth > innerWidth}; }"}
{
  "columns": "163.5px 163.5px",
  "overflowX": false
}
>>> browser_take_screenshot {"type":"png","filename":"catalogue-mobile.png"}

Result: five columns at 1280 pixels (criterion of at least four met), two columns of 163.5 pixels at 375 pixels, no horizontal overflow, title 32 pixels. All three design criteria pass, so there was nothing to fix. When a criterion fails, the documented continuation is the same loop with an edit step: the agent changes the stylesheet through the host’s file tools, reloads, and measures again until the numbers match. That edit step was not exercised here.

Two points make this example reliable. First, browser_evaluate returns numbers (computed grid columns, scrollWidth) that a person can check, which is better evidence than "the screenshot looks right". Second, the screenshots are an artifact for a human reviewer and not the verdict. A model can misjudge a screenshot; it cannot misread "overflowX": false. For pixel-exact regression checking use a visual-regression tool instead, see Visual Regression Testing.

Source: Claude Code: Best practices

Example 3: the Copilot cloud agent validates its own pull request

Tools: GitHub Copilot cloud agent with its built-in Playwright MCP browser. Ran: no. It needs a Copilot plan and a repository on GitHub, and was documented from the official pages.

The Copilot cloud agent works in its own GitHub Actions-based environment. According to the GitHub changelog it has "its own web browser" powered by the Playwright MCP server (enabled by default together with the GitHub MCP server), so it can reproduce a bug, interact with the application while it changes it, and share screenshots of what it did in the pull request. The feature was announced as a public preview for paid Copilot plans, with an administrator opt-in for Business and Enterprise. The agent’s network access is governed by a firewall, and MCP servers are configured per repository; see Cloud & Background Agents for the surrounding model.

An issue for it to work on, written in the script-less style, would be:

Title: Show "No reviews yet" on books without reviews

Acceptance criteria:
- Opening /books/20 (a book with no reviews) shows the text "No reviews yet"
  where the review list would be.
- Opening /books/1 still shows the existing reviews and no empty-state text.

Before opening the pull request, start the app (npm start), open both pages in
the browser, and attach a screenshot of each to the pull request description
together with a one-line result per criterion.

What the reviewer should then find in the pull request is the code change, the two screenshots and the per-criterion results. The screenshots are evidence of what the agent saw; they are produced by the same agent that wrote the change, which is a weakness covered under Limits. The practical rule is that the agent’s self-validation shortens review but does not replace it: the reviewer still checks the screenshots against the criteria, and the repository’s scripted checks still run on the pull request.

Source: GitHub changelog: Copilot coding agent now has its own web browser

Source: GitHub Docs: Extend Copilot cloud agent with MCP

Example 4: the VS Code browser-agent testing loop

Tools: VS Code Chat in Agent mode with the Integrated Browser tools. Ran: no. It needs VS Code with a Copilot chat subscription; documented from the official guide.

VS Code gives its chat agent browser tools: the agent can open pages in the integrated browser, read their structure, click, and check results. The tools are on by default and are controlled by the setting workbench.browser.enableChatTools; if they are missing, check that the setting is on, that Agent mode is selected and that the Integrated Browser tool is enabled in the tool picker (an organisation policy can also turn them off). Pages the agent opens use isolated, ephemeral sessions; a page the user shares with the agent through Share with Agent uses the user’s existing session, cookies and login included, and sharing can be revoked.

The official guide builds a loop: build, open in the browser, validate against criteria, fix, re-test. Applied to Bookshelf the same loop reads:

1. Build:    "Add a 'Clear search' button next to the catalogue search box."
2. Validate: "Start the app, open the catalogue in the browser and validate against
              these criteria, reporting each result:
              - typing 'compiler' and searching shows '1 book found';
              - 'Clear search' empties the box and shows '24 books found';
              - the button has an accessible name."
3. Debug:    "If a criterion fails, reproduce it, fix the code, and re-test."

The guide’s own example uses a calculator and a deliberately introduced division-by-zero bug to show the reproduce, fix and re-test cycle. The point for testers is step 2: the acceptance criteria are written once, in the prompt, and the agent turns them into browser actions.

Source: VS Code: Browser agent testing guide

Example 5: console errors and Web Vitals with Chrome DevTools MCP

Tools: Claude Code (or any host) with Chrome DevTools MCP. Ran: yes, chrome-devtools-mcp 1.12.0 with --headless --isolated --no-usage-statistics --no-performance-crux, against Bookshelf on localhost:3971.

Functional agents read the accessibility tree; DevTools-level checks read what the browser itself reports. The prompt:

Open http://localhost:3971/books/1. Report: (1) any console errors or warnings on
load, (2) failed network requests, (3) the Largest Contentful Paint and Cumulative
Layout Shift from a performance trace with a reload, and what dominates the LCP.
Then open /books/9999 and report the console. Finally run a mobile Lighthouse audit
of /books and list what fails.

The first calls, trimmed:

>>> navigate_page {"pageId":1,"type":"url","url":"http://localhost:3971/books/1"}
>>> list_console_messages {"pageId":1}
<no console messages found>
>>> list_network_requests {"pageId":1}
reqid=1 GET http://localhost:3971/books/1 [200]
reqid=2 GET http://localhost:3971/app.css [200]
reqid=3 GET http://localhost:3971/app.js [200]
reqid=4 GET http://localhost:3971/covers/1.svg [200]
reqid=5 GET http://localhost:3971/favicon.ico [204]
>>> performance_start_trace {"pageId":1,"reload":true,"autoStop":true}
Metrics (lab / observed):
  - LCP: 188 ms
    - TTFB: 0.8 ms
    - Load delay: 2 ms
    - Load duration: 1 ms
    - Render delay: 184 ms
  - CLS: 0.00
Metrics (field / real users): n/a - no data for this page in CrUX
>>> performance_analyze_insight {"pageId":1,"insightSetId":"NAVIGATION_0","insightName":"LCPBreakdown"}
The LCP element (IMG) is an image fetched from http://localhost:3971/covers/1.svg
- Element render delay: 184 ms (97.7% of total LCP time)
## Estimated savings: none

The agent can report: no console output, five requests all 2xx, LCP 188 ms (an SVG cover, 97.7 percent of it render delay, not network), CLS 0. The "field" line matters for honesty: a local page has no real-user data, so only lab numbers exist, and the agent should say so instead of quoting Web Vitals thresholds as if they were measured on users. Two of the tool calls had real API quirks worth knowing: performance_start_trace with a filePath outside the operating system temp directory was refused ("Access denied"), so the agent re-ran it without a path; and navigate_page needs a numeric pageId.

The negative page and the audit:

>>> navigate_page {"pageId":1,"type":"url","url":"http://localhost:3971/books/9999"}
1: Not found - Bookshelf
>>> list_console_messages {"pageId":1}
msgid=1 [error] Failed to load resource: the server responded with a status of 404 (Not Found) (0 args)
>>> emulate {"pageId":1,"colorScheme":"dark","viewport":"375x812x2,mobile,touch","networkConditions":"Slow 3G","cpuThrottlingRate":4}
Emulation configured successfully
>>> lighthouse_audit {"pageId":1,"mode":"navigation","device":"mobile"}
### Category scores
- Accessibility: 98 (accessibility)
- Best Practices: 100 (best-practices)
- SEO: 82 (seo)
- Agentic Browsing: 100 (agentic-browsing)
Passed: 49  Failed: 3
- heading-order [accessibility]: Heading elements are not in a sequentially-descending order
- meta-description [seo]: Document does not have a meta description
- crawlable-anchors [seo]: Links are not crawlable   (the disabled "Previous" pager link)

The 404 page produces one console error, as expected for a missing resource. The audit ran with the emulation left over from the previous call (Slow 3G, 4x CPU slowdown, dark scheme), which the output echoes; the agent has to remember to reset emulation, and the emulate tool’s networkConditions accepts only Offline, Slow 3G, Fast 3G, Slow 4G and Fast 4G, so "no throttling" cannot be requested by value (a call with "No emulation" failed validation). The three failed audits are real findings about Bookshelf, and a good report cites the selector (ul.grid > li.card > a > h3 for the heading order). For the accessibility meaning of such results see Accessibility Checks in E2E Tests.

Source: Chrome DevTools for agents

Source: Chrome DevTools MCP on GitHub

Example 6: the Playwright MCP verify tools (--caps testing)

Tools: Playwright MCP started with --caps testing. Ran: yes, @playwright/mcp 0.0.83 against Bookshelf on localhost:3971.

By default the server has no assertion tools; the agent reasons over snapshots. The testing capability adds five tools: browser_generate_locator, browser_verify_element_visible, browser_verify_text_visible, browser_verify_list_visible and browser_verify_value. Each verify tool checks the live page and, on success, returns Done together with the equivalent Playwright expect() code, which is how a script-less check turns into scripted code. The --help text of 0.0.83 lists only vision, pdf and devtools for --caps; testing (and network, storage, config) are accepted and add tools anyway, so the help output is incomplete and the tool list from the running server is the reference.

Open the catalogue. Verify, using the verify tools only, that the heading
"Catalogue" is visible, that the result count reads "24 books found", and that
the list contains "The Quiet Compiler" and "Rivers of Static". Then search for
"compiler" and verify "1 book found". Finally verify that "99 books found" is
NOT shown, and give me a locator for the Search button.

>>> browser_verify_element_visible {"role":"heading","accessibleName":"Catalogue"}
Done
    await expect(page.getByRole('heading', { name: 'Catalogue' })).toBeVisible();
>>> browser_verify_text_visible {"text":"24 books found"}
Done
    await expect(page.getByTestId('result-count')).toBeVisible();
>>> browser_verify_list_visible {"element":"book list","target":"e16","items":["The Quiet Compiler","Rivers of Static"]}
Done
    await expect(page.locator('body')).toMatchAriaSnapshot(`
      - list:
        - listitem: "The Quiet Compiler"
        - listitem: "Rivers of Static"
      `);
>>> browser_type {"element":"Search books","target":"e13","text":"compiler","submit":true}
>>> browser_verify_text_visible {"text":"1 book found"}
Done
    await expect(page.getByTestId('result-count')).toBeVisible();
>>> browser_verify_text_visible {"text":"99 books found"}
### Error
Text not found
>>> browser_generate_locator {"element":"Search button","target":"e14"}
### Error
Error: Ref e14 not found in the current page snapshot. Try capturing new snapshot.
>>> browser_snapshot {}
      - button "Search" [ref=f1e14] [cursor=pointer]
>>> browser_generate_locator {"element":"Search button","target":"f1e14"}
getByTestId('search-submit')

Two findings from this run are exactly the review points a script-less check needs:

browser_verify_text_visible produced a weak assertion. It returned Done for "24 books found" and again for "1 book found", but the generated code is expect(page.getByTestId('result-count')).toBeVisible(), which only asserts that the result-count element is visible and never mentions the text. The live check did see the text (the "99 books found" probe fails with Text not found), but the code that would be saved into a test would pass whatever the count says. When you copy a verify tool’s output into a spec, strengthen it by hand, for example await expect(page.getByTestId('result-count')).toHaveText('24 books found').
Refs go stale. After the search navigated, e14 no longer existed and the call failed; refs after a navigation carry a frame prefix (f1e14). The agent recovered by taking a fresh snapshot, which is the correct behaviour and the cheapest fix.

Source: Playwright MCP: Introduction

Source: Playwright MCP on GitHub

Example 7: cy.prompt, the middle ground

Tools: Cypress 16.2 with cy.prompt and Cypress Cloud. Ran: no. cy.prompt needs a Cypress Cloud account or a record key and sends the steps to a model hosted by Cypress, which was not available here; the code is documented from the command reference.

cy.prompt sits between a script and an open-ended agent: the test is committed in the repository as natural-language steps, but it runs inside the Cypress runner, and the generated commands are cached. The signature is cy.prompt(steps: string[], options?: { placeholders?: { [key: string]: string } }). A value that is sensitive or changes between runs goes in {{placeholders}}: placeholder values are "excluded from AI calls" and do not invalidate the cache. Only E2E testing is supported, only Chromium-based browsers (Chrome and Edge), the prompts are optimised for English, and canvas and iframe elements are not supported.

// cypress/e2e/add-review.cy.ts
describe('add review, described in natural language', () => {
  it('lets a reader add a valid review', () => {
    cy.env(['readerPassword']).then(({ readerPassword }) => {
      cy.prompt(
        [
          'visit /signin',
          'type "reader" in the username field',
          'type {{password}} in the password field',
          'click the sign in button',
          'visit /books/1',
          'select 4 in the rating field',
          'type "Clear and practical, with a great index." in the review field',
          'click the submit review button',
          'verify the review text appears in the list of reviews',
        ],
        { placeholders: { password: readerPassword } }
      );
    });
  });
});

Source: Cypress: cy.prompt

Behaviour, per the AI test generation guide:

The first run generates Cypress commands with the model; the code is "cached and shared across machines and environments, including CI". A re-run does not call the model again unless a step string changes (changing a placeholder value does not invalidate the cache) or the DOM changes in a way the cached selectors cannot handle.
Self-healing: if a cached selector fails, Cypress regenerates the code for that step on the next run. A step healed from the cache is labelled "Self-healed via Cache" and a step regenerated by the model "Self-healed via AI". A step that heals on every run is a sign to target something more stable.
Ejecting to code: the Command Log shows a Code button next to each cy.prompt that opens the generated commands with a comment per step. Copy them, or save them to the test file, which replaces the cy.prompt call with plain Cypress commands. An entire recorded run’s prompts can be downloaded from Cypress Cloud (Properties tab, Download report, as JSON, YAML or Markdown).

The status is inconsistent across Cypress pages at the check date: the Cloud AI features page lists cy.prompt as generally available, while the command’s own API page still says it was moved to beta in Cypress 15.13.0 (introduced in 15.4.0). Treat it as a Cypress Cloud feature whose labelling may change. More on Cypress’s AI features in AI Features in Cypress & Selenium and on the Cloud side in Cypress Cloud.

Source: Cypress: AI test generation

Prompt patterns

A script-less prompt works when it states what success looks like and what evidence to collect. The patterns below each fit the Bookshelf examples above.

Pattern What to say Bookshelf example

Pattern	What to say	Bookshelf example
Acceptance criteria	A numbered list of observable outcomes; ask for PASS or FAIL per item with evidence.	The three review criteria in example 1.
Persona	Who is using the app, with what permissions and goal; forbids steps that persona would not take.	"You are a first-time reader without an account. You may browse and search, but you cannot sign in. Find a Technology book under 10 dollars and say what stops you from reviewing it."
Exploratory charter	A time-boxed mission with a focus area, not steps: "Explore X with Y to discover Z".	"Explore the catalogue search with unusual input (empty, very long, markup, accents) to discover errors or layout breakage. Report each oddity with the exact input and a screenshot. Stop after 15 minutes of actions."
Negative paths	Ask for invalid input, missing permission and server failure explicitly; give the trigger if the app has one.	Short review text (rejected) and the `FAIL` trigger (500) in example 1.
Accessibility	Ask for named checks and the evidence for each: keyboard order, accessible names, contrast, landmarks.	"Using only the keyboard, sign in and add a review. Report any control that cannot be reached or has no accessible name." Combine with a scripted axe check from the accessibility page.
Responsive	Give viewport sizes and measurable expectations; ask the agent to measure, not eyeball.	The 1280 and 375 pixel grid check in example 2.
Console and network hygiene	Ask for console errors, failed requests and slow resources on a list of pages.	Example 5.

Acceptance criteria

A numbered list of observable outcomes; ask for PASS or FAIL per item with evidence.

The three review criteria in example 1.

Persona

Who is using the app, with what permissions and goal; forbids steps that persona would not take.

"You are a first-time reader without an account. You may browse and search, but you cannot sign in. Find a Technology book under 10 dollars and say what stops you from reviewing it."

Exploratory charter

A time-boxed mission with a focus area, not steps: "Explore X with Y to discover Z".

"Explore the catalogue search with unusual input (empty, very long, markup, accents) to discover errors or layout breakage. Report each oddity with the exact input and a screenshot. Stop after 15 minutes of actions."

Negative paths

Ask for invalid input, missing permission and server failure explicitly; give the trigger if the app has one.

Short review text (rejected) and the FAIL trigger (500) in example 1.

Accessibility

Ask for named checks and the evidence for each: keyboard order, accessible names, contrast, landmarks.

"Using only the keyboard, sign in and add a review. Report any control that cannot be reached or has no accessible name." Combine with a scripted axe check from the accessibility page.

Responsive

Give viewport sizes and measurable expectations; ask the agent to measure, not eyeball.

The 1280 and 375 pixel grid check in example 2.

Console and network hygiene

Ask for console errors, failed requests and slow resources on a list of pages.

Example 5.

General rules that follow from the runs:

Give the agent a stop condition. "Stop after the three checks" avoids a long, expensive wander.
Say what not to touch. "Do not change the application", or a list of allowed hosts and actions. Pair the prompt with the tool-level limits described in Securing Browser Agents.
Ask for evidence per criterion, never for a verdict alone, and ask for the absence evidence too ("no second request was sent").
Use test accounts and a throw-away profile (--isolated); never give an agent a real account’s password or a production site.
Make the data known. Seed the application (Bookshelf has /api/test/seed) so the expected values in the criteria are stable.

Evidence

A pass or fail is only as good as what backs it. The tool servers used above produce these kinds of evidence:

Evidence Source in the examples Strength

Evidence	Source in the examples	Strength
Network request and status	`browser_network_requests`, `list_network_requests`	Strong: machine-readable, unambiguous (the 201, 500, and the absence of a second POST).
Console messages	`browser_console_messages`, `list_console_messages`	Strong for errors; silent on logic faults.
Accessibility snapshot text	`browser_find`, `browser_snapshot`	Strong for content and roles; blind to appearance.
Computed values	`browser_evaluate` (grid columns, `scrollWidth`)	Strong when the value is the criterion.
Metrics and audits	`performance_start_trace`, `lighthouse_audit`	Strong for lab numbers; no field data for a local page.
Screenshots	`browser_take_screenshot`	Good for humans; weak as the sole basis of a model’s verdict.
Traces	`playwright-cli tracing-start` and `tracing-stop` write `.playwright-cli/traces/`	Strong for replay; open in the trace viewer.
Video "receipts"	`--caps devtools` tools such as `browser_start_video` and `browser_video_chapter`; `playwright-cli video-start`	A recording of what the agent did; useful in a pull request. Not exercised here.

Network request and status

browser_network_requests, list_network_requests

Strong: machine-readable, unambiguous (the 201, 500, and the absence of a second POST).

Console messages

browser_console_messages, list_console_messages

Strong for errors; silent on logic faults.

Accessibility snapshot text

browser_find, browser_snapshot

Strong for content and roles; blind to appearance.

Computed values

browser_evaluate (grid columns, scrollWidth)

Strong when the value is the criterion.

Metrics and audits

performance_start_trace, lighthouse_audit

Strong for lab numbers; no field data for a local page.

Screenshots

browser_take_screenshot

Good for humans; weak as the sole basis of a model’s verdict.

Traces

playwright-cli tracing-start and tracing-stop write .playwright-cli/traces/

Strong for replay; open in the trace viewer.

Video "receipts"

--caps devtools tools such as browser_start_video and browser_video_chapter; playwright-cli video-start

A recording of what the agent did; useful in a pull request. Not exercised here.

playwright-cli keeps the evidence on disk and prints only the path, which is cheaper in tokens than returning a snapshot inline. Traces, videos and screenshots as scripted-test artifacts are covered in Screenshots, Videos & Traces and the trace viewer in Playwright: Debugging, Tracing & Codegen.

Turning a session into a regression test

An exploratory session is disposable; the findings that matter should end up as scripted tests that run on every commit. Four documented routes exist, from most to least manual.

Copy the server’s code. Playwright MCP echoes the Playwright code it executed for every action (the Ran Playwright code blocks in the transcripts above). Joining the code of example 1 into a spec gives a normal Playwright test. This spec was assembled that way from the example-1 transcript, with the assertions written by hand because the agent’s "evidence" step was a snapshot read and not an assertion, and it was run against Bookshelf: 3 tests passed in Chromium.

import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }) => {
  await page.goto('/signin');
  await page.getByTestId('username').fill('reader');
  await page.getByTestId('password').fill(process.env.BOOKSHELF_READER_PASSWORD ?? '');
  await page.getByTestId('signin-submit').click();
  await expect(page).toHaveURL(/\/account$/);
});

test('a valid review is saved', async ({ page }) => {
  await page.goto('/books/1');
  await page.getByTestId('review-rating-input').selectOption('4');
  await page.getByTestId('review-text-input').fill('Clear and practical, with a great index.');
  const saved = page.waitForResponse((r) => r.url().endsWith('/api/reviews') && r.request().method() === 'POST');
  await page.getByTestId('review-submit').click();
  expect((await saved).status()).toBe(201);
  await expect(page.getByText('Clear and practical, with a great index.')).toBeVisible();
});

test('a short review is rejected without a request', async ({ page }) => {
  await page.goto('/books/1');
  let posts = 0;
  page.on('request', (r) => { if (r.method() === 'POST' && r.url().endsWith('/api/reviews')) posts++; });
  await page.getByTestId('review-rating-input').selectOption('3');
  await page.getByTestId('review-text-input').fill('Too short');
  await page.getByTestId('review-submit').click();
  await expect(page.getByRole('alert')).toHaveText('Write at least 10 characters');
  expect(posts).toBe(0);
});

test('a server failure rolls the review back', async ({ page }) => {
  await page.goto('/books/1');
  await page.getByTestId('review-rating-input').selectOption('2');
  await page.getByTestId('review-text-input').fill('This text will FAIL on save.');
  const failed = page.waitForResponse((r) => r.url().endsWith('/api/reviews') && r.request().method() === 'POST');
  await page.getByTestId('review-submit').click();
  expect((await failed).status()).toBe(500);
  await expect(page.getByRole('alert')).toHaveText('Could not save the review');
  await expect(page.getByRole('list').getByText('This text will FAIL on save.')).toHaveCount(0);
});

Source: Playwright: Test agents

Use the generator agent. npx playwright init-agents --loop=claude (loops: claude, codex, copilot, opencode, vscode, vscode-legacy) writes a planner, a generator and a healer agent definition plus a playwright-test MCP server entry. The first agent, a planner, explores the app and saves a Markdown plan; the generator turns a saved plan into a spec file after trying the steps live; the healer repairs failing tests. In a run on Bookshelf, the first agent saved specs/catalogue-search.md, the generator wrote tests/catalogue-search/search-no-matches.spec.ts (it passed), and the healer found the replacement for a deliberately stale locator (getByTestId('search-input')). Details and the files are in Playwright Test Agents.

Use playwright-cli to find locators. playwright-cli -s=bk generate-locator <ref> prints a durable locator for an element on the current page, and every command prints the Playwright code it ran, so a command-line session can be pasted into a test. playwright-cli install --skills writes a skill (.claude/skills/playwright-cli/SKILL.md and references, including one on test generation) so Claude Code knows the commands. See Playwright MCP & CLI.

Eject a cy.prompt. Save the generated commands over the cy.prompt call as described in example 7, and the test no longer needs the model or Cypress Cloud to run.

In all four routes, review the result: replace weak assertions (see the browser_verify_text_visible finding), replace generated waits and brittle selectors with the project’s conventions from Test Design & Page Objects, and run the test several times, as described in Flaky Tests & Reliability, before it joins the suite.

Running unattended in CI

A script-less check can run without a person if the agent is started non-interactively with the tools pre-approved. The shape of such a call with Claude Code is documented in Headless & CI Automation. The command below combines the documented flags (-p, --mcp-config, --allowedTools, --output-format, --max-turns) and was not run here, because the sandbox that produced this page could not authenticate Claude Code:

# Smoke-check a deployed preview with a script-less prompt; read-only tools, bounded turns
claude -p "$(cat prompts/review-smoke.txt)" \
  --mcp-config .mcp.playwright.json \
  --allowedTools "mcp__playwright__browser_navigate,mcp__playwright__browser_snapshot,mcp__playwright__browser_click,mcp__playwright__browser_type,mcp__playwright__browser_network_requests,mcp__playwright__browser_console_messages" \
  --max-turns 25 \
  --output-format json > result.json

Source: Claude Code: Run non-interactive mode

.mcp.playwright.json would register npx @playwright/mcp@latest --headless --isolated as the playwright server. Points that make an unattended run safe and useful:

Narrow tool allowlist. Do not allow browser_run_code_unsafe or browser_evaluate unless the criteria need them; see Excessive Agency & Tool Security and Securing AI-Assisted Development.
Bounded cost. --max-turns and a prompt-level stop condition cap tokens; the job also needs a timeout.
Machine-readable verdict. Ask the prompt to end with one JSON object (criterion, status, evidence) and parse it, or fail the job when the object is missing or malformed.
Isolation. A fresh runner, a throw-away profile (--isolated), a test account and an allowlist of hosts. Page content is untrusted input; see Prompt Injection.
Keep the scripted suite as the gate. The agent job reports; it should not be the only job that can block a release. The CI mechanics of the scripted suite are in Running in CI/CD.

The cloud variants (the Copilot cloud agent of example 3, Claude Code on the web, scheduled agents) are in Cloud & Background Agents.

Limits

Script-less testing has real limits, and several of them showed up in the runs above.

Non-determinism. The same prompt can produce different click paths, different wording and occasionally a different verdict. A run that passes today is not a promise about tomorrow, and a failure may be the agent’s mistake. A test whose result varies with the model’s mood cannot be a regression gate. Mitigations: criteria phrased as measurable outcomes, evidence from the network and the console rather than prose, and re-running a failure before believing it.

Cost and latency. A scripted Playwright step takes milliseconds; an agent step is a model call that reads a snapshot and decides. Example 1 took a dozen tool calls, plus an extra wait and re-read per criterion, for a feature a scripted test covers in a second. Large pages make snapshots large. Snapshot files instead of inline output, the --snapshot-mode option and playwright-cli all exist to reduce this, but the order of magnitude stays.

Hallucinated passes. Models want to finish. A model can report PASS because a snapshot looked plausible, because it confused a cached element with a new one, or because a verify tool returned Done for a weaker assertion than the prompt implied. The browser_verify_text_visible result in example 6 is a documented case: the check passed and the generated code asserted only that an element existed. Ask for evidence, check the evidence, and have a separate pass (a second agent, or a person) review the report, as the Claude Code best practices suggest with a verification subagent that has not seen the first agent’s reasoning.

Coverage you cannot measure. A scripted suite has a list of tests and, with instrumentation, a coverage figure. A script-less session has a transcript. There is no way to say which of the application’s states a run touched, and no way to know a criterion nobody wrote down was checked. Charters improve exploration but do not make it measurable.

Self-validation is not independent. The agent that changed the code and then validates it (example 3) shares its own blind spots. Independent criteria written by a person, and a review of the evidence, are what make the result trustworthy.

Security. The agent reads page text and may follow instructions hidden in it. Use isolated profiles and test data only, as covered in Securing Browser Agents and OWASP Top 10 for LLM Applications. Human approval for irreversible actions is covered in Human-in-the-Loop and Agent Guardrails & Safety.

For these reasons scripted tests stay the regression gate: they are deterministic, cheap, measurable and run in parallel. The agent’s job is to find what to script, to review what cannot be scripted cheaply, and to diagnose failures. The wider guidance on AI help for tests is in Testing & Debugging with AI.

Scripted, AI-generated or script-less?

Aspect Scripted AI-generated script Script-less

Aspect	Scripted	AI-generated script	Script-less
Who writes the steps	A person, in code.	An agent writes the code once (generator agent, `cy.prompt` ejected, codegen); a person reviews it.	The agent chooses the steps on every run.
Repeatability	Deterministic apart from flakiness.	Deterministic once committed.	Varies from run to run.
Cost per run	Milliseconds of compute.	Same as scripted after generation.	Model tokens and seconds per step.
Maintenance	Locators and flows need updating; a healer agent can propose fixes.	Same as scripted, plus review of generated code.	None for the test, but the prompt and criteria still need upkeep.
Evidence	Assertions, traces, screenshots in the report.	Same as scripted.	Whatever the agent collected; must be reviewed.
Coverage	Countable (tests, lines, user flows).	Countable.	Not measurable.
Best fit	Regression gate, CI, release checks.	Bootstrapping tests for a new feature quickly.	Exploratory sessions, new-feature verification, design and accessibility reviews, debugging.
Weak spot	Slow to write.	Weak or brittle generated assertions.	Hallucinated passes, cost, no coverage figure.

Who writes the steps

A person, in code.

An agent writes the code once (generator agent, cy.prompt ejected, codegen); a person reviews it.

The agent chooses the steps on every run.

Repeatability

Deterministic apart from flakiness.

Deterministic once committed.

Varies from run to run.

Cost per run

Milliseconds of compute.

Same as scripted after generation.

Model tokens and seconds per step.

Maintenance

Locators and flows need updating; a healer agent can propose fixes.

Same as scripted, plus review of generated code.

None for the test, but the prompt and criteria still need upkeep.

Evidence

Assertions, traces, screenshots in the report.

Same as scripted.

Whatever the agent collected; must be reviewed.

Coverage

Countable (tests, lines, user flows).

Countable.

Not measurable.

Best fit

Regression gate, CI, release checks.

Bootstrapping tests for a new feature quickly.

Exploratory sessions, new-feature verification, design and accessibility reviews, debugging.

Weak spot

Slow to write.

Weak or brittle generated assertions.

Hallucinated passes, cost, no coverage figure.

A practical combination: let a script-less session verify a new feature and discover the edge cases, convert the valuable checks into scripted tests by one of the four routes, and keep the agent for exploratory charters and failure diagnosis.

What changed recently

Playwright MCP 0.0.83 has an extended tool set: 25 core tools plus capability groups (testing, devtools, vision, pdf, network, storage, config). Element arguments use target, not ref. Options such as --secrets, --allowed-hosts, --snapshot-mode and --test-id-attribute exist. Playwright 1.62 announced that the MCP server and CLI ship bundled with Playwright, and npx playwright mcp --help and npx playwright cli --help work in 1.64.1.
Playwright Test Agents (planner, generator, healer) have existed since Playwright 1.56 and are configured with npx playwright init-agents.
Chrome DevTools MCP 1.x (1.12.0 at the check date) added Lighthouse audits, which report an extra "Agentic Browsing" category, and exposes usage-statistics and CrUX opt-outs.
Playwright 1.64 adds WebMCP support (page.webmcp) and page.getByRef(). Bookshelf registers no WebMCP tools, so the CLI’s webmcp-list command reported none.
cy.prompt was introduced in Cypress 15.4.0 and moved to beta in 15.13.0, with its experimentalPromptCommand flag removed; the Cloud AI features page lists it as generally available. Cypress 16 replaced the global environment accessor with cy.env(), which the example uses.
The Copilot coding agent is now called the Copilot cloud agent in the documentation, and VS Code’s browser tools are on by default.

Script-less Testing with AI Agents

The idea: goals instead of steps

The loop

Setting up Claude Code with a browser server

Example 1: verify the add-review feature from acceptance criteria

Example 2: compare the page with a design and fix the difference

Example 3: the Copilot cloud agent validates its own pull request

Example 4: the VS Code browser-agent testing loop

Example 5: console errors and Web Vitals with Chrome DevTools MCP

Example 6: the Playwright MCP verify tools (--caps testing)

Example 7: cy.prompt, the middle ground

Prompt patterns

Evidence

Turning a session into a regression test

Running unattended in CI

Limits

Scripted, AI-generated or script-less?

What changed recently

References