Setting the file. One moment.
Subchapter 28.6
references/spec-driven-testing.mdMarkdown11 KBView on GitHub
End-to-end workflow for authoring and maintaining Playwright tests using playwright-cli. The three sections below can be used independently:
All three lean on the same mechanic: run npx playwright test --debug=cli in the background, then playwright-cli attach tw-XXXX to drive the paused page interactively. See playwright-tests.md for the debug/attach mechanics and test-generation.md for how every playwright-cli action emits Playwright TypeScript.
Goal: produce a spec file (e.g. specs/<feature>.plan.md) that enumerates the scenarios to test. Always write the spec to a file.
Check the workspace has Playwright installed before anything else:
# Either of these confirms a workspace:
test -f playwright.config.ts || test -f playwright.config.js
npx --no-install playwright --versionIf there is no Playwright install, bootstrap one and let the user pick the defaults:
npm init playwright@latestA seed test is a minimal test that lands the page in the state every scenario starts from: navigation to the app, any required login, feature flags, etc. Scenarios assume a fresh start after the seed. --debug=cli pauses inside this test, so the seed is where every planning and generation session begins.
Minimum viable seed:
// tests/seed.spec.ts
import { test } from '@playwright/test';
test('seed', async ({ page }) => {
await page.goto('https://example.com/');
});Preferred — push navigation into a fixture so scenario tests reuse it:
// tests/fixtures.ts
import { test as baseTest } from '@playwright/test';
export { expect } from '@playwright/test';
export const test = baseTest.extend({
page: async ({ page }, use) => {
await page.goto('https://example.com/');
await use(page);
},
});// tests/seed.spec.ts
import { test } from './fixtures';
test('seed', async ({ page }) => {
// Fixture already navigates. This empty body tells agents where to start.
});If no seed exists, create one that at least navigates to the app.
Launch the app via the seed in the background and attach:
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/seed.spec.ts --debug=cli
# wait for "Debugging Instructions" and the session name tw-XXXX
playwright-cli attach tw-XXXXResume so the seed runs, then probe the app:
playwright-cli resume # resume so that seed test runs fully
playwright-cli snapshot # inventory of interactive elements
playwright-cli click e5 # follow a flow
playwright-cli eval "location.href" # read URL / state
playwright-cli show --annotate # ask the user to point at somethingMap out:
Important: Do not just open the app url with playwright-cli, always go through the test to capture any custom setup done there. Important: Stop the background test when done exploring.
Save under specs/<feature>.plan.md. Use this structure:
# <Feature> Test Plan
## Application Overview
<One paragraph describing what the feature does and why it matters.>
## Test Scenarios
### 1. <Group Name>
**Seed:** `tests/seed.spec.ts`
#### 1.1. <kebab-case-scenario-name>
**File:** `tests/<group>/<kebab-case-scenario-name>.spec.ts`
**Steps:**
1. <Concrete user step>
- expect: <observable outcome>
- expect: <another observable outcome>
2. <Next step>
- expect: <outcome>
#### 1.2. <next-scenario>
...
### 2. <Next Group>
**Seed:** `tests/seed.spec.ts`
...Guidelines:
should-add-single-todo → should-add-single-todo.spec.ts).fill“).- expect: bullets; each becomes an assertion during generation.Goal: take a spec file and produce Playwright test files. Optionally update the spec if it has drifted.
specs/basic-operations.plan.md.1.2), a whole group (1), or all.**Seed:** line of the scenario’s group.For each target scenario, in sequence (never in parallel — scenarios share the seed session):
PLAYWRIGHT_HTML_OPEN=never npx playwright test <seed-file> --debug=cli # background
playwright-cli attach tw-XXXX
# resumeDo not just open the app url with playwright-cli, always go through the test to capture any custom setup done there.
Walk the scenario’s Steps: one by one with playwright-cli, treating the spec as the plan and the live app as the source of truth. If a step is vague (“click the button” — which button?), references an element that no longer exists, or contradicts the app’s actual behaviour, use your judgement: update the spec to match what the app really does, then keep going. Editing the spec mid-generation is expected.
Every action prints the equivalent Playwright TypeScript (see test-generation.md):
playwright-cli snapshot # find refs
playwright-cli fill e3 "John Doe" # -> page.getByRole('textbox', {...}).fill(...)
playwright-cli press Enter
playwright-cli click e7For each - expect: bullet, add an explicit assertion. See test-generation.md for details.
Collect the generated code and write the test file at the path given in the spec:
// spec: specs/basic-operations.plan.md
// seed: tests/seed.spec.ts
import { test, expect } from './fixtures'; // or '@playwright/test' if no fixtures file
test.describe('Singing in and out', () => {
test('should sign in', async ({ page }) => {
// 1. Navigate to the application
// (handled by the seed fixture)
// 2. Type 'John Doe' into the username field
await page.getByRole('textbox', { name: 'username' }).fill('John Doe');
// 3. Type password
await page.getByRole('textbox', { name: 'password' }).fill('TestPassword');
// 4. Press Enter to submit
await page.getByRole('textbox', { name: 'password' }).press('Enter');
await expect(page.getByRole('heading')).toContainText('Welcome, John Doe!');
});
});Rules:
// N. <step text> comment before its actions.1. ordinal)../fixtures if the project has one; otherwise @playwright/test.Loop 2.2 over the targeted scenarios one at a time, restarting the seed between each so every test starts from a clean page. This is safe to parallelise due to unique generated session names - just make sure each test run is stopped.
After generation, run the new tests once:
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/<group>/<scenario>.spec.tsAny failure goes to Section 3.
Goal: fix failing tests, and update the spec if the app’s intended behaviour changed.
PLAYWRIGHT_HTML_OPEN=never npx playwright testRecord the list of failing <file>:<line> entries and process them one at a time. Do not attempt parallel fixes — shared state and the single CLI session make that fragile.
Run the single failing test in debug mode in the background, then attach:
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/<group>/<scenario>.spec.ts:<line> --debug=cli
# wait for "Debugging Instructions" and the tw-XXXX session name
playwright-cli attach tw-XXXXThe test is paused at the start. Step forward or run to until just before the failing action or assertion, then diagnose:
playwright-cli snapshot # did the element change / move / rename?
playwright-cli console # app-side errors?
playwright-cli network # failed request? wrong payload?
playwright-cli show --annotate # ask the user to point somewhereCommon causes: selector drift, new wrapper element, label/ARIA rename, timing (transition, async load), assertion text updated in the app, test data leaking between runs.
Rehearse the corrected interaction with playwright-cli — the generated code in the output is what you paste back into the test.
Edit the test file: update the locator, assertion, step order, or inputs to match the corrected behaviour. Stop the background debug run. Rerun the single test to confirm green.
Never skip hooks or add sleeps as a fix. Never use networkidle.
Open the spec referenced by the // spec: header in the test file and locate the scenario that matches the test.
2.3),Only after the user answers, either update the spec (intentional change) or file/flag the test as covering a bug (regression).
test.fixme(...) with a comment pointing at the user’s decision or issue link. Never silently skip.| For… | See |
|---|---|
--debug=cli / attach mechanics | playwright-tests.md |
How playwright-cli actions become TS | test-generation.md |
| Mocking requests during exploration/generation | request-mocking.md |
| Managing the CLI browser session | session-management.md |