How to run browser tests in GitHub Actions

To run Zerocheck tests in GitHub Actions, you write a job that installs the CLI and Chromium, then runs zerocheck run. A failing blocking test exits non-zero, which fails the job. There's no workflow generator, PR comment or GitHub App to install. If you require the job in branch protection, it can stop a merge.

Before you start

• Reviewed tests as YAML in your repository, with zerocheck.yaml and zerocheck in devDependencies. Download YAML brings a web test across.
• A stable environment the runner can reach, usually staging, with test data.
• ZEROCHECK_API as a repository variable, set to your instance URL.
• ZEROCHECK_TOKEN as a secret, using a project token from Settings.
• A secret for each test credential your environment references.

The workflow

name: Zerocheck
on:
  workflow_dispatch:
  pull_request:

permissions:
  contents: read

jobs:
  browser-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    env:
      ZEROCHECK_API: ${{ vars.ZEROCHECK_API }}
      ZEROCHECK_TOKEN: ${{ secrets.ZEROCHECK_TOKEN }}
      # Add the variables zerocheck.yaml references, for example:
      # APP_TEST_PASSWORD: ${{ secrets.APP_TEST_PASSWORD }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '22'
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx zerocheck run --env staging --json zerocheck-run.json
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: zerocheck-run
          path: zerocheck-run.json
          if-no-files-found: ignore

The last step keeps the run record even when tests fail. Pull requests from forks don't receive your secrets, so this job can't run for them.

Understand the exit status

Exit code	Meaning
0	Every blocking test passed
1	A blocking test failed, or with --fail-on-flaky passed only after a retry
2	No verdict: setup, configuration, policy or infrastructure errors, or a test that could not complete

Choose which tests should block a merge, and set blocks_merge: false on the others so a failed outcome stays visible without failing the job. To enforce any of this, require the job in your branch protection or ruleset.

Make sure the job tests your change

A test result can go stale as a pull request changes, and a shared staging URL may not be serving the commit under review. A CLI run record doesn't carry the repository, branch or commit, so wait for your preview deployment or start the app on the runner. Don't assume the job can reach a private app without network setup.

Where results appear

Your instance keeps the results and screenshots from CLI runs, and the job log prints a link to each one. The browser runs on your CI machine, while your instance makes the decisions. Recordings are available only when the environment has no configured credentials. For more detail, see running in an existing CI job.