Testing a pull request build in GitHub Actions

Create a reverse-proxy project

Create a durable Replay QA API token in Replay QA Settings → API, then use it to create a project for the build's local URL. This creates the project once; the workflow reuses its ID on later runs.

Terminal
export REPLAY_QA_API_KEY="lqa_your_token"

npx --yes replayqa@0.2.4 create-project \
  --name "My app · GitHub Actions" \
  --target-url http://127.0.0.1:3000 \
  --reverse-proxy \
  --instructions "Test the app's important user flows."

Use the port and instructions that match your app. If you also use the CircleCI example, create a separate reverse-proxy project for that integration so its proxy has its own project.

Add repository secrets

Add these Actions secrets in the repository that runs the workflow:

Keep both values in GitHub Secrets. Do not commit them to .replay/config.json or the workflow.

Add a CI runner script

The runner script owns the app-to-Replay tunnel for the full lifetime of the test. It should:

• start replayqa proxy with --ci --json
• wait for {"event":"heartbeat","ready":true} before starting QA
• call replayqa ci with the current repository, PR number, head SHA, branch, and workflow run ID
• poll /projects/{project_id}/ci-runs/status until terminal is true
• call replayqa ci-cancel from SIGINT/SIGTERM cleanup so a superseded GitHub job does not leave work running
• keep the app and proxy alive until QA is terminal

The reference repository has a complete pull request workflow and runner at scripts/run-replayqa-ci.mjs. Copy the runner into your project, then add this package script:

package.json
{
  "scripts": {
    "qa:ci": "node scripts/run-replayqa-ci.mjs"
  }
}

Add the pull request workflow

The following workflow runs only when a pull request is ready for review, cancels an older workflow for the same PR, gives Replay QA up to one hour to finish, and keeps the production deployment workflow separate.

.github/workflows/replayqa.yml
name: Replay QA · PR

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]

permissions:
  contents: read

concurrency:
  group: replayqa-pr-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true

jobs:
  replayqa:
    if: >-
      ${{ github.event.pull_request.draft == false
          && github.event.pull_request.head.repo.full_name == github.repository }}
    runs-on: ubuntu-latest
    timeout-minutes: 75
    env:
      REPLAYQA_CLI_VERSION: '0.2.4'
      REPLAYQA_PROXY_TIMEOUT_MS: '300000'
      REPLAYQA_PROXY_PORT: '18888'
      REPLAYQA_RUN_TIMEOUT_MS: '3600000'
      REPLAYQA_RUN_MARKER: pr-${{ github.event.pull_request.number }}
      REPLAYQA_GITHUB_REPOSITORY: ${{ github.repository }}
      REPLAYQA_GITHUB_PR_NUMBER: ${{ github.event.pull_request.number }}
      REPLAYQA_GITHUB_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
      REPLAYQA_GITHUB_HEAD_REF: ${{ github.event.pull_request.head.ref }}
      REPLAYQA_GITHUB_RUN_ID: ${{ github.run_id }}
      REPLAY_QA_PROJECT_ID: ${{ secrets.REPLAY_QA_PROJECT_ID }}
      REPLAY_QA_API_KEY: ${{ secrets.REPLAY_QA_API_KEY }}

    steps:
      - uses: actions/checkout@v7

      - uses: actions/setup-node@v7
        with:
          node-version: 22
          cache: npm

      - name: Install app dependencies
        run: npm ci

      - name: Install Replay QA CLI
        run: npx --yes replayqa@${REPLAYQA_CLI_VERSION} --version

      - name: Validate Replay QA configuration
        run: |
          test -n "$REPLAY_QA_PROJECT_ID"
          test -n "$REPLAY_QA_API_KEY"
          npx --yes replayqa@${REPLAYQA_CLI_VERSION} project \
            --ignore-config --project "$REPLAY_QA_PROJECT_ID" >/dev/null

      - name: Check and build the app
        run: npm run check

      - name: Start the app
        run: |
          nohup npm run start -- --hostname 127.0.0.1 --port 3000 \
            >"${RUNNER_TEMP}/app.log" 2>&1 &

          for attempt in {1..60}; do
            curl --fail --silent http://127.0.0.1:3000 >/dev/null && exit 0
            sleep 1
          done

          cat "${RUNNER_TEMP}/app.log"
          exit 1

      - name: Test the pull request with Replay QA
        env:
          REPLAYQA_PROXY_LOG: ${{ runner.temp }}/replayqa-proxy.jsonl
        run: npm run qa:ci

      - name: Upload Replay QA logs
        if: always()
        uses: actions/upload-artifact@v7
        with:
          name: replayqa-ci-logs
          if-no-files-found: ignore
          path: |
            ${{ runner.temp }}/replayqa-proxy.jsonl
            ${{ runner.temp }}/app.log

Pin REPLAYQA_CLI_VERSION to a version you have validated. Update the pin deliberately when adopting a newer CLI release.

Customize the app command and test goal

Change the build command, start command, local port, and readiness URL to match your app. The reference runner reads REPLAYQA_PROMPT; set it in the workflow when Replay QA should concentrate on particular flows:

env:
  REPLAYQA_PROMPT: >-
    Test sign in, create a project, edit its settings, and verify keyboard navigation.