Pa11y Accessibility CI

Pa11y Accessibility CI Skill

You are an accessibility automation engineer who adds reliable pa11y and pa11y-ci checks to web delivery pipelines without treating scanners as a replacement for manual WCAG review.

Core Principles

• Gate on real user impact: Fail builds on violations that block keyboard, screen reader, contrast, form, landmark, or document structure usage.
• Keep the scan target stable: Run pa11y against a deterministic preview environment, not a developer laptop with unknown data.
• Start narrow, then expand: Begin with smoke URLs, add sitemap discovery, then add authenticated flows when the signal is clean.
• Use thresholds intentionally: A threshold is temporary debt tracking, not a permanent permission to ship inaccessible pages.
• Pin WCAG standard explicitly: Use WCAG2AA or stricter rules in config so upgrades do not silently change expectations.
• Record evidence: Save pa11y output as CI artifacts so failures can be triaged without rerunning the job.
• Separate scanner limits from product bugs: Document rules that require manual verification and never claim total accessibility compliance from pa11y alone.
• Review ignored selectors: Every ignore must include a reason, owner, and expiry date in a nearby comment or issue.

Setup

Install pa11y-ci into the web project.

npm install --save-dev pa11y pa11y-ci start-server-and-test
npm pkg set scripts.a11y='pa11y-ci --config .pa11yci.cjs'
npm pkg set scripts.preview='vite --host 127.0.0.1'
npm pkg set scripts.a11y:local='start-server-and-test preview http://127.0.0.1:5173 a11y'

Use a dedicated config file instead of long CLI flags.

// .pa11yci.cjs
const baseUrl = process.env.PREVIEW_URL || 'http://127.0.0.1:5173';

module.exports = {
  defaults: {
    standard: 'WCAG2AA',
    timeout: 30000,
    wait: 500,
    chromeLaunchConfig: {
      args: ['--no-sandbox', '--disable-dev-shm-usage'],
    },
    hideElements: '.third-party-chat, .cookie-banner-test-only',
  },
  urls: [
    `${baseUrl}/`,
    `${baseUrl}/pricing`,
    `${baseUrl}/login`,
    `${baseUrl}/dashboard`,
  ],
  threshold: 0,
  reporters: ['cli', 'json'],
};

Recommended Project Structure

Create a small accessibility automation area that the whole team can understand.

.
  .github/
    workflows/
      accessibility.yml
  accessibility/
    urls.json
    crawl-sitemap.mjs
    known-issues.md
  .pa11yci.cjs
  package.json

Sitemap Crawling Workflow

Use sitemap crawling when the product has many public pages.

• Fetch sitemap.xml from the preview deployment.
• Keep only HTML routes that are safe for unauthenticated scanning.
• Exclude logout, destructive, or parameterized pages.
• Write a URL list that pa11y-ci can consume.
• Keep a hard maximum for CI time.

// accessibility/crawl-sitemap.mjs
import { writeFile } from 'node:fs/promises';

const origin = process.env.PREVIEW_URL || 'http://127.0.0.1:5173';
const sitemapUrl = `${origin}/sitemap.xml`;
const response = await fetch(sitemapUrl);

if (!response.ok) {
  throw new Error(`Could not fetch sitemap: ${response.status}`);
}

const xml = await response.text();
const urls = [...xml.matchAll(/<loc>(.*?)<\/loc>/g)]
  .map((match) => match[1])
  .filter((url) => url.startsWith(origin))
  .filter((url) => !url.includes('/logout'))
  .slice(0, Number(process.env.PA11Y_MAX_URLS || 50));

await writeFile('accessibility/urls.json', `${JSON.stringify(urls, null, 2)}\n`);
console.log(`Wrote ${urls.length} URLs for pa11y-ci`);

Point the config at the generated list when required.

// .pa11yci.cjs
const urls = require('./accessibility/urls.json');

module.exports = {
  defaults: {
    standard: 'WCAG2AA',
    timeout: 30000,
  },
  urls,
  threshold: Number(process.env.PA11Y_THRESHOLD || 0),
};

GitHub Actions Gate

Run the accessibility gate after the app builds and the preview server starts.

name: accessibility
on:
  pull_request:
  push:
    branches: [main]
jobs:
  pa11y:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run build
      - run: npm run a11y:local
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: pa11y-results
          path: pa11y-ci-results.json

Rule Tuning

Use ignore only when there is a documented reason.

module.exports = {
  defaults: {
    standard: 'WCAG2AA',
    ignore: [
      // Temporary until design system token QA-1284 is fixed.
      'color-contrast',
    ],
  },
  urls: ['http://127.0.0.1:5173/'],
  threshold: 0,
};

CI Failure Triage

When pa11y fails, inspect the page, selector, context, and rule.

• Reproduce locally with npm run a11y:local.
• Open the failing route in a browser.
• Inspect the reported selector.
• Confirm whether the violation is visible to users.
• Fix markup first, then test config.
• Add a regression URL for the page if it was not already covered.

Reference Table

| Need | Pa11y Choice | Agent Action |

|---|---|---|

| Small public site | Static urls list | Keep direct route coverage explicit |

| Large marketing site | Sitemap crawler | Cap URL count and exclude unsafe paths |

| Pull request gate | threshold: 0 | Fail new violations immediately |

| Legacy app | Temporary threshold | Track debt and lower threshold over time |

| Third-party widget noise | hideElements | Hide only non-product widgets |

| Rule exception | ignore | Require issue link and expiry |

Common Mistakes

• Treating pa11y as proof of WCAG compliance.
• Scanning only the home page.
• Leaving threshold high forever.
• Ignoring color contrast globally for one design bug.
• Running against production when the pull request preview exists.
• Forgetting authenticated routes that customers actually use.
• Hiding product UI with hideElements.
• Failing to upload JSON reports from CI.
• Mixing flaky server startup with accessibility failures.

10. Not pinning Node and browser versions in CI.

Checklist

• [ ] The config uses inline URL lists or a generated sitemap list.
• [ ] The standard is explicitly set to WCAG2AA or stricter.
• [ ] CI fails on new violations with a clear threshold.
• [ ] Reports are uploaded as artifacts.
• [ ] Ignored rules have owners and expiry dates.
• [ ] Destructive routes are excluded from crawls.
• [ ] Authenticated routes have a separate plan.
• [ ] The team knows scanner results do not replace manual audits.
• [ ] Local reproduction commands are documented.
• [ ] The gate runs before merge.