Tools

Pa11y

An open-source command-line accessibility tester. The CI-friendly companion to axe-core CLI; pa11y-ci powers the nightly site-wide accessibility regression workflows most large accessibility programmes adopt.

Pa11y is an open-source automated accessibility tester. Originally built at Nature Publishing Group and now maintained by an independent community, Pa11y has been the go-to “accessibility tests in CI” tool for the past decade.

What Pa11y does

Pa11y at the command line takes a URL and runs accessibility checks against the rendered page, outputting a list of violations:

$ pa11y https://example.com
Welcome to Pa11y

> Running Pa11y on 1 URLs:
...

Error: This element does not have an attribute "alt".
   • Element: <img src="..." />
   • Code: HTML_CS.Principle1.Guideline1_1.1_1_1.H37
   • Selector: body > section:nth-child(2) > img

Pa11y runs in a headless browser (Puppeteer or Playwright under the hood) so it sees the rendered DOM, not just the source HTML — meaning it can audit JavaScript-rendered content the same way a user would encounter it.

What runners it supports

Pa11y has historically supported two rule engines:

  • HTML_CodeSniffer (the default; the original Pa11y backbone).
  • axe-core (via pa11y --runner axe).

You can run both at once and merge the results — a “everything I can detect automatically” mode. Most modern Pa11y deployments use the axe-core runner exclusively, because axe-core has the most active rule development.

pa11y-ci — the multi-URL workflow

The headline use case is pa11y-ci, a wrapper that runs Pa11y against many URLs at once and exits with an appropriate status code for CI:

$ pa11y-ci --sitemap https://example.com/sitemap.xml

This is exactly the workflow most accessibility programmes adopt for nightly regressions: feed the site’s sitemap into pa11y-ci on a cron schedule, fail the build if any new violations appear, alert when the violation count goes up.

Configuration lives in .pa11yci (or a JSON file), allowing per-URL exclusions, severity thresholds, viewport sizes, custom actions (login, click-through), and other automation.

Pa11y vs axe-core vs Lighthouse

A reasonable mental model:

  • axe-core is the rule engine — the underlying detection library. You use it through a wrapper.
  • Lighthouse is one wrapper, optimised for full-site quality audits (Perf + A11y + Best Practices + SEO).
  • Pa11y is another wrapper, optimised for accessibility-only CI gating across many URLs at scale.
  • axe-core CLI is the third — bare axe, no Pa11y wrapper, also CI-friendly.

For per-page deep-dives, Lighthouse is better. For site-wide regression testing with many URLs, Pa11y is the practical default. For build-blocking PR checks on a single URL, axe-core CLI or Lighthouse-CI both work fine.

Limitations

Pa11y inherits all automated-tool limitations:

  • 30-40% of WCAG issues are automatically detectable; the rest needs human judgment.
  • It can’t test keyboard navigation, focus order, or screen-reader behaviour.
  • It runs in a headless browser, so it can’t catch issues that only appear during real user interaction (rage-clicks, form abandonment, unexpected modal stack-up).

But within the automated-detection ceiling, Pa11y is the most operationally flexible tool of the major options — particularly when the audit scope is “the whole site, every night” rather than “this specific PR.”