Accessibility conformance results for Headless UI, ahead of publication (one finding)

What package within Headless UI are you using?

@headlessui/react

What version of that package are you using?

v2.2.10

What browser are you using?

Chromium 151.0.7922.34 (Playwright), macOS

Reproduction URL

https://github.com/amansoomro062/railing

Not a CodeSandbox, and I want to be upfront about why: the reproduction here is the measurement harness itself. From a clone, two commands mount Disclosure exactly as your docs describe and replay the one failing check, and the scored run has a replayable Playwright trace:

pnpm --filter @railing-dev/adapter-headlessui run dev
npx @railing-dev/runner run --target headlessui --component accordion \
  --only accordion.header-is-heading --base-url http://localhost:5183

The adapter that mounts the component is about fifty lines and is included in full in the report below, so you can check the mounting before trusting anything else in this issue.

Describe your issue

This is not the usual bug report, so thank you for reading this far. These are accessibility conformance results from Railing, an open source project that tests React component libraries against the W3C ARIA Authoring Practices Guide and publishes the results. Headless UI is one of the seven libraries measured, and I wanted you to hear the results from me before anything is public.

The short version: Headless UI passes 63 of 64 checks. Four of five components pass everything. One moderate finding on the accordion pattern, detailed below.

Components were mounted with default configuration, following your documentation, with no hand-written ARIA. Headless UI ships no styles by design, so the harness is unstyled. That is the library working as intended, not a missing step.

The one finding: accordion.header-is-heading. The APG accordion pattern expects each header control to sit inside a heading (h1-h6 or role="heading"), because screen reader users navigate by heading and an accordion whose headers are not headings disappears from that shortcut. Mounted per your Disclosure documentation, Disclosure.Button has no heading ancestor. Severity moderate, cited against the APG pattern and WCAG 1.3.1.

I would also like to contribute a fix. If you would take a docs change adding the heading wrapper to the Disclosure example, or a change in the component if you would rather it ship there, I am happy to open the PR; tell me which you would prefer and I will push it. A fix that ships inside the notice window means the fixed score is what gets published.

We can also see the reasonable counterargument, and we want you to make it if you hold it: Disclosure is a primitive, not an accordion, and the heading wrapper may be the application's responsibility rather than the library's. If that is your position, say so and we publish your reasoning beside the score, in full and unedited.

Why tell you at all? Three reasons:

1. You should hear about a public score from us rather than come across it. No score for Headless UI is on our index yet, and none will be for at least fourteen days.
2. You may think we measured something wrongly, and please let us know. The likeliest source of error in this project is our adapter, not your library, and we publish corrections in the open (decision log).
3. When the index goes live, Headless UI will be on it at 63 of 64, with a caveat we print on every page: a high score means no violations were detected by automated testing, not that a component is accessible. If you would rather phrase your own caveat beside it, we publish responses in full and unedited.

How the measurement works: railing.dev/method · scoring · disclosure policy · source

Accessibility conformance results for Headless UI

You are hearing this before we publish anything. No score for Headless UI is on our index, and none will be for at least fourteen days, whatever this report says.

Railing runs component libraries against the W3C ARIA Authoring Practices Guide and publishes the results. Every check cites the clause it measures, every score names an exact version, and every result is run repeatedly and discarded if the answer changes.

The short version

Read this before the findings

We mounted your components like this: Mounted with default configuration, using only components Headless UI exports and no hand-written ARIA. Headless UI ships no styles by design, so the harness is unstyled, that is the library working as intended, not a missing step.

If that choice is the disagreement rather than the findings themselves, say so and we will publish your reasoning beside the score. It is a judgement call, not a measurement, and you are better placed to argue it than we are.

The 1 finding

One or more header controls are not contained in a heading.

• Check: accordion.header-is-heading, "Each header control sits inside a heading"
• Component: accordion
• Severity: moderate, meaning behaviour departs from the specified pattern; users are inconvenienced
• Why it matters: Screen reader users navigate documents by heading. If accordion headers are not headings, an entire page structure disappears from the shortcut that most users rely on to move around it.
• Expected: each control inside an h1–h6 or role=heading element
• We measured: hr-header-1, hr-header-2, hr-header-3 have no heading ancestor
• Measured against: APG pattern · WCAG 1.3.1 Info and Relationships
• Reproduce: pnpm railing run --target headlessui --component accordion --only accordion.header-is-heading --base-url http://localhost:5183

How we tested it

• Versions: @headlessui/[email protected]
• Browser: chromium 151.0.7922.34
• Configuration: default. We use only what the library exports and never hand-write ARIA, even where the documentation instructs the developer to. If we did, the score would measure how carefully we copied your docs rather than what ships in the box.
• Traces: every run above has a replayable Playwright trace, captured from exactly the run that was scored. Ask and we will send them, or regenerate them with the command below.

Reproduce any of it from a clone of the repository:

pnpm --filter @railing-dev/adapter-headlessui run dev
pnpm railing run --target headlessui --component <component> \
  --base-url http://localhost:5183 --repeat 3

Where we might be wrong

The most likely cause of a wrong result is our adapter, not your library.

We have got this wrong repeatedly. One library's first run scored 27% and almost all of it was a selector of ours. Another was reported as having a broken focus trap when the trap worked and our test was reading focus too early.

So the adapter source is included below. Please tell us if we mounted your component in a way you would not recommend, or if a check misreads the specification. We will correct it and, if it has already been published, correct that too.

Where to check us

• How the measurement works, including why the runner never learns which library it is testing.
• How scoring works, with every weighting and what is deliberately left out.
• Our disclosure policy, covering notice, right of reply, and conflicts of interest.
• The decision log, which is largely a record of results we got wrong and corrected.
• The source. Every adapter, spec and score, and the code that produced this report.

What happens next

• You have fourteen days before anything about this library is published.
• If you reply, your response is published beside the score, in full and unedited.
• If you ship a fix in that window, we publish the fixed score. A finding that gets fixed before publication is the best outcome this project has, not a story we lost.
• If fourteen days is not enough for your release process, ask for more and you will get it.

The adapter we used

This is the whole of the library-specific code. There is nothing else.

import { Disclosure, DisclosureButton, DisclosurePanel } from "@headlessui/react";
import { TEXT } from "@railing-dev/harness-kit";

/**
 * Headless UI accordion, built from three Disclosures.
 *
 * Headless UI ships no accordion component: Disclosure is a single collapsible
 * section, and stacking them is the documented way to build one. That is
 * recorded here because it bears on the heading requirement: the library gives
 * you a button and a panel, and nothing that would wrap them in a heading.
 */
export function AccordionHarness() {
  const sections = [
    { header: TEXT.accordionHeader1, panel: TEXT.accordionPanel1 },
    { header: TEXT.accordionHeader2, panel: TEXT.accordionPanel2 },
    { header: TEXT.accordionHeader3, panel: TEXT.accordionPanel3 },
  ];

  return (
    <>
      <button data-testid="hr-before" type="button">
        Before
      </button>

      {sections.map((section, index) => (
        <Disclosure key={section.header}>
          <DisclosureButton data-testid={`hr-header-${index + 1}`}>{section.header}</DisclosureButton>
          <DisclosurePanel data-testid={`hr-panel-${index + 1}`}>{section.panel}</DisclosurePanel>
        </Disclosure>
      ))}

      <button data-testid="hr-after" type="button">
        After
      </button>
    </>
  );
}

import { useState } from "react";
import { Combobox, ComboboxInput, ComboboxOption, ComboboxOptions, Field, Label } from "@headlessui/react";
import { TEXT } from "@railing-dev/harness-kit";

const OPTIONS = [TEXT.comboboxOption1, TEXT.comboboxOption2, TEXT.comboboxOption3];

/** Headless UI combobox. Label comes from the library's own Field/Label pair. */
export function ComboboxHarness() {
  const [value, setValue] = useState<string | null>(null);
  return (
    <>
      <button data-testid="hr-before" type="button">
        Before
      </button>

      <Field>
        <Label>{TEXT.comboboxLabel}</Label>
        <Combobox value={value} onChange={setValue}>
          <ComboboxInput data-testid="hr-combobox" />
          <ComboboxOptions data-testid="hr-listbox" static={false}>
            {OPTIONS.map((option, index) => (
              <ComboboxOption key={option} value={option} data-testid={`hr-option-${index + 1}`}>
                {option}
              </ComboboxOption>
            ))}
          </ComboboxOptions>
        </Combobox>
      </Field>

      <button data-testid="hr-after" type="button">
        After
      </button>
    </>
  );
}

import { useEffect, useState } from "react";
import { Dialog, DialogPanel, DialogTitle } from "@headlessui/react";
import { stampTestIds, TEXT } from "@railing-dev/harness-kit";

/** Headless UI dialog, mounted as its documentation describes. */
export function DialogHarness() {
  const [open, setOpen] = useState(false);
  // Headless UI puts role=dialog on the Dialog root, not on DialogPanel.
  // Tagging the panel measured the wrong element entirely.
  useEffect(() => stampTestIds({ "hr-dialog": '[role="dialog"]' }), []);
  return (
    <>
      <button data-testid="hr-before" type="button">
        Before
      </button>
      <button data-testid="hr-trigger" type="button" onClick={() => setOpen(true)}>
        {TEXT.dialogTrigger}
      </button>

      <Dialog open={open} onClose={() => setOpen(false)}>
        <DialogPanel>
          <DialogTitle data-testid="hr-title">{TEXT.dialogTitle}</DialogTitle>
          <input data-testid="hr-field-1" type="text" aria-label="Field one" />
          <input data-testid="hr-field-2" type="text" aria-label="Field two" />
          <button data-testid="hr-close" type="button" onClick={() => setOpen(false)}>
            {TEXT.close}
          </button>
        </DialogPanel>
      </Dialog>

      <button data-testid="hr-outside-content" type="button">
        {TEXT.outsideContent}
      </button>
      <button data-testid="hr-after" type="button">
        After
      </button>
    </>
  );
}

import { Menu, MenuButton, MenuItem, MenuItems } from "@headlessui/react";
import { TEXT } from "@railing-dev/harness-kit";

/** Headless UI menu. It is headless by design, so the elements are ours. */
export function MenuHarness() {
  return (
    <>
      <button data-testid="hr-before" type="button">
        Before
      </button>

      <Menu>
        <MenuButton data-testid="hr-trigger">{TEXT.menuTrigger}</MenuButton>
        <MenuItems data-testid="hr-menu">
          <MenuItem>
            <button data-testid="hr-item-1" type="button">
              {TEXT.menuItem1}
            </button>
          </MenuItem>
          <MenuItem>
            <button data-testid="hr-item-2" type="button">
              {TEXT.menuItem2}
            </button>
          </MenuItem>
          <MenuItem>
            <button data-testid="hr-item-3" type="button">
              {TEXT.menuItem3}
            </button>
          </MenuItem>
        </MenuItems>
      </Menu>

      <button data-testid="hr-after" type="button">
        After
      </button>
    </>
  );
}

import { Tab, TabGroup, TabList, TabPanel, TabPanels } from "@headlessui/react";
import { TEXT } from "@railing-dev/harness-kit";

/** Headless UI tabs. */
export function TabsHarness() {
  return (
    <>
      <button data-testid="hr-before" type="button">
        Before
      </button>

      <TabGroup>
        <TabList data-testid="hr-tablist">
          <Tab data-testid="hr-tab-1">{TEXT.tab1}</Tab>
          <Tab data-testid="hr-tab-2">{TEXT.tab2}</Tab>
          <Tab data-testid="hr-tab-3">{TEXT.tab3}</Tab>
        </TabList>
        <TabPanels>
          <TabPanel data-testid="hr-panel-1">{TEXT.panel1}</TabPanel>
          <TabPanel data-testid="hr-panel-2">{TEXT.panel2}</TabPanel>
          <TabPanel data-testid="hr-panel-3">{TEXT.panel3}</TabPanel>
        </TabPanels>
      </TabGroup>

      <button data-testid="hr-after" type="button">
        After
      </button>
    </>
  );
}

Thanks for building a library where the hardest thing to find was one heading.

Aman