Intelligence Community Design System

Passing automated accessibility tests doesn't mean your app is accessible, it just means that no common issues were found.

A limitation with automated accessibility testing is that results often need someone to ensure the result is correct. This includes checking for false positives and false negatives.

Checks in automated accessibility testing tools have a yes or no outcome. The biggest limitation of these automated tools is that they can't test for context.

Deque’s Accessibility Developers' Guide shows what type of testing is needed to meet WCAG 2.2 Level AA .

Of the 66 success criteria needed to meet WCAG AA, there are none that can be met fully with automated testing alone.

It's possible for an app to be WCAG 2.2 Level AA compliant but still have usability issues. Some of these usability issues may also be accessibility issues.

Usability and context matter to everyone, including those with accessibility needs. Usability includes user experience, covering whether the tool is effective and efficient.

Manual accessibility testing, and other functional testing, is needed to make sure your apps or services are usable.

For some success criteria, such as those around colour contrast, automated testing tools are vital.

1.4.3 Contrast (Minimum) requires the visual presentation of text and images of text to have a contrast ratio of at least 4.5:1 (with exceptions).

An automated testing tool can't test for context. For example, it can't tell you if the failing contrast ratio was one of the permitted exceptions, which does actually meet the success criteria, and so returns a false positive.

Guideline 1.1 – Text Alternatives requires you to provide text alternatives for any non-text content so that it can be changed into other forms people need, such as large print, braille, speech, symbols or simpler language.

Most automated accessibility checking tools will warn you if alternative text does not exist on images. However, automated tools can't tell whether the alternative text provided is useful and providing enough context of the image.

If the alternative text was a file name, such asalt="picture1234.jpg" , this would make no sense to the reader.

Likewise, an overly visual description of your app logo does does not provide the reader with the brand recognition, whereas the app name would.

The same image could also be used twice in the same page, but with different purposes, so the alternative text would need to reflect this.