Automated testing on accessibility

Rotterdam, 11 March 2016 (update)

There are several tools (online or downloadable) for testing your website on accessibility. In this overview I have tried to determine what parts of WCAG 2.0 level A and AA are tested. I have not discovered a tool that covers all techniques in one or more success criteria, but I might be wrong and one slipped through. Manual testing is still needed if you want to be sure that your website is accessible and conforms to WCAG 2.0.

Be aware that the more a tool can test it does not necessarily mean that this tool is better. Some tools prefer less false negatives (or positives).

Automated testing WCAG
WCAG aXe gt tenon.io eiii quail AChecker
1.1.1 Non-tekst content x x x x x x
1.2.1 Audio/video-only o x x
1.2.2 Captions x x x
1.2.3 Audio descr./alt. o x
1.2.4 Captions (live) o x
1.2.5 Audio descr. (prerec.) x o
1.3.1 Info and relationship x x x x x x
1.3.2 Meaningful sequence x
1.3.3 Sensory characteristics x x
1.4.1 Use of color x x x x x
1.4.2 Audio control o
1.4.3 Contrast x x x x
1.4.4 Resize tekst x x x
1.4.5 Images of tekst x x x
2.1.1 Keyboard x x x x x
2.1.2 Keyboard trap x
2.2.1 Timing adjustable x x x x x
2.2.2 Pause,stop,hide x x x x
2.3.1 3 flshes/below treshold x x
2.4.1 Bypass blocks x x x x
2.4.2 Page titles x x x x x
2.4.3 Focus order x x x
2.4.4 Link prupose x x x x x x
2.4.5 Multiple ways x x x
2.4.6 Headings and labels x x x x
2.4.7 Focus visible x x
3.1.1 Language of page x x x x x x
3.1.2 Language parts x x x x
3.2.1 On focus x x
3.2.2 On input x x x
3.2.3 Consistent navigation x x
3.2.4 Consistent identification x
3.2.5 Change on request x x
3.3.1 Error identification o x
3.3.2 Labels or instructions x x x x x
3.3.3 Error suggestion o x x
3.3.4 Error prevention o x
4.1.1 Parsing x x x x x x
4.1.2 Name,role,value x x x x x

Sources of these findings:

Wishlist for this overview: pa11y, WAVE, ...

Any information, additions and corrections are welcome!

Other resources