Automated accessibility testing with cypress-axe – Part 2

In the first part of the blog post , we presented an example of how to explicitly create automated accessibility tests with Cypress thanks to cypress-axe. In this part, we want to look at a much more extensive approach in the sense of shift left. For example, there is a11y-Linter, which can help to implement accessible applications well before they are processed in a CI/CD pipeline.

The URLs are provided via a YAML file, which is iterated in a shell script and the results are returned for a pipeline similar to a junit.xml. Finally, it describes how a11y-Linter can be used to check accessibility before a CI/CD pipeline is run.

In general, for example, integration with Cypress is easier to use than axe core via the CLI. However, it is also possible that you are working with other solutions in a project. For example, Selenium, Playwright or others can be used instead of Cypress. There are also integration options for axe for Playwright and Selenium.

There are also solutions for automated accessibility testing that are subject to licence fees, such as axe Monitor from deque, Applitools, Browserstack and many others. These are often much easier to use than the free options. QF-Test from version 9.0.0 with the a11y extensions from mgm is also particularly interesting here. The version that will be released at the end of the year offers integration of the axe libraries. It also offers additional a11y tests that significantly increase the possibilities of automated coverage. Furthermore, non-validatable elements are declared as warnings and improved localisation is provided in a clear report using screenshots.

Who should use the axe-core approach? Anyone who wants to automate without having to install or integrate e2e or GUI regression test tools such as Cypress, Selenium or Playwright.

Prerequisites

The following tools are required:

• Node.js and npm
• axe-core (CLI)
• YAML file for the page URLs
• Shell script for testing and evaluation

Generator for report in junit.xml format

Step 1: Installing Node and axe-core CLI

The installation of Node depends on the operating system. You can download the installer for Windows, Linux or MacOS from the official site. There is also a variant with choco for Windows and a variant with brew for Mac.

After that, we install the axe-core CLI tool globally via npm: npm install -g axe-core

Step 2: Creating the YAML file

A YAML file (e.g. menue.yml) is created that contains the menu structure and the URLs of the pages to be tested:

	
		yaml
		menu:
		  - name: "Home"
		    url: "https://example.com"
		  - name: "About"
		    url: "https://example.com/about"
		  - name: "Contact"
		    url: "https://example.com/contact"
	

Step 3: Shell script for iteration and evaluation

Now a shell script is created that reads the YAML file, retrieves the URLs, performs axe-core tests and stores the results in a JUnit XML format:

	
		shell
		#!/bin/bash
		# Install necessary tools
		npm install -g js-yaml junit-report-builder
		# Read URLs from YAML file
		URLS=$(js-yaml menu.yml | jq -r '.menu[] | .url')
		# Initialize JUnit XML report
		REPORT_FILE="a11y-report.xml"
		echo '<?xml version="1.0" encoding="UTF-8"?>' > $REPORT_FILE
		echo '<testsuites>' >> $REPORT_FILE
		echo '<testsuite name="Accessibility Tests">' >> $REPORT_FILE
		# Function to run axe-core and generate report
		run_axe() {
		  URL=$1
		  OUTPUT=$(axe $URL --quiet --format=json)
		  ERRORS=$(echo $OUTPUT | jq '.violations | length')
		  if [ $ERRORS -gt 0 ]; then    echo '<testcase classname="Accessibility" name="'$URL'">' >> $REPORT_FILE
		    echo '<failure message="Accessibility issues found">' >> $REPORT_FILE
		    echo "<![CDATA[$OUTPUT]]>" >> $REPORT_FILE
		    echo '</failure>' >> $REPORT_FILE
		    echo '</testcase>' >> $REPORT_FILE
		    return 1
		  else
		    echo '<testcase classname="Accessibility" name="'$URL'"/>' >> $REPORT_FILE
		    return 0
		  fi
		}
		# Iterate over URLs and run tests
		EXIT_CODE=0
		for URL in $URLS; do
		  run_axe $URL || EXIT_CODE=1
		done
		# Close XML tags
		echo '</testsuite>' >> $REPORT_FILE
		echo '</testsuites>' >> $REPORT_FILE
		# Exit with appropriate code
		exit $EXIT_CODE
	

Then save the script in a file, for example run-a11y-tests.sh and make it executable:

	
		shell
		chmod +x
	

Of course, the script can also be created in Python, Javascript or other languages. You could also expand the report file so that the issues are explicitly available.

Generally, it is also possible to transfer several websites directly via axe cli. However, with several sites, this would also mean a very long call.

Further left approach: a11y linter

Shift left with accessibility linter: accessibility from the outset for Webstorm/IntelliJ

In software engineering, ‘Shift Left’ is increasingly becoming the standard for identifying problems as early as possible and solving them faster. Ideally, this should be done directly in the development environment. In web development and especially in the area of accessibility (a11y for short), a shift-left approach means identifying and avoiding barriers at an early stage. Accessibility linters play a central role in this by helping developers to identify and eliminate barriers directly in the code. For Webstorm or IntelliJ, there are a number of plugins that integrate accessibility testing directly into the IDE, thus supporting developers in implementing an inclusive and accessible web application.

Why use accessibility linters?

Accessibility linters help developers ensure an accessible user experience by flagging common problems such as missing alt text, inaccessible colour contrast, or inadequately labelled forms. They ensure that developers recognise these barriers without having to constantly carry out manual testing or consult specialised tools. A shift-left approach with accessibility linters means that these checks take place as early as possible – ideally when the code is being written.

Accessibility linters for use during development

• 1. WAVE Accessibility Linter

  • Summary: WAVE is one of the most popular accessibility testing tools and offers browser plugins for all common browsers. It checks HTML code for barriers and displays errors directly in the browser.
  • Functions: Among other things, it finds missing alt texts, checks colour contrasts and recognises form elements without labelling.
  • Advantages: Very clear presentation of problems and suggested solutions directly in the browser and thus directly at the corresponding point. In addition, the code can be displayed at the corresponding point.

• 2. aXe Accessibility Linter

  • Summary: aXe is a widely used accessibility checking tool that offers a plugin for Webstorm and IntelliJ. It is based on the WCAG (Web Content Accessibility Guidelines) and was developed specifically for development teams that want to ensure a high level of accessibility.
  • Features: Checks for general accessibility issues and highlights these in the editor. Checks can also be customised to focus on specific WCAG criteria.
  • Pros: Extensive rule sets and a flexible approach that fits well into development workflows.

• 3. SonarLint

  • Summary: SonarLint supports many programming languages, but also offers some accessibility checks for web development. It works together with SonarQube and provides an overview of code quality and accessibility.
  • Features: Supports the search for semantic HTML errors, displays contrast issues and checks the structure of the code.
  • Advantages: Easy integration into DevOps workflows and easy collaboration with SonarQube.

4. Linting with ESLint and the @angular-eslint plugin

• Summary: A plugin for Angular that can be integrated directly into the ESLint configuration. This plugin checks the HTML code directly for accessibility and marks corresponding places. In addition, the developers are provided with detailed help on how to fix the problems found.
• Features: Includes rules for accessibility including keyboard usability, ARIA conformance and accessible HTML.
• Advantages: Easy to use as the plugin is integrated directly into the ESLint configuration. No additional installation required, as the plugin is ideally already used for other linter tasks that have nothing to do with accessibility.
• A shift-left approach: how accessibility linters help in practice

An accessibility linter enables developers to recognise and avoid possible barriers as they write the code. Instead of carrying out accessibility tests at the end of the development process, these linters support the shift-left approach and help to create accessible components and pages. Using such linters in the development phase has a direct learning effect: developers are trained in the principles of accessibility and learn how to implement these standards from the outset.

The following is a practical example of how linters are used.

When developing a user interface in Angular, a linter like the @angular-eslint plugin can alert developers directly in their IntelliJ environment if a button does not have an ARIA label, as shown in the following example.

This is our button element, which directly indicates that the linter has found an error.

Hovering over the error will display the error.

The WAVE plug-in, which runs as an extension in the browser, displays the same error:

These issues should be fixed directly before the code is checked in. In the case of our button, this is easily done by adding an aria label.

These hints help to avoid accessibility issues before they spread further in the code or require costly changes at the end of the accessibility check.

Recommendations for use

• Defined standards: Determine which accessibility criteria should be mandatory for your project and configure the linter accordingly.
• Automated tests: Integrate the basis of a linter into your CI/CD pipeline to ensure that new commits and merges meet accessibility requirements, for example using axe core.
• Team-wide use: Accessibility linters should be implemented for all developers on the team to ensure a consistent level of quality and to minimise barriers in everyday work.

Conclusion

The shift-left approach to accessibility can be successfully implemented using accessibility linters in the development environment. Tools such as WAVE, aXe, SonarLint and the @angular-eslint plugin can be used to detect and fix barriers in the editor. This saves time and money while helping to create more inclusive web applications. A shift-left approach that prioritises accessibility fosters a culture of inclusive development and makes accessibility a natural part of the development process.

axe core is an approach that does not require any additional e2e or GUI regression testing tools. However, you do lose some convenience here compared to integrations. There are also solutions available that are explicitly subject to a fee and that further increase convenience. If these fee-based, partly cloud-based solutions cannot be used, the axe core cli can be a good first approach.

Would you like to learn more about exciting topics from the adesso world? Then take a look at our previously published blog posts.

Also interesting:

• Automated accessibility testing with cypress-axe