QA Toolkit: A step-by-step approach to comprehensive and successful quality assurance

The district administration's new citizens' portal looked chic. “It all works great!” was the final verdict. The project team was very relieved, as the pressure of the deadline had increased the tension before the go-live. Shortly after the launch, however, the first angry citizens contacted the team: ’Is this intentional so that we don't submit any applications?’ It turned out that important buttons were not visible to people with visual impairments because of the chosen colour contrasts. Could testing and QA measures have prevented this aspect from being overlooked?

The German Federal Office of Administration (Bundesverwaltungsamt – BVA) developed the QA toolkit together with adesso to support the development of knowledge about quality assurance (QA) methods and standards. It also offers project teams the opportunity to assess their current situation, including specific suggestions for improvement on relevant QA topics.

The initial situation

Studies such as the Chaos Report by the Standish Group have consistently shown for years that only about a third of software development projects are successfully completed. The vast majority of projects are either completed outside the time and budget or not to the desired quality. There are many reasons for this. One of the most common causes is the lack of consistent integration of quality assurance measures in the development process from the outset.

In public administration, numerous large-scale software projects are implemented with a large number of people involved. External development and test service provider teams often work together with internal experts in projects who are responsible for the technical requirements and acceptance of the software. The number of projects with successful agile elements is steadily increasing over time in favour of the well-known classical methods (e.g. V-model), which means that new requirements and challenges for cooperation arise. Joint responsibility for quality is often not yet sufficiently established.

In addition, a lack of experience and knowledge in the areas of agile project implementation and integrated and holistic quality assurance often makes efficient implementation more difficult. In practice, time pressure towards the end of a project before acceptance usually makes QA appear to be a cost-intensive ‘luxury’ rather than a sustainable investment in the success of the project. As a consequence, quality assurance is often seen as an afterthought, which means that errors discovered late in the process are expensive to correct. The potential and the connection between holistic quality assurance and successful project management are still underestimated.

The recipe for success: agile elements

Instead of extensive documentation, agile methods rely on a reduction of complexity. The documentation created in agile projects is intended to be lively and practical in order to keep the information dynamic and up to date. Tools such as ticket systems or wikis can help. On the other hand, large and static mandatory documentation, which often disappears in a drawer, is avoided. It is the task to ensure sufficient documentation that enables maintenance or familiarisation. This saves time and effort without compromising project quality.

Iterative development in manageable units enables early error detection. Involving customers from the outset also ensures that the product meets actual needs through regular feedback. Regular retrospectives of past iterations enable continuous improvement of the team.

These agile success factors provided the inspiration and framework for the joint development of the QA modular system by the BVA and adesso.

The QA modular system helps build knowledge

In order to meet the challenges described in the initial situation, the idea of creating a modular tutorial with lightweight and independent learning units (microlearning) on the topic of quality assurance in software development arose several years ago at the BVA as part of a community of practice around Dr. Oliver Kortendick and Simone Mester. The software quality characteristics according to ISO 25010 and the use of the internationally valid and proven standards of the International Software Testing Qualifications Board (ISTQB) were to form the framework.

The QA toolbox was designed in such a way that it provides sufficient explanation and background information for familiarisation, even for users with little knowledge of the subject, and the content is related to the procedures and applications of public administration. The aim was to provide immediate benefits and the possibility of continuous improvement instead of difficult-to-apply and impractical textbook knowledge.

The project team, consisting of the initiators at BVA and adesso, finally began implementing the QA toolbox, each of whose modules deals with a selected topic and is structured as follows:

• Official definition according to the ISTQB glossary
• Vivid and practical application example (‘storytelling’)
• Description including images and videos
• Maturity assessments and suggestions for improvement for the selected module

Like this article, these small learning units each begin with an example of use that combines factual information with emotional experiences and stories from everyday work. It should take about 10-15 minutes to work through a topic, including the supporting information. A knowledge database also provides further in-depth information as needed.

Positioning for software projects

The project teams can then use simple means to determine their individual position regarding the test and QA measures currently in use. A four-level maturity model is provided for this purpose.

Participants from the project determine their current maturity level for the desired topics using a self-assessment with the help of a targeted catalogue of questions.

This self-assessment leads to suggestions for improvement in order to reach the next level of maturity in small stages. The practice-oriented suggestions are supported by best practices such as proven templates and checklists with a view to implementation.

In addition to the suggestions for improvement, a training concept was developed that offers in-depth exercises on specific building block topics. In four-hour units with a high practical component, initial workshops were held on the topics of test design procedures, reporting and risk-based testing.

This approach enables continuous improvement in small, achievable steps, and the QA toolbox offers projects and procedures immediate added value.

Practical application in a pilot project

The BVA put the QA toolbox into operation in March 2020 and, together with adesso, started full-scale practical application as part of a pilot project during the second half of the year. The following method was used:

• Kick-off workshop: The project participants were first introduced to the procedure and the QA toolbox, and a project profile with the overarching information (start, end, implementation method) for the project was prepared together.
• Baseline measurement of the maturity levels: In small mixed teams (technical, business), all questionnaires were then processed separately to initially determine the maturity levels. The teams then came together to discuss the individual results and consolidate them into a common view. In this way, all the building blocks for the test and QA measures used in the project were examined. The result was a comprehensive and jointly agreed outcome, along with initial optimisation approaches.
• Definition of measures: On the basis of the maturity levels determined, specific suggestions for improvement were jointly identified and implemented from the QA toolbox, such as the introduction of experience-based test measures and the strengthening of the risk-based approach
• Monitoring: Regular surveys and updates of the maturity assessment are used to monitor the effectiveness of the measures taken, both now and in the future.

Publication as an open-source project

After practical testing and the interest shown by other authorities and external companies in the QA toolbox, the idea of publishing it as an open-source project for the general public was born.

In order to meet the desire for reuse and joint work on the QA construction kit, the OpenCoDE platform was identified as a suitable location, originally initiated by the Federal Ministry of the Interior and for Home Affairs and the states of Baden-Württemberg and North Rhine-Westphalia.

The QA construction kit has been available for use on this public administration platform for the exchange of open source software since December 2024.