In the inves-tigated works of literature, only three pieces of literaturecompared multiple automatic accessibility testing tools toevaluate their effectiveness. This limited number of com-parative analyses is not sufficient to show the usefulness ofthe existing automated tools. Thus, it is crucial to devotecontinued effort to perform further comparative analysisconsidering the benefits of automatic testing tools in futureaccessibility research.

it could also be interesting to further research what accessibilitytesting practices are used in websites and their developing teams. These testing practicescould then be compared regarding their efficiency in achieving a high level of accessibilitycompared to time spent on testing and fixing accessibility failures.

Researching that very question – ifapplying these testing practices in the forementioned manner actually is most effective –could be the next step, in an effort to establish how to efficiently and effectively test foraccessibility.

identifying currently used testing practices it is possible to find and useaccessibility testing practices that are performed at a similar testing level using similartesting methods.

3.2.2 Accessibility testing techniques in web development

Even though testing might be partly or completelycorresponding with software testing theories, the practice can be quite different from thetheory, depending on the organization and development team. To identify accessibilitytesting methodologies that can be used in practice, not only in theory, I chose to collectdata on how testing is being done from a sample of developers in teams that are affectedby accessibility legislation

Brajnik, Yesilada and Harper (2011) studied if expertise in accessibility evaluation mattersand found that depending on the metric used it does. They also found that single expertcan identify about 72% of accessibility problems, two can reach 94% and three expertscan cover all of the problems. In comparison at least 14 nonexperts would be necessaryto identify all true problems and one expert could only identify about an average of 50% ofthe problems. This study also shows that experts spend much less time on the evaluationwhile being more confident on the results.

The EN standard references almost directly the WCAG2.1 levels A and AA guidelines, which means these guidelines can effectively be used asa reference point to develop an accessible website when trying to conform to the EUdirective (Aluehallintovirasto 2019).

The question of how testing is currently being done, does not state a clear hypothesis, butis set to look at the collected data without being certain what that data will conceal.According to Saaranen-Kauppinen and Puusniekka (2009, 6-7) the lack of a clearhypothesis and data-driven research are factors to categorize this research as qualitative.Typical methods for gathering information in qualitative research are observations andinterviews, or at present the use of existing material and documents as well (Saaranen-Kauppinen & Puusniekka 2009, 47). An observative approach could yield deep insights inthe existing testing practices in an organization, but would restrict the data collection toone or two development processes due to the small scope of this research. Due to thelack of existing material of the organizations in question I chose to conduct interviewsinstead, as I expect to gather a good understanding of existing testing processes indifferent types of relevant organizations.

During her research she found that developing accessible websites isn’t necessarilyexpensive, but can become expensive when the accessibility requirements have to beapplied at the very end of a development process. This confirms my assumption that thechosen topic is currently quite relevant and important.

To conclude, the objective of this research is to identify opportunities to apply webaccessibility testing methods within readily used testing practices.

This research aims to identify the intersection of these three groups of testingmethods, the currently applied testing methods, software testing methods more generallyand web accessibility testing methods