H810 – Introduction to evaluation
How do I evaluate the web accessibility of a website, respectively our online learning resource was always my main question and I felt practically unable to do that. Well, I relied mostly on human testing but as Seale points out, because evaluating accessibility and understanding the guidelines and tools is so difficult, actually expert human judgement would be needed to do a proper check.
Well, reading Seale’s chapter took a little bit pressure away and I feel not completely incompetent, but realized that others, more proficient learning technologist, or other professionals struggle as well with the ever-increasing and changing guidelines and standards, but also to keep up with the newest evaluation tools.
Seale introduces four types of accessibility tools:
- filter and transformation tools – assist main.y web users, either modify a page or supplement an assistive technology or browser.
- design and authoring tools – fall into two categories: multimedia design tools and tools that simulate disability, but the number of tools that exists is small
- evaluation tools – conduct a static analysis of web pages and return a report or rating
- evaluation and repair tools – identify problems and recommend improvements, e.g. W3C HTML Validation Service
However the validity and reliability of evaluation and repair tools is often questionable. Brajnik (2001 cited in Seal, 2006) argues that a crucial property that needs to be assessed is the validity of the rules that tools operate. A valid rule needs to be correct and complete and can be assessed using one of three methods: comparative experiments, rule inspection and testing, and page tracking.
Choosing an evaluation or repair tool according the needs and context one operates is also an issue that needs to be considered and depends on a number of factors, e.g. whether the designer wants to focus on general or specific accessibility issues, if he wants to test against WCAG-1 (2), Section 508 or both, whether the website to be evaluated are small or large, or if the designer is working as an individual or as part of a larger organisation, to name just a view.
However, like mentioned above there are problems with evaluation and repair tools. The main problems are:
- difficulties with judging the severity of the error.
- dangers of inducing a false sense of security.
- dangers of encouraging over-reliance.
- difficulties understanding the results and recommendations.
Seale and others therefore recommend a combination of automatic and manual, respectively human (expert) judgement of accessibility. But that also requires time and resources, a certain number of people and a level of expertise, which might explain why a lot of websites do not fulfil web accessibility standards/guidelines.
Well, I my online resource will not get the W3C logo, yet but I am working on it 😉