The opening keynote by Royal College of Art professor Anthony Dunne focused on the provocative ‘what if?’ design he and his RCA students have been producing. I’d seen some of this work (mouse-powered TVs, anyone?) in Graham Pullin’s Design Meets Disability book (a review of which you can expect here very soon), and this talk, like the book, made me wonder if we could do with some more critical design in web accessibility: let’s try some accessibility solutions that might on first thoughts seem ‘wrong’ but actually have a positive contribution to make.

I also enjoyed William Hudson revisiting his CHI 2009 paper on the ICT profession, user-centred design and the empathisers/systemisers theory of Simon Baren-Cohen and his work with people on the autistic spectrum. We’ll all be familiar with the natural attraction of programming to people with Asperger syndrome, but the results of William’s survey of ICT professionals, with interesting gender differences, remind us that there can be an issue surrounding lack of empathy amongst developers of their end users. So the question is, how far can/should we raise empathy, and what’s the best way of doing so? With data? With ‘eureka experiences’, the kind of which seem to be particularly successful from an accessibility perspective?

Throughout the conference I heard from and talked to people about work in diverse areas: from Pradipta Biswas‘ simulations for inclusive design to a study of anxiety and wiki contribution to an investigation of older people’s attitudes to social networking. My own paper presentation lasted all of 5 minutes, followed by a 30 minute discussion with Tom McEwan and several others on the challenges facing human-centred technology commercialisation (getting people to use/buy your innovation in a human-centred way). This was not the direction I’d anticipated the discussion heading, but actually it was extremely rewarding for me to be forced to consider our work in this way.

The closing keynote from Bill Buxton was extremely entertaining, but with some serious lessons for ICT designers – in particular to be context-aware (socially, culturally, politically) of what they are doing and what has gone before. He mused over the fact that not one of the reports he’s read on the iPhone’s innovative qualities mentions a 1993 mobile phone that used the same type of buttonless touch based interaction . This was an illustration of the long-nose effect of innovation (the opposite of the long tail in retail strategy) – the many years an innovation typically spends ‘under the radar’, being rethought, reworked and refined before suddenly bang! it becomes a commercial success. How can we shorten this lag – give innovation a nose-job(!)?

From an accessibility perspective, listening to the two keynotes made me realise how much more I need and want to know about innovations that may still be under the radar- and also how frustrated I get with accessibility research and design that is horribly context-unaware, that seems ignorant of the efforts that have gone before, and as a result solves the wrong problem, or is no solution at all. This is quite different to critical design for accessibility, where there is a genuine contribution to be made, even if it is to say “ah, no, that doesn’t work right now”.

So at the end of it all, I left Cambridge with a desire to learn more, to look in new places for information and inspiration, but also reassured that the way I think about accessibility and inclusion technology is, I reckon, headed in roughly the right direction!

Read more

HCI 2009 conference web site

BBC article on HCI 2009 – focusing on the Open House demonstration of innovative technology.

In a previous blog post I touched on some of the problems with academic web accessibility research – in particular the cost barriers to accessing research published in some journals, and the sometimes lengthy time-to-publication, which can reduce the impact of late-breaking work in a fast moving area like web accessibility.

Copyright reasons prevent me from replicating the work published in the journal. So instead I’ve provided a brief overview of each of the papers that appear, with a link to the page where you can access a copy of the full paper. The authors are also bound by copyright agreements, but I’m sure each would be happy to answer further questions about their work.

The subjects range from from accessibility evaluation and measurement, to supporting accessibility of Web 2.0 applications to investigating the accessibility benefits of semantic markup to effective policies for using the Web to its full potential in enabling access to disabled people to online information and experiences.

From Web accessibility to Web adaptability; Brian Kelly, Liddy Nevile, Sotiris Fanou, Ruth Ellison, Lisa Herrod and David Sloan.

A review of web accessibility from an organisational and policymaker’s perspective. This paper focuses on ways to strike a balance between a policy that limits the chances of unjustified accessibility barriers being introduced in web design while also providing enough flexibility to allow the web in a way that provides the best possible user experience for disabled people by acknowledging and supporting the diversity of and the occasional conflicts between the needs of different groups.

Experimental evaluation of usability and accessibility of heading elements; Takayuki Watanabe.

This paper describes an investigation into the usability and accessibility impact of effective use of HTML heading elements on accessibility, and provides data to support the argument that these features positively impact on usability and accessibility for visually impaired web users.

WAI-ARIA live regions and channels: ReefChat as a case example; Peter Thiessen, Erin Russell.

A case study describing application of the World Wide Web Consortium (W3C)’s Web Accessibility Initiative Accessible Rich Internet Application (WAI-ARIA) in creating a chat application usable and accessible by sighted and visually impaired people.

Web accessibility and open source software; Željko Obrenović.

This is a review of the many diverse technologies and solutions that have been developed under an Open Source Software (OSS) approach, and a discussion of how accessible web browsing can be improved through using these technologies in innovative ways.

Tool independence for the web accessibility quantitative metric; Markel Vigo, Giorgio Brajnik, Myriam Arrue and Julio Abascal.

Efficient and accurate web accessibility evaluation on a large scale remains an important objective in the quality assurance of web content, yet the limitations of automated evaluation methods are well known. The authors of this paper propose the Web Accessibility Quantitative Metric as a reliable and independent measure of a Web site’s accessibility.

Editing Wikipedia content by screen reader: Easier interaction with the Accessible Rich Internet Applications suite; Marina Buzzi and Barbara Leporini.

This paper investigates the increasingly important topic of supporting disabled people as producers, and not just consumers, of web content. Wikipedia is the ‘Web 2.0′ example site used, and the authors explain how WAI-ARIA can be used to enhance non-visual usability of editing features of the Wikipedia interface.

Enabling web users and developers to script accessibility with Accessmonkey; Jeffrey Bigham, Jeremy Brudvik, Jessica Leung and Richard Ladner.

This paper focuses on supporting extension of browser functionality for accessibility purposes. AccessMonkey is a scripting framework that can be used to improve accessibility by modifying standard browser – and the authors demonstrate examples of how this can be used to enhance the user experience for disabled people, for example where appropriate assistive technology may not be available.

I hope by providing an introduction to each paper I’ve given you an insight into the diversity of web accessibility research currently taking place, and introduced you to some of the people who are carrying out this important work.

I’d been asked to give a perspective from the UK on developments in web accessibility over the years, and in putting together my talk, I ended up with a 10 year biography of web accessibility. I thought this was a nice, round figure, given that it’s almost 10 years to the day since version 1 of WCAG was published by the W3C on 5th May 1999; and nearly 10 years since I started working in this area as a researcher/consultant in the newly formed Digital Media Access Group.

Of course, a lot of very valuable research and development had been carried out in the field of web accessibility before then. But WCAG brought together existing knowledge and expertise in a way that allows us to define its publication as a catalyst for the popularisation of accessibility as a major topic of web development.

Four Stages of the Evolution of Web Accessibility

Slides from my talk are available on Slideshare, but here’s a summary of my perspective of what’s happened since WCAG 1.0 was published. I plot developments in four stages, with approximate time periods for each.

1. Advocacy and conformance (1999-2001)

Activity is all about raising awareness of web accessibility – of how disabled people use the web, putting forward financial, technical, legal and moral arguments for considering accessibility in web design, and encouraging following WCAG 1.0.

2. Accessibility joins the Web Standards movement (2002-2004)

Encouraged by books from Zeldman, Clark, Thatcher et al and Slatin and Rush, the Web Standards movement adopts WCAG conformance as part of a suite of technical skills a web developer should adopt in order to create web sites that look good but function appropriately across diverse browsing platforms. Grass-roots web developers embrace accessibility, and become creative in solving accessibility-related design and development problems; larger organisations like Yahoo! make significant efforts to promote web standards.

3. Guidelines are not enough (2005-2007)

The UK Disability Rights Commission Formal Investigation into Web accessibility finds that disabled people could use some sites for their intended purpose even when WCAG conformance indicated otherwise, and vice versa. This illustrates the challenges of making the WAI model of accessibility (requiring WCAG-conformant content and UAAG-conformant browsers and assistive technologies) work in the real world; and more specifically the limitations of an organisational accessibility policy defined solely by conformance with standards for accessibility of web content.

So, the concept of holistic accessibility is developed, inspired by blended approaches to learning, where using multiple routes to achieving accessible end goals on the web are encouraged in cases where practical challenges may make a single, universally accessible solution, impossible.

4. Web 2 and new challenges for human-centred accessibility (2008-present)

Defining accessibility as supporting disabled people perform tasks on the web becomes ever more important, with the emergence of rich internet applications and the gradual move from the web as a passive collection of information to a place where users can create, share, communicate – and in general become contributors in a more powerful way then ever before.

Today, the iPhone and emergence of the mobile phone as the key web access device in the developing world are key factors in making the accessible web/mobile web crossover even more business-relevant. Opera lead the way in creating and providing open-source educational resources on web standards and accessibility; NVDA becomes a significant addition to the open source assistive technology arena.

Developing and rolling out WAI-ARIA and accessibility challenges posed by authoring tools are current hot topics; with the emergence of producer-consumers (prosumers), ATAG is more important than ever, whether when applying to Facebook or a corporate content management system.  So too is the challenge of supporting disabled people and in particular those experiencing age-related impairments in finding and using the most appropriate accessibility solutions for their needs.

The ongoing research into systems to allow adaptation of interfaces based on a user’s (often changing) accessibility requirements is not just an excuse to rescue the accessibility of poorly designed legacy web pages through transcoding, but actively supporting enhanced personalisation while reducing the burden on designers from trying to cater for diverse (and possibly competing) needs.

Reflection

These four stages summarise what I think has characterised the evolution of web accessibility as a topic in recent years, and also reflect the evolution of my personal approach to working in web accessibility, as a researcher and consultant based in a university in the UK. You may find it interesting to chart progress of particular countries and sectors against this timeline; you may also want to argue with me over the definitions I’ve used above!

I realise some people may not entirely agree with my fairly positive look; maybe I missed an event or two which were key in the evolution of accessibility as a design issue. I know there are many battles still to be fought and won, but a discussion of the role of a fundamentalist approach to disability rights advocacy versus a more moderate approach is one for another blog post.

Though I will say (and apologies for the military metaphors) that I have always appreciated the frontline work of organisations like the RNIB, and people like John Foliot and William Loughborough, who are not afraid to ask awkward accessibility questions, and take the initial flak that might ensue, but ultimately force accessibility to be considered in situations or organisations where previously it might be been ignored. And the rest of us follow in behind, once the resistance has been softened up.

But whatever you think about this brief 10 year biography of web accessibility, I’ll be happy to hear any feedback!

Another view: You might want to look at Brian Kelly’s Web accessibility timeline for a similar consideration of what’s happened over the years.

But first, some background. W4A was started up by a group of accessibility researchers at the University of Manchester, and was first held in 2004; founders Simon Harper and Yeliz Yesilada still do an enormous amount of work behind the scenes each year.

I’ve been lucky enough to be involved in the last three W4As in various roles. What makes W4A different from other accessibility, web standards and human computer interaction/usability conferences?

I think it’s a combination of a few things. Firstly, it’s a research conference – our presentations are all instances of where boundaries in web accessibility research are being pushed back; so there’s no ‘Accessibility 101′ or ‘Introduction to WCAG 2′. There’s a quality control process – all submissions are peer-reviewed and the best selected for inclusion. W4A is cross-disciplinary, so we receive contributions and attract attendees from academia, from industry and from the public sector, from all over the world. W4A is held alongside WWW, the largest annual web conference in the world, so we always have a great venue in an exciting city (past venues include Beijing, Edinburgh and New York).

These factors mean that we attract an audience that is highly knowledgeable about accessibility, so are ideally placed to critique what they hear, challenging or supporting the speaker as appropriate. And, as a 2 day conference, there are plenty of opportunities for interaction beyond the conference schedule.

The theme this year was Web Accessibility and Older Users; and our two excellent keynotes from Andrew Arch and Mike Paciello, along with several other talks, focused on this subject. But we also heard about a range of topics, from user-generated content accessibility to a new approach to delivering user-agent sensitive web content through innovative server-side technology. Every paper presented at the conference will be available in full from the ACM Digital Library, and presentation slides will also be on the Slidehare W4A09 event.

Rather than go into each paper in depth, here are the key messages I heard from throughout the two days:

• The overlap – and also the differences – between designing for older and for disabled people were clearly identified. We heard that older people do not necessarily want assistive technologies that may improve access to the Web but may also identify them as ‘special’ or ‘different’ or somehow.
• We mustn’t forget just how important ATAG and UAAG are to the web accessibility picture; but we can look to an example (ATutor) of where authoring tool and user agent support (through personalisation) for accessibility have driven development.
• Because of shortcomings in user agents (and user awareness of functionality available in user agents), there is developer frustration at having to do more than just meet WCAG to support accessibility – but the resultant demand for ‘more training for users’ is a very challenging request to meet in the short term.
• Objectively and accurately measuring Web accessibility remains an important strategic and administrative exercise – and hence a research topic.
• User stories allow empathy and understanding to be fostered, and realisations of the often huge gulfs between expert developers and designers, and older people unfamiliar, wary, and sceptical of web technology (but who are still – and this is important – potentially enthusiastic users).

W4A 2010 takes place on 26-27 April next year in Raleigh, North Carolina. What can you expect if you come? Well, a great two days hearing about some exciting work and also meeting with accessibility people from all over the world. If you want to present, you have three options:

1. A Technical paper – a full 8 page submission describing completed research or a new theory relating to web accessibility.
2. A Communication paper – a 4 page submission providing a description of work in progress, or other emerging work that isn’t yet complete enough to form a Technical paper.
3. The Web Accessibility Challenge – a more practical option for people who have developed a new technology or technique for promoting accessibility, and who are prepared to demonstrate it to the audience and judges.

I want to re-emphasise the importance of W4A as a cross disciplinary conference – it’s not just for academics, but for anyone who’s doing any kind of research and development with a web accessibility focus. We want views and opinions from as many different perspectives as possible, because in my view that’s what’s made W4A so successful so far.

The organising team have had some great feedback from delegates this year, and have many ideas for how to make the conference even better than this year – but more suggestions will be gratefully received. We were particularly pleased to see how well using Twitter helped us raise awareness of what was being discussed beyond the conference, and it was fantastic to see so much interest from beyond the walls of the conference room (see the #W4A09 Twitter feed)! My thoughts on making the conference experience as accessible as possible are something I’ll be blogging about separately.

To finish – some other favourite moments from my trip to Spain:

• A wonderful few days in Barcelona, including giving a web accessibility talk at the Universitat de Barcelona, the topic of which will be the subject of another blog post… I discuss in a separate post;
• Visiting the Camp Nou football stadium and immediately becoming a Barça fan;
• Travelling at 300km/h on the AVE train from Barcelona to Madrid. 690km in less than three hours!
• Catching up with old friends, putting faces to names that previously only existed as email or blog authors, and meeting up with new accessibility people at W4A;
• Chatting about football with the jamón-carving waiter at lunch during W4A, who turned out to be a Real Madrid and Liverpool supporting madrileño with a Scouse accent;
• A night of tapas (including black pudding and fried eggs!) and sangria in the buzzing area of La Latina, courtesy of Yod Samuel Martí­n García;
• A fantastic couple of hours in El Prado, being moved not only by some of the world’s most marvellous paintings but also by the sheer enthusiasm of the people there – especially the many groups of young school kids gathered round a classic Goya or Velázquez listening to the story behind the picture.

It was a great week, and I’m already looking forward to W4A 2010. Maybe see you there?

The paper critiques 5 prominent (i.e. published in prominent academic publications and subsequently cited) studies that compared different Usability Evaluation Methodologies (UEMs). It found that for each study the experimental design casts doubt over the validity of the conclusions made.

In a clear and accessible fashion, the paper:

• outlines the value of UEMs in interface design, and explains the relative merits of empirical UEMs (involving watching users interact with a system) and analytical UEMs (using some pre-defined knowledge to methodically assess the system for potential barrriers) in identifying true barriers and providing the design team with information necessary to fix them;
• reminds us that the value of experiments is in establishing causality (that X causes Y) and generality (that X will cause Y across different circumstances);
• introduces 4 measures of validity that can be applied to an experiment (from Quasi-Experimentation: Design and Analysis issues for field settings; Cook T and Campbell D, 1979);
• uses these measures to identify ‘threats to validity’ that might exist in the design of an experiment;
• treats each UEM comparison as a case study of how validity of the experiment and the results it presents can be questioned;
• offers advice for minimising threats to validity through experimental design and analysis.

What are the four measures of validity? Two concern causality, and two concern generality.

1. Causality issues:
   • statistical conclusion validity – concerning whether real differences do exist between experiment groups. Did the experiment really find differences in the results of using different UEMs? Validity may be affected by the impact of low numbers of participants; lack of appropriate statistical analysis; ‘random heterogeneity’ (or the influence of wildcard participants on results). This is explored in a post on bias in DIY usability testing.
   • internal validity – concerning whether measured differences are causal or correlational. Were these differences definitely due to using different UEMs? Or could some other factor have influenced results? Selection (of participant groups) and setting (conditions under which the experiment was carried out) can influence internal validity.
2. Generality issues:
   • Construct validity: in the words of the authors, “are the experimenters manipulating what they claim to be manipulating?” (this is causal construct validity) and “are they measuring what they claim to be measuring?” (this is effect construct validity)
   • External validity: how valid are claims that results can be generalised across different settings and persons?

There is a fifth validity issue – conclusion validity, where the conclusions are not based on the data generated by the experiment. The authors note the tendency of usability evaluators to include general ‘good advice’ amongst conclusions based on the findings of an experiment, when the data gathered cannot possibly support this advice. If it is accepted as good advice, it should be presented as such, not as the findings of the experiment.

Why is this work important? Well, given that these studies were selected as being of particularly high impact in the community, there is potential for major decisions to have been made relating to using one UEM over another, or for further research to have been conducted, based on unsafe assertions. What’s not clear to me, 11 years on, is just how big the impact has been on usable technology design of the flaws identified in these studies.

But more practically, for all of us who do usability or accessibility testing, this paper reminds us of the difference between analytical evaluation methods and empirical methods. There’s a danger that our eagerness to promote what we believe is best practice may obscure what we actually find out in empirical testing (the “guideline compliance vs designing for humans” argument in another form). Finding participants can be difficult; finding disabled participants for testing is very difficult, so while of course user involvement is still recommended in order to achieve valuable insight, presenting results with due qualifications and caveats is essential.

For a lot of people, this stuff will be nothing new – it’s basic good practice in science. But, like many people who have come into applied science from other areas, I don’t have a background in rigorous experimental design. And while designing major experiments is not something I do often, knowing how to devise and follow a process of generating new knowledge that is reliable and repeatable – such as conducting a usability testing programme of a software application or web application – is certainly wisdom worth having.

Read more:

• Damaged Merchandise – the original paper, and a rejoinder – commenting on feedback the authors received.
• Wikipedia on experimental design.