[论文解读] Automated Usability Testing: Analysing Asia Web Sites

A. Rukshan, Andres Baravalle|arXiv (Cornell University)|Dec 9, 2012

Usability and User Interface Design参考文献 19被引用 12

一句话总结

本研究提出AWebHUT，一种基于PHP和MySQL的自动化工具，通过分析XHTML源代码与17项可用性准则的符合度，评估网页可用性。该工具评估了45,126个亚洲网站主页，发现普遍存在可用性问题，尤其在“天气”类别中平均违规率达42.28%，凸显了通过自动化评估提升亚洲网页可用性以达到全球标准的迫切需求。

ABSTRACT

Web usability is continuing to be a pressing problem. For number of years researchers have been developed tools for doing automatic web usability testing. This study uses our own PHP, and MySQL based tool AWebHUT: Automated Web Homepage Usability Tester to evaluate web usability of full Dmoz (www.dmoz.org) Asia web sites (45126 on time stamp 2011-12-03 04:12:46 GMT). The tool uses an extensive automated quantitative analysis of XHTML source code of homepages against seventeen organised web usability guidelines. The automated quantitative approach is effective on large scale to achieve better usability. The AWebHUT uses four web usability levels such as N: Neutral, V: Violate, R: Respect, and E: Error to evaluate web usability. The main objective of the study is to produce data which is used to answer research questions, (1) Are there any categories of web sites which have usability problems? Which ones? and (2) Are there any categories in which the usability is typically higher? Why? The findings were indicated that all Asia categories have usability problems. Furthermore, there are four web sites which have highest web usability problem with violation percentage 71. One step further, the Asia category: Weather has highest usability problems with 42.2819 as the average of the violation percentage. The category Weather uses tables and images, considerable amount of those were not satisfying web usability guidelines which relates to tables and images. One step further, the Asia wants to get the same level of usability as North America, Europe, and Australia therefore it is essential to have an automated web usability evaluation in Asia web sites to identify web usability problems which are important for improving Asia web sites.

研究动机与目标

• 通过自动化分析，大规模识别亚洲网站的可用性问题。
• 确定亚洲网站中哪些类别存在最严重的可用性问题。
• 比较亚洲各类网站的可用性水平，识别出可用性始终较高的类别。
• 确立针对亚洲网站定制的自动化可用性评估工具的必要性，以缩小与北美、欧洲和澳大利亚之间的差距。
• 为亚洲网站主页在结构和内容相关可用性违规方面提供数据驱动的洞察。

提出的方法

• AWebHUT使用基于规则的系统处理网页主页的XHTML源代码，依据17项标准化网页可用性准则进行分析。
• 该工具为每项准则检查分配四个可用性等级之一：N（中性）、V（违规）、R（遵守）、E（错误）。
• 自动化定量分析评估表格、图像和语义标记等结构元素是否符合可用性最佳实践。
• 分析应用于从DMOZ亚洲目录中收集的45,126个网页主页，数据采集时间为特定时间戳（2011-12-03 04:12:46 GMT）。
• 通过汇总每个类别中所有准则和网站的V与E评分，计算各分类的违规率百分比。
• 通过类别间的统计比较识别相对可用性表现，结果显示‘天气’类别问题最为严重。

实验结果

研究问题

• RQ1 是否存在某些特定类别的亚洲网站表现出显著的可用性问题？
• RQ2 亚洲哪些网站类别展现出更高的可用性水平，其表现优异的因素是什么？
• RQ3 亚洲网站中表格和图像等结构元素在多大程度上违反了既定的可用性准则？
• RQ4 亚洲网站的可用性水平与北美、欧洲和澳大利亚网站相比如何？
• RQ5 自动化评估工具能否在大规模区域网站群体中有效识别并量化可用性问题？

主要发现

• 所分析的18个亚洲网站类别均存在可用性问题，表明该地区普遍存在系统性问题。
• ‘天气’类别平均违规率最高，达42.2819，为最严重的问题类别。
• 共发现4个网站的违规率高达71%，表明存在极端的可用性缺陷。
• ‘天气’类别违规的主要原因是表格和图像的不当使用，未能满足可访问性和可用性标准。
• 研究证实，亚洲网站整体上未达到北美、欧洲和澳大利亚所见的可用性基准。
• 研究结果强调了迫切需要自动化可用性评估工具，以系统性地识别并修复亚洲网站的可用性问题。