Background Literature on Evaluation for Information Visualization

Enrico Bertini
bertini@dis.uniroma1.it
(March, 2006)


This is an initial collection of papers that contain studies related to the evaluation of visualizations. Its main purpose is to provide a reference list for people interested in the topic and in particular as support material for the AVI2006 workshop BELIV'06 (BEyond time and errors: novel evaLuation methods for Information Visualization). The papers are organized in the following classes:
Any idea or comment to improve this material is more than  welcome.


Component/system level evaluation

  Kang Shi, Pourang Irani, and Ben Li. An evaluation of content browsing techniques for hierarchical space-filling visualizations. In INFOVIS '05: Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization, page 11, Washington, DC, USA, 2005. IEEE Computer Society.
[ bib | http ]
  Alfred Kobsa. User experiments with tree visualization systems. In INFOVIS '04: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), pages 9-16, Washington, DC, USA, 2004. IEEE Computer Society.
[ bib | http ]
  Mohammad Ghoniem, Jean-Daniel Fekete, and Philippe Castagliola. A comparison of the readability of graphs using node-link and matrix-based representations. In INFOVIS '04: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), pages 17-24, Washington, DC, USA, 2004. IEEE Computer Society.
[ bib | http ]
  Michael Granitzer, Wolfgang Kienreich, Vedran Sabol, Keith Andrews, and Werner Klieber. Evaluating a system for interactive exploration of large, hierarchically structured document repositories. In INFOVIS '04: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), pages 127-134, Washington, DC, USA, 2004. IEEE Computer Society.
[ bib | http ]
  Chaomei Chen and Steven Morris. Visualizing evolving networks: Minimum spanning trees versus pathfinder networks. In INFOVIS '03: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'03), 2003.
[ bib | http ]
  Qing Li and Chris North. Empirical comparison of dynamic query sliders and brushing histograms. In INFOVIS '03: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'03), 2003.
[ bib | http ]
  Kenneth L. Summers, Timothy E. Goldsmith, Steve Kubica, and Thomas P. Caudell. An experimental evaluation of continuous semantic zooming in program visualization. In INFOVIS '03: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'03), 2003.
[ bib | http ]
  Catherine Plaisant, Jesse Grosjean, and Benjamin B. Bederson. Spacetree: Supporting exploration in large node link tree, design evolution and empirical evaluation. In INFOVIS '02: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'02), page 57, 2002.
[ bib | http ]
  Alfred Kobsa. An empirical comparison of three commercial information visualization systems. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 123, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | www: ]
  Todd Barlow and Padraic Neville. A comparison of 2-d visualizations of hierarchies. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 131, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | http ]
  Kerry Rodden, Wojciech Basalaj, David Sinclair, and Kenneth Wood. Evaluating a visualization of image similarity as a tool for image browsing. In INFOVIS '99: Proceedings of the 1999 IEEE Symposium on Information Visualization, page 36, Washington, DC, USA, 1999. IEEE Computer Society.
[ bib | http ]
  George Robertson, Mary Czerwinski, Kevin Larson, Daniel C. Robbins, David Thiel, and Maarten van Dantzich. Data mountain: using spatial memory for document management. In UIST '98: Proceedings of the 11th annual ACM symposium on User interface software and technology, pages 153-162, New York, NY, USA, 1998. ACM Press.
[ bib | http ]
  Chris North and Ben Shneiderman. Snap-together visualization: can users construct and operate coordinated visualizations? Int. J. Hum.-Comput. Stud., 53(5):715-739, 2000.
[ bib | http ]
  Mary Czerwinski, Susan Dumais, George Robertson, Susan Dziadosz, Scott Tiernan, and Maarten van Dantzich. Visualizing implicit queries for information management and retrieval. In CHI '99: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 560-567, New York, NY, USA, 1999. ACM Press.
[ bib | http ]
  Kirsten Risden, Mary P. Czerwinski, Tamara Munzner, and Daniel B. Cook. An initial examination of ease of use for 2d and 3d information visualizations of web content. Int. J. Hum.-Comput. Stud., 53(5):695-714, 2000.
[ bib | http ]
  Peter Pirolli, Stuart K. Card, and Mija M. Van Der Wege. The effect of information scent on searching information: visualizations of large tree structures. In AVI '00: Proceedings of the working conference on Advanced visual interfaces, pages 161-172, New York, NY, USA, 2000. ACM Press.
[ bib | http ]
  Doug Schaffer, Zhengping Zuo, Saul Greenberg, Lyn Bartram, John Dill, Shelli Dubs, and Mark Roseman. Navigating hierarchically clustered networks through fisheye and full-zoom methods. ACM Trans. Comput.-Hum. Interact., 3(2):162-188, 1996.
[ bib | http ]


Low level components/preceptual studies

  Sunghee Kim, Haleh Hagh-Shenas, and Victoria Interrante. Conveying shape with texture: an experimental investigation of the impact of texture type on shape categorization judgments. In INFOVIS '03: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'03), 2003.
[ bib | http ]
  Lucy Nowell, Robert Schulman, and Deborah Hix. Graphical encoding for information visualization: An empirical study. In INFOVIS '02: Proceedings of the IEEE Symposium on Information Visualization (InfoVis'02), page 43, Washington, DC, USA, 2002. IEEE Computer Society.
[ bib | http ]
  Mats Lind, Geoffrey P. Bingham, and Camilla Forsell. The illusion of perceived metric 3d structure. In INFOVIS '02: Proceedings of the IEEE Symposium on Information Visualization (InfoVis'02), page 51, Washington, DC, USA, 2002. IEEE Computer Society.
[ bib | http ]
  Lucy Nowell, Elizabeth Hetzler, and Ted Tanasse. Change blindness in information visualization: A case study. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 15, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | http ]
  Monica Tavanti and Mats Lind. 2d vs 3d, implications on spatial memory. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 139, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | http ]
  Benjamin B. Bederson and Angela Boltman. Does animation help users build mental maps of spatial information? In INFOVIS '99: Proceedings of the 1999 IEEE Symposium on Information Visualization, page 28, Washington, DC, USA, 1999. IEEE Computer Society.
[ bib | http ]
  Christopher G. Healey, Kellogg S. Booth, and James T. Enns. High-speed visual estimation using preattentive processing. ACM Trans. Comput.-Hum. Interact., 3(2):107-135, 1996.
[ bib | http ]
  Bernice E. Rogowitz, Lloyd A. Treinish, and Steve Bryson. How not to lie with visualization. Comput. Phys., 10(3):268-273, 1996.
[ bib | http ]
  Gloria Mark, Alfred Kobsa, and Víctor M. González. Do four eyes see better than two? collaborative versus individual discovery in data visualization systems. In IV, pages 249-, 2002.
[ bib | http ]

Longitudinal studies, case studies

  Victor Gonzalez and Alfred Kobsa. Benefits of information visualization systems for administrative data analysts. In IV '03: Proceedings of the Seventh International Conference on Information Visualization, page 331, Washington, DC, USA, 2003. IEEE Computer Society.
[ bib | http ]
  Martin Wattenberg. Baby names, visualization, and social data analysis. In INFOVIS '05: Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization, page 1, Washington, DC, USA, 2005. IEEE Computer Society.
[ bib | http ]
  Kresimir Matkovic, Helwig Hauser, Reinhard Sainitzer, and M. Eduard Groller. Process visualization with levels of detail. In INFOVIS '02: Proceedings of the IEEE Symposium on Information Visualization (InfoVis'02), page 67, Washington, DC, USA, 2002. IEEE Computer Society.
[ bib | http ]
  Nina Amenta and Jeff Klingner. Case study: Visualizing sets of evolutionary trees. In INFOVIS '02: Proceedings of the IEEE Symposium on Information Visualization (InfoVis'02), page 71, Washington, DC, USA, 2002. IEEE Computer Society.
[ bib | http ]
  Mei Chuah. Demystifying venture capital investing. In INFOVIS '02: Proceedings of the IEEE Symposium on Information Visualization (InfoVis'02), page 161, Washington, DC, USA, 2002. IEEE Computer Society.
[ bib | http ]
  T. Alan Keahey and Stephen G. Eick. Visual path analysis. In INFOVIS '02: Proceedings of the IEEE Symposium on Information Visualization (InfoVis'02), page 165, Washington, DC, USA, 2002. IEEE Computer Society.
[ bib | http ]
  Todd Barlow and Padraic Neville. Case study: Visualization for decision tree analysis in data mining. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 149, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | http ]
  Jeffrey Brainerd and Barry Becker. Case study: E-commerce clickstream visualization. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 153, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | http ]
  Robert M. Edsall, Alan M. MacEachren, and Linda Pickle. Case study: Design and assessment of an enhanced geographic information system for exploration of multivariate health statistics. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 159, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | http ]
  James Agutter, Noah Syroid, Frank Drews, Dwayne Westenskow, Julio Bermudez, and David Strayer. Graphic data display for cardiovascular system case study. In INFOVIS '01: Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), page 163, Washington, DC, USA, 2001. IEEE Computer Society.
[ bib | http ]


Metrics, benchmarks, model-based evaluation, frameworks, taxonomies

  Robert Amar, James Eagan, and John Stasko. Low-level components of analytic activity in information visualization. In INFOVIS '05: Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization, page 15, Washington, DC, USA, 2005. IEEE Computer Society.
[ bib | http ]
  Robert A. Amar and John T. Stasko. A knowledge task-based framework for design and evaluation of information visualizations. In INFOVIS '04: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), pages 143-150, 2004.
[ bib | http ]
  Melanie Tory and Torsten Moller. Rethinking visualization: A high-level taxonomy. In INFOVIS '04: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), pages 151-158, Washington, DC, USA, 2004. IEEE Computer Society.
[ bib | http ]
  Ed H. Chi. A taxonomy of visualization techniques using the data state reference model. In INFOVIS '00: Proceedings of the IEEE Symposium on Information Vizualization 2000, page 69, Washington, DC, USA, 2000. IEEE Computer Society.
[ bib | .pdf ]
  Issei Fujishiro, Rika Furuhata, Yoshihiko Ichikawa, and Yuriko Takeshima. Gadget/iv: A taxonomic approach to semi-automatic design of information visualization applications using modular visualization environment. In INFOVIS '00: Proceedings of the IEEE Symposium on Information Vizualization 2000, page 77, Washington, DC, USA, 2000. IEEE Computer Society.
[ bib | http ]
  Julie Yang-Peláez and Woodie C. Flowers. Information content measures of visual displays. In INFOVIS '00: Proceedings of the IEEE Symposium on Information Vizualization 2000, page 99, Washington, DC, USA, 2000. IEEE Computer Society.
[ bib | http ]
  S. K. Card and J. Mackinlay. The structure of the information visualization design space. In INFOVIS '97: Proceedings of the 1997 IEEE Symposium on Information Visualization (InfoVis '97), page 92, Washington, DC, USA, 1997. IEEE Computer Society.
[ bib | http ]
  R. Brath. Metrics for effective information visualization. In INFOVIS '97: Proceedings of the 1997 IEEE Symposium on Information Visualization (InfoVis '97), page 108, Washington, DC, USA, 1997. IEEE Computer Society.
[ bib | http ]
  Nancy Miller, Beth Hetzler, Grant Nakamura, and Paul Whitney. The need for metrics in visual information analysis. In NPIV '97: Proceedings of the 1997 workshop on New paradigms in information visualization and manipulation, pages 24-28, New York, NY, USA, 1997. ACM Press.
[ bib | http ]
  Ben Shneiderman. The eyes have it: A task by data type taxonomy for information visualizations. In VL '96: Proceedings of the 1996 IEEE Symposium on Visual Languages, page 336, Washington, DC, USA, 1996. IEEE Computer Society.
[ bib | http ]
  Jock Mackinlay. Automating the design of graphical presentations of relational information. ACM Trans. Graph., 5(2):110-141, 1986.
[ bib | http ]
  G. G. Grinstein, P. Hoffman, S. J. Laskowski, and R. M. Pickett. Visualization in Data Mining and Knowledge Discovery, chapter Benchmark Development for the Evaluation of Visualization for Data Mining, pages 129-176. Morgan Kaufmann Publishers Inc., 2001.
[ bib | http ]


Novel evaluation methodologies, non-conventional methods/parameters

  Kenneth Allendoerfer, Serge Aluker, Gulshan Panjwani, Jason Proctor, David Sturtz, Mirjana Vukovic, and Chaomei Chen. Adapting the cognitive walkthrough method to assess the usability of a knowledge domain visualization. In INFOVIS '05: Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization, page 26, Washington, DC, USA, 2005. IEEE Computer Society.
[ bib | http ]
  Purvi Saraiya, Chris North, and Karen Duca. An evaluation of microarray visualization tools for biological insight. In INFOVIS '04: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), pages 1-8, Washington, DC, USA, 2004. IEEE Computer Society.
[ bib | http ]
  Tobias Skog, Sara Ljungblad, and Lars Erik Holmquist. Between aesthetics and utility: Designing ambient information visualizations. In INFOVIS '03: Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'03), 2003.
[ bib | http ]
  Monika Buscher, Dan Shapiro, Michael Christensen, Preben Mogensen, and Peter Orbaek. Creativity, complexity, and precision: Information visualization for (landscape) architecture. In INFOVIS '00: Proceedings of the IEEE Symposium on Information Vizualization 2000, page 167, Washington, DC, USA, 2000. IEEE Computer Society.
[ bib | http ]


Reviews

  Catherine Plaisant. The challenge of information visualization evaluation. In AVI '04: Proceedings of the working conference on Advanced visual interfaces, pages 109-116, New York, NY, USA, 2004. ACM Press.
[ bib | http ]
  Chaomei Chen and Yue Yu. Empirical studies of information visualization: a meta-analysis. Int. J. Hum.-Comput. Stud, 53(5):851-866, 2000.
[ bib | http ]

This file has been generated by bibtex2html 1.79