In this module, we will try to be more “scientific” about how we approach visualizations: bringing more formality into how we evaluate visualization, examining studies to build our understanding, and exploring the cognitive and perceptual foundations. We start with evaluation: why assessing visualizations is hard, and the nested model as a way to locate where a design succeeds or fails. We look at empirical studies: how experiments produce knowledge about visualization, and how to read them critically. Then we turn to the viewer: the perceptual and cognitive foundations of visualization, color (a surprisingly complicated topic), and the cognitive biases that affect how visualizations are read and designed. Throughout, we connect the science to design practice, and students continue to create and critique visualizations.

In Module 3, we explore different ways of bringing more formalism into visualization. While these may seem like distinct topics, they all share the common thread of trying to be more rigorous and scientific in how we approach visualization.

1. Evaluation - We will explore how we validate visualizations and visualization research. This will lead to some methods for evaluation that we can apply when creating a visualization, a system, or a research idea.
2. Cognitive and Perceptual Foundations - We will introduce some of the foundations of how people perceive and use visualizations, which will provide evidence for some design decisions.
3. Empirical Studies - Running experiments with people to understand how they perform is an important strategy. We will look at some examples of experiments and think about how to assess them and use them to influence design.

Mixed in, we’ll talk about Color - which is a specific aspect of perception that has many implications for visualization.

As usual, week 1 has more readings (and a reading survey and seek and find), week 2 mixes readings and a design warmup, and week 3 is the design exercise.

Module Learning Outcomes

After completing this module, students will:

1. Understand the challenges of evaluating visualizations, be aware of the strategies, and apply the nested model to locate where a design succeeds or fails.
2. Read empirical research critically and apply its findings to visualization design and analysis.
3. Describe the human visual system and identify its implications for visualization design.
4. Explain the basics of the physical and perceptual aspects of color and apply them to visualization design.
5. Identify cognitive mechanisms that influence visualization effectiveness.
6. Be aware of cognitive biases that affect how visualizations are read and designed.

Week 1 (Module 3)

In the first week, we’ll look at evaluation strategies and the beginning of applying perceptual and cognitive science in visualization.

The main points in evaluation are to understand the complexity of it (why is evaluating visualization so hard?) and some of the key strategies. In particular, we will learn about Munzner’s Nested Model for Validation.

Week 3-1 Readings

1. backbone Tamara Munzner. Analysis. Chapter 4 from Munzner's Visualization Analysis & Design. (Canvas File) (UW Library) (Summary) - OK to read summary
2. backbone Alberto Cairo. The five qualities of great visualizations. Chapter 2 of The Truthful Art. (Canvas File) (UW Library) - Cairo’s 5 qualities of great visualizations: a way to think about evaluation to contrast the more academic works.
3. historic Card, Mackinlay, and Schneiderman. Information Visualization. The first 17 pages of the Introduction to “Information Visualization: Using Visualization to Think. (Canvas File) - Only the first 16 pages (there are lots of pictures)
4. core Steve Franconeri. Thinking with Data Visualizations, Fast and Slow. Open Vis 2018 Conference Talk. (video) (Summary)
5. modern Danielle Albers; Michael Correll; Michael Gleicher. Task-Driven Evaluation of Aggregation in Time Series Visualization. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 551–560, May 2014. (doi) (web pdf) (url)
6. optional Chris North. Visualization Viewpoints: Toward Measuring Visualization Insight. IEEE Computer Graphics & Applications, 26(3): 6-9, May/June 2006. (doi)
7. optional Gordon Kindlmann and Carlos Scheidegger. An Algebraic Process for Visualization Design. IEEE Transactions on Visualization and Computer Graphics, 20(12) 2014.. (doi) (web pdf) (Summary)
8. optional Danielle Albers Szafir; Steve Haroz; Michael Gleicher; Steven Franconeri. Four Types of Ensemble Encoding in Data Visualizations. Journal of Vision, 16(5):11, March 2016. (doi) (web pdf) (url)

This week there are 4 required readings and 1 required “watching”. This is a first week of module reading load - and it isn’t so bad because at least one the summary is OK. (working on summaries for others)

Evaluation: Reading #1 (the Munzner Nested Model book chapter) is so important and central to the class that I recommend you read my summary of it first. The summary will help you appreciate why it is so important. Read #2 Cairo to get his “non-academic view”. #6 (North’s Measuring Insight) gets at why its so hard to assess visualizations - but we’ll get the main points in lecture.

#7 is optional - I’ll mention it in class, but its a nice example of how an elegant formalism can be useful in thinking about visualizations. Reading the summary is a start.

Cognition and Perception: Reading #3 (Card et al) and Watching #4 (Franconeri video) will introduce you to the cognitive and perceptual aspects. Watch the video for #4 (the summary gets some key points, but not as well).

Reading #3 is an experiential reading: this is a foundational document in the field. They were naming and defining the field of Information Visualization, and describing the cognitive science basis it builds on. The section “How Visualization Amplifies Cognition” (starting on page 15), with Table 1.3 is particularly important. It really gets at why visualizations help us do things. (you can stop after the table on page 16) Even though this is from 1999 (and is dated in many ways) its both a great history of the field and still stands up as an introduction to the cognitive aspects of visualization.

Empirical Evaluation: Reading #5 is meant to give you a taste of empirical papers, and to introduce you to a different perceptual phenomenon that is useful in visualization: ensemble coding. The paper is a set of clean, simple experiments. It also gets at a really interesting aspect of perception: the ability of the eye to make judgements over groups of marks. If you want to learn more about ensemble coding, optional reading #8 give a survey (of everything up to 2016 - there’s a ton of new stuff).

Week 3-1 Assignments

• Reading Survey 3: Principles and Formalism (due Fri, Oct 30)
• Seek and Find 3: Color (due Fri, Oct 30)

Week 2 (Module 3)

In the second week of the module, we will focus on perceptual aspects of visualization. We’ll spend a day talking about how perceptual science impacts visualization design generally, and then some time talking about color.

Week 3-2 Readings

1. core Karen B. Schloss. Perceptual and Cognitive Foundations of Information Visualization. Annual Review of Vision Science 11:303–330 (2025). (doi)
2. backbone Colin Ware. The Dance of Meaning. Chapter 10 of Visual Thinking for Design, or Chapter 10 of the newer Visual Thinking for Information Design. (Canvas File)
3. backbone Tamara Munzner. Map Color and Other Channels. Chapter 10 from Munzner's Visualization Analysis & Design. (Canvas File) (UW Library)
4. modern Cindy Xiong; Lisanne Van Weelden; Steven Franconeri. The Curse of Knowledge in Visual Data Communication. IEEE Transactions on Visualization and Computer Graphics (Volume: 26, Issue: 10, October 2020). (doi) (web pdf)
5. optional Alberto Cairo. The Eye and Visual Brain. Chapter 5 of The Functional Art. (Canvas File) (UW Library)
6. optional Alberto Cairo. Visualizing for the Mind. Chapter 6 of The Functional Art. (Canvas File) (UW Library)
7. optional Brian Ondov; Nicole Jardine; Niklas Elmqvist; Steven Franconeri. Face to Face: Evaluating Visual Comparison. IEEE Transactions on Visualization and Computer Graphics (Volume: 25, Issue: 1, January 2019). (doi)
8. optional Brian D. Ondov; Fumeng Yang; Matthew Kay; Niklas Elmqvist; Steven Franconeri. Revealing Perceptual Proxies with Adversarial Examples. IEEE Transactions on Visualization and Computer Graphics (Volume: 27, Issue: 2, February 2021). (doi) (web pdf)
9. optional Borland, D., & Taylor, R.. Rainbow Color Map (Still) Considered Harmful. IEEE Computer Graphics and Applications, 27(2), 14–17. (doi)
10. optional Colin Ware, Maureen Stone, Danielle Albers Szafir. Rainbow Colormaps Are Not All Bad. IEEE Computer Graphics and Applications, Volume 43, Issue 3, 2023. (doi) (web pdf)
11. optional Kennedy Elliot. 39 Studies about human perception in 30 minutes. Medium Posting. (url)
12. resource Ghulam Quadri and Paul Rosen. A Survey of Perception-Based Visualization Studies by Task. IEEE Transactions on Visualization and Computer Graphics 28, 12 (December 2022). (doi) (web pdf) (url) - resource readings are optional
13. resource Evanthia Dimara, Steven Franconeri, Catherine Plaisant, Anastasia Bezerianos, Pierre Dragicevic. A Task-based Taxonomy of Cognitive Biases for Information Visualization. IEEE Transactions on Visualization and Computer Graphics (Volume: 26, Issue: 2, February 2020). (doi) (web pdf) - resource readings are optional

Yes, this is a long list - but most of them are optional. And I could give a huge list of readings on color alone…

Perception: Read #1 (Schloss) to get a thorough overview of the perceptual aspects of visualization (Karen is a Prof. in our Psych Department). Read #2 Ware to get a less formal description, that is more tied to how perception is tied to design practice. Reading #5 and #6 (Cairo) is interesting for his non-scientific (designer’s intuition) take on perception and cognition.

Empirical Studies: #4 (Xiong et al.) is a good example of a modern emprical study paper. Read it to get a sense of how experiments are used to support arguments. It will also give you an introduction to cognitive biases. Reading #11 (optional) will give you a quick, informal tour through a number of key emprical results. Reading #13 (Quadri&Rosen, optional) will give you a survey of many of the studies about perception in visualization - it’s a well organized library, keep it in mind when you want to know what is out there. #7 and #8 are two other optional examples of empirical papers.

Color: I don’t even know where to begin. I love the topic, and it is suprisingly complicated, and there are many different angles. Read #3 (Munzner VAD Ch 10) to get a quick introduction. #9 and #10 talk about rainbow color maps - which is a good example of how many of the color perception issues connect.

Week 3-2 Assignments

• Content Survey 3: Principles and Formalism (due Sat, Nov 07)
• Design Warmup 3: Working with Census Data (due Sat, Nov 07)

Week 3 (Module 3)

For the last week of the module, you will focus on Design Exercise 3. Class time will build on concepts from prior weeks, and expose you to more visualization research.

Week 3-3 Assignments

• Anonymous Survey 3: Check In (due Sat, Nov 14)
• Design Exercise 3: Census (aggregations) (due Fri, Nov 13)