Due date: 11:59pm, Monday, February 9th.
Turn-in link: Discussion on Canvas
Last time, we saw that there are wide range of designs available, and that sometimes these designs can address tasks (that is, solve problems). Sometimes, they don’t.
Which brings up the question: how do we know that a design (or implementation, or visualization) is any good? We can make lots of things – which one should we make?
Evaluation is often an afterthought. One of the big concepts I’d like to impress upon you (and I think this is a theme in Munzner’s book) is that it really should be central to our thinking.
So, to get you thinking on evaluation, I want you to read a few different perspectives on it:
- Chapter 4 of Munzner.
The main ideas here I like a lot. They come from an earlier paper, that I think was an important milestone in the field. The chapter is similar enough to the paper that reading the paper is a little redundant (if you want to see it, check here). For 838 students, I will ask you to read the paper too (in a different assignment) since I think its an important case of “read the original sources.” - Edward Tufte. The Fundamental Principles of Analytical Design. in Beautiful Evidence. (in the protected reader).
Of course, we can’t talk about “what is good” without consulting Tufte for his strong opinions. (not that he isn’t going to make his opinions clear). In hindsight, this Tufte chapter is actually much better in the “how” to make a good visualization, and trying to distill the general principles, than many of the others we’ve read. But its Tufte, so its still full of his opinions on “what is good.” - Bateman, S., Mandryk, R.L., Gutwin, C., Genest, A.M., McDine, D., Brooks, C. 2010. Useful Junk? The Effects of Visual Embellishment on Comprehension and Memorability of Charts. In ACM Conference on Human Factors in Computing Systems (CHI 2010), Atlanta, GA, USA. 2573-2582. Best paper award. DOI=10.1145/1753326.1753716. (get paper at the project page here)
This is a pretty provacative paper. You can pick apart the details, but I think the main ideas are important. There is a ton written about this paper (those of the Tufte religon view this as blasphemy). Stephen Few has a very coherent discussion of it here. In some sense, I’d say it’s more useful than the original paper – but I would really suggest you look at the original first. While more level-headed than most, Few still has an Tufte-ist agenda.Reading the Few article is not optional – in some ways, its more interesting than the original.(3b) optional:
Michelle Borkin et. al. What Makes a Visualization Memorable? [pdf] InfoVis 2013.
This is a less radical thought of “maybe Tufte-ism isn’t all there is – and we can measure it.” Again, we can quibble with the details, but they really re getting at something real here. - Chris North, “Visualization Viewpoints: Toward Measuring Visualization Insight”, IEEE Computer Graphics & Applications, 26(3): 6-9, May/June 2006. [pdf] (doi)
I think this is an important paper (well, it’s a magazine article that is a lightweight version of a paper) because it gets at the challenge of evaluation at the higher levels. Reading the original paper (which details their experiment) isn’t necessary for getting this point – but it does show of how hard these kinds of experiments are. (I will ask 838 students to read this paper in a separate assignment).
For the reading discussion…
In the past, these readings have really got people thinking and discussing. So I expect that you will have interesting conversations without much prompting.
However, since we have to keep score: in your initial posting for the discussion, describe how Tufte and North’s views fit into Munzner’s framework. Then give your thoughts on evaluation in general. Comment on where you stand in the chartjunk debate.
If you want some food for thought on the chartjunk debate, here are some of my key thoughts (not to bias you too much):
- There is a difference between junk and embellishment.
- Minimalism is a particular aesthetic choice / style. Baroque-ness is another. Any style can be done well or badly, which is often orthogonal to whether it is an appropriate style. Some styles are harder to pull off than others.
- Task matters. (couldn’t you have guessed I would have said that?)
