Eval Exercise
Understanding the nested model is really important. So, we’ll have an exercise that will force you to use it and think about how it applies to DC3: The Tree of Stuff (which you should be working on).
Background
Make sure you’ve done the readings on evaluation ( Readings 13: Evaluation). You might want to wait until after the Wednesday lecture where I will talk about the nested model. It might even help to look at the paper (the book is probably good enough). Figure 2 from the paper is so helpful for this assignment that I will reproduce it here (it was also in the slides). (it is Figure 4.5 in the book)
- Munzner, T. (2009). A Nested Model for Visualization Design and Validation. IEEE Transactions on Visualization and Computer Graphics, 15(6), 921–928. (pdf) (doi)
The Idea
The nested model for evaluation and design gives four levels to think about visualization at:
- domain situation
- data/task abstraction
- encoding/interaction techniques
- algorithm
Note: I prefer to think of “algorithm” as “detail”, and call “encoding/interaction idiom” “design”. You can choose my levels (Task, Data/Resources, Design, Details/Implementation) or Munzner’s (they are very similar). I think hers are a little too specific, but they are better adjusted for validation.
In fact, you hopefully recognize the whole thing from my “recipe for visualization” ( 1: What Is Visualization and How do We Do It?) where I even said this is where it comes from.
In a sense, the nested model shouldn’t be new to you: it is so ingrained in the way that I think that it should have come across throughout the class.
What might be new, however, is to see how the nested model serves for validation (which is the point of the original paper - the fact that it is so useful for design/development came out afterwards).
One point the paper makes is the concept of “upstream/downstream threats.” That is, there are things that can go wrong at each level. Some of them can be thought about before you descend to the next level (i.e. upstream), some of them only come out after you’ve completed the next level.
In the paper/book, the focus is on the evaluation technique for each level and upstream/downstream - but you can also think of them as different problems.
Overall, there are 8 places where something can go wrong (4 levels, upstream and downstream), and 8 places you can perform validation.
The Assignment
Think about Design Challenge 3.
For each of the 8 “places” describe: a specific example of a “threat” that you might encounter, and what you might do to perform validation that you don’t have that problem.
Note: part of the lesson here is that you can do a lot of validation before you ever implement anything.
For the Nested Model Exercise (due Mon, Nov 30), there will be a Canvas “quiz” where you answer 8 questions: for each “place” (level x up/down) describe a specific threat for DC3, and how you might test it.
A warning: Canvas will allow you to “re-take” the quiz, but you might lose your prior answers. I strongly recommend you write the answers off-line and copy-paste them into Canvas.
If you are doing the alternate DC3, you may either do this for your project, or for the regular one.
One thing I hope you get out of this exercise is to realize all the potential for getting a design right before you do any implementation. For your DC3 this may give you ideas on how to convince us your project is good, even if you aren’t able to make a great implementation.
Grading
We will grade this and count it as part of your participation grade.