Design Exercise Rubrics 04,05,06,07

in Posts
Design Exercise

This page has the rubrics (evaluation criteria) for Design Exercises 4-7.

We intentionally didn’t tell you too much about how we were grading design exercises 3-5: we had pretty low standards, since we are aware that you haven’t learned what you needed to make great visualization. But for DE6, we are expecting great visualizations, and want to make clear how we are looking for it.

Design Exercises 4 and 5

For Design Exercises 4 and 5, the visualizations are graded as a series of check/no check criteria. With these exercises, we count the number of checks, and then there is a mapping from number of checks to grade.

Note: we are using the codes (see DE5-6 Codes: Feedback) to provide some detailed feedback. Reading over the codes can help you appreciate the kinds of things we look for in visualizations. The specific feedback that is returned to you will include some of the codes we thought were applicable (but possibly not all of them).

DE4: “Rubric” for visualizations - basically we wanted you to make something, little expectation of something “good”. For each chart there are 3 different checkboxes (and one for the “writeup” which is often shared between 2 charts) - each image gets 0-3 points:

  • is it a correct chart (with the required type)?
  • is is a valid chart (not too many/severe “invalidities”)?
  • does it attempt to address the question?
  • is there a reasonable answer to the associated question (1pt shared between images)

DE5: - we aren’t necessarily expecting great visualizations (but we appreciate the ones we got!) - excellence will be rewarded in DE6. Each image gets 0-8 points. (each is a binary check):

  • is there a question
  • does the visualization plausibly address the question
  • is it a “valid” visualization (no invalid encodings, unreasonable chart types) - usually connected with a IE/IA code
  • Not “really bad” (DD or NMV codes, or bad overall design decision)
  • title/labels/legend/caption (at least some effort)
  • some choices in the details to make the story clear (preferably obvious, but might be from rationale)
  • acceptable rationale/description (has basics)
  • good rationale (complete, thoughtful)

For the overall DE5 assignment the “final visualizations and rationales” were only 2 of 5 parts. For the other 3 parts (question, initial vis, initial observation), we give 2 points each. We just gave these points (2 visualizations * 6pts) to everyone. The final scores are out of 28.

Design Exercise 6

Here we are looking for good visualizations that tell stories of the form the assignment encourages you to tell.

For DE6, we are looking for good visualizations that tell the story. (see the “criteria” section of the assignment description) The story should emerge from the visualization (with its title, caption, and design choices) - we shouldn’t need to look at the rationale to know what the visualization is trying to convey (but its there just in case). Unlike DE4/5 these aren’t just checkoffs - but more categories we will consider. For DE6, it is possible to make up for a deficient category with excellence in some other question. We will score each category on an A-F (4pt) scale, possibly adjusting upwards.

  • Question: should be clearly stated, compelling, and sufficiently complex (multi-variate); good questions can be answered with good visualizations (this is a weird definition of good, unique to this assignment). Clarity of the question matters. If we have to guess (or read your rationale) to understand the intent of the visualization, you are unlikely to get full points in this category.
  • Answer: does the visualization effectively “answer” the question - does it make the desired things easy to see. This is a holistic assessment of the impression. This is a subjective question of effectiveness. This category does overlap with the others. For example, a poorly chosen design or missing details might make a visualization less effective overall.
  • Design: does the “basic” design choice (e.g., chart type/encoding/layout) support the goals of the visualization. This includes not making “invalid” choices (although, an invalid chart may be ineffective). It also includes scalability and focusing strategies.
  • Details: this includes the details of the design, presentation, and embellishments. Design details include colors. Presentation details include proper formatting such as not showing the UI of the system used to make the visualization. Embellishments include the title, caption, labels, and legends.
  • Rationale: this is a score on what you wrote. Ideally, the visualization speaks for itself - the best assignments can get an A in the other categories before we look at the rationale. But we will grade the rationale as its own thing, and may take it into account in grading other aspects.

For DE6, you will get a grade for each visualization (all 4), each the average of its parts (the 5 components above). You will also get a grade for the “overall package”. This is where we will reward diversity of designs and questions (see the assignment).