DE4-2 Codes (How to interpret your grade)

in Posts

For grading, we looked at visualizations and assigned codes for things we saw. We generally used the same codes from last year (see DE5-6 Codes). We added some new codes, and also a new notation where we assess the severity of the problem (or the benefit of the good thing).

Because of problems in how Canvas lets us put grading information into comments, you will see your feedback as a weird looking string. Each question is labeled, then a list of codes with a “comment” in parentheses, and then a vertical bar “|” to denote where we want to put a line break (but Canvas won’t let us!).

How we graded DE4-2

Evaluating Designs: Some thoughts on grading should give you a pretty good idea of how we think about evaluation. We considered which category of design each one was in, and used that to adjust our expectations. We assigned codes for specific things that we saw that were notable (good or bad - although we mainly noticed bad things - its harder to pinpoint why things are good). We also considered the written parts (including making deductions for missing the critique).

We then assigned a numerical score (on the 90=A, 85=AB scale) based on our overall opinion, and the number of codes. If there were lots of Bad codes, or many more U codes than G codes, your score was lower. There was no strict “numerical equivalent” - we looked at the whole assignment. We factored in things like not expecting the alternates to be as good as the primaries, having a diversity of designs across the 6 visualizations, and remembering to explain how you did things (by answering that question).

Some explanations of codes.

With each code, we preface it with a level (BUGS - Bad, Ugly, Good, Splendid). We use lower case levels to mean a “lower level” (so u is “slightly ugly”).

The codes are a level (BUGS) and a code. For the levels, I’ll also use ? (or little u) - for slightly ugly (less than a full ugly).

The full list of codes is at DE5-6 Codes from last year. But here are some explanations that include levels

  • B:IE - inappropriate encoding

  • B:PH - misuse of part-whole a special case of IE

  • U:Cap - needs a caption (might also be a “no clear story”)

  • U:CHTS - Comparison Hard to See (design choices actually hide important comparison)

  • u:CHTS - Comparison Hard to See (no choices that bring it out). Sometimes this is used for places where the fact that the comparison is hard may not be easy to understand.

  • u:NCS - No Clear Story - not always given. if the chart is clear but there it is still hard to find a takeaway (especially without )

  • u:Noisy - Noisy - excess data and small features make the overal picture hard to see; usually results in too much overdraw

  • u:CIB - California is Bigger - showing a statistic that is tied to population as an absolute (u for seemingly intensional)

  • S:Novel - Novel - something that we don’t see much of, but makes sense to tell the story it tells

  • u:MMC - Mis-Matched Charts - there are multiple charts without consistency which makes them difficult to interpret together

  • S:Sel - Selection - selection is a key/creative part of storytelling

  • G:Sel - Selection - very active use of selection to focus

  • g:Sel - Selection - choices made to simplify that make a design more effective; or are at least well documented

  • u:NMV - Not Multi-Variate

  • g:Details - Good Details to make a standard chart better

  • S:Details - superlative Details bring out a story

  • g:Story - Good Story - usually an ambitious story that isn’t necessarily told well by the visualization (story is hard to see, but comes out with effort)

  • S:Story - Superlative Story - really shows something interesting in the data, and brings it out through design choices

Codes for rationale

  • g:BV - Beyond Vis - the story from the rationale goes beyond what is easy to see in the visualization. The Rationale is good (has a good story).