Design Exercise 1: Sketches and Questions: Grading Information

in Posts

This page explains the feedback for Design Exercise 1: Sketches and Questions. Your feedback is a comment on your Canvas submission. For each item it gives the assessment, any codes, and comments, then an overall comment, the score, and the grade.

See Design Exercise 1 Grade Explanation for an explanation of the unusual process for DE1 grading.

This page is automatically generated from the data used by the grading tool, and then hand-tweaked.

Items

Some items cover several questions in the Canvas quiz; they are numbered Item 1, Item 2, and so on.

Item 1: Two Answers: Urban vs. Rural - show two data scenarios

  • Question 1: Upload your first visualization of unemployment vs. urbanness
  • Question 2: Upload your second visualization of unemployment vs. urbanness
  • Question 3: (a) Explain your designs in terms of how the variables are encoded. (a few sentences), an…

Also rated for: Does the design correctly display the data?, Does the data actually have the differences?.

Item 2: Migration: Two Designs with Similar Data - different stories

  • Question 4: First design for why people migrate
  • Question 5: Second design for why people migrate
  • Question 6: Describe your two designs and what you can see in each. How do the designs differ to make…

Also rated for: Do the designs correctly display the data?, Are there different things to see?.

Item 3: Biggest improvements - two designs

  • Question 7: Visualization 1 about biggest life expectancy changes
  • Question 8: Visualization 2 about biggest life expectancy changes
  • Question 9: Describe your designs, and explain what each one makes easy to see (the story it is desig…

Also rated for: Is Vis1 a valid design (show data properly)?, Is there a story to Vis1 (question/answer)?, Is Vis2 a valid design (show data properly)?, Is there a story to Vis2 (question/answer)?.

Item 4: Questions about Life Expectance and One Picture

  • Question 10: 5 Questions about Life Expectance Progress
  • Question 11: Pick one of those questions and sketch a visualization that “answers” it.
  • Question 12: Explain the encodings of your visualization, and how it makes the answer to the question …

Also rated for: 5 Questions, Does the visualization answer a question?.

Item 5: Differences in Lifespan - 2 versions

  • Question 13: First lifespan differences visualization
  • Question 14: Second lifespan differences visualization
  • Question 15: Describe what each of those visualizations makes easy to see.

Also rated for: Is Vis1 a valid design (show data properly)?, Is there a story to Vis1 (question/answer)?, Is Vis2 a valid design (show data properly)?, Is there a story to Vis2 (question/answer)?.

Item 6: Re-Submission

  • Question 16: Is this a re-submission of an assignment that was previously submitted on time.
  • Question 17: Briefly describe the “feedback” that you received and explain how you have changed your d…

Assessment scale

Each item is given one of these levels:

  • 0: Missing
  • 1: Poor
  • 2: Adequate
  • 3: Good
  • 4: Excellent
  • 5: Superlative

Criteria

Some items are also classified by these criteria:

Does the design correctly display the data?

  • NSD: not clearly the same design
  • NSDG: two different (reasonable) designs (prompt wanted one)
  • NE: not effective for the data/scenario
  • NC: not effective for the comparison
  • ACC: acceptable design
  • SUP: notable/superlative design

Does the data actually have the differences?

  • ND: design makes it too hard to tell
  • NO: no visible differences
  • YES: visible differences

Do the designs correctly display the data?

  • NO: neither design acceptable
  • 1: only one design
  • ND: not different
  • ACC: acceptable designs
  • SUP: notable/superlative designs

Are there different things to see?

  • ND: design makes it too hard to tell
  • NO: no visible differences
  • YES: visible differences
  • RATIONALE: requires rationale

Is Vis1 a valid design (show data properly)?

  • NO: unacceptable design
  • NA: unable to tell (presentation)
  • ND: not different
  • ACC: acceptable design
  • SUP: notable/superlative design

Is there a story to Vis1 (question/answer)?

  • ND: design makes it too hard to tell
  • NO: no real story
  • YES: reasonable story
  • SUP: superlative story
  • RATIONALE: requires rationale

Is Vis2 a valid design (show data properly)?

  • NO: unacceptable design
  • NA: unable to tell (presentation)
  • ND: not different
  • ACC: acceptable design
  • SUP: notable/superlative design

Is there a story to Vis2 (question/answer)?

  • ND: design makes it too hard to tell
  • NO: no real story
  • YES: reasonable story
  • SUP: superlative story
  • RATIONALE: requires rationale

5 Questions

  • NO: didn’t provide adequate questions
  • NQ: unclear if most could be answered with data/visualization
  • ACC: acceptable questions
  • SUP: superlative questions

Does the visualization answer a question?

  • NV: Visualization not clear enough to determine if it answers the question
  • NO: Visualization does not effectively answer a question
  • YES: Visualization clearly answers a question
  • SUP: superlative vis effectiveness
  • RATIONALE: requires rationale

Codes

Codes are short labels for things we noticed, so the same observation gets the same name for everyone. A code is written like G:AD: the letter before the colon is its valence (B = Bad, U = Ugly, N = Neutral, G = Good, S = Superlative). The letter shows how the code applies to your answer, which can differ from the code’s usual valence listed here. A code written without a letter, like :AD, has its usual valence.

Bad

  • IE Inappropriate Encoding
  • NORAT No rationale: The rationale is missing or does not describe the designs.
  • PARTWH Part-whole misuse: A part-whole design used for something that isn’t part-whole, or that throws away the variation asked about.

Ugly

  • CALISBIG Counts not rates: Absolute values mostly reflect overall size rather than the point at hand.
  • CHTS Comparison hard to see: The key comparison is hard to make with this design.
  • CLAIM Unsupported claim: The rationale claims something is easy to see that the design does not actually support.
  • IA Inappropriate aggregation: Poor aggregation choice (e.g. averaging county rates instead of summing or population weighting).
  • LABELS Labels missing: Needed labels (axes, series, which end is which) are missing.
  • MISMATCH Sketch and rationale disagree: The rationale describes something the sketch does not show, or the sketches are inconsistent with each other.
  • NCS No Clear Story
  • ND Designs not different: The two designs are essentially the same and do not emphasize different aspects.
  • NEEDRAT Needs rationale: The design can only be understood with an explanation the rationale does not adequately give.
  • NMV Not enough data: Too few items or variables to show the pattern asked about.
  • NOISY Noisy: Clutter, overdraw or decoration obscures the picture.
  • NOQ Question not identified: The Q12 rationale does not say which question the sketch answers.
  • NSDG Two designs where one was asked: Q1-3: the two scenarios should be shown in the same design; two different designs miss the point of the exercise.
  • PARTIAL Partial answer: Does not show the whole story asked for (e.g. change but not final level; regions instead of countries; a part missing).
  • POORX Poor execution: Presentation is too unclear to read the intended design.
  • Q10FMT Question format: Q10 is not five one-sentence questions about trends in progress.
  • RAMP Color ramp: Color ramp doesn’t fit the data (e.g. rainbow for a quantitative value).
  • WEAKRAT Weak rationale: Substandard or incomplete rationale (encodings not named, no explanation of why the answer is easy to see).
  • WHYSEL Selection unexplained: Data selection is arbitrary or unexplained (e.g. one state, a handful of counties).

Neutral

  • CBE Could be better: Not wrong, but a better encoding would help.
  • MINIMAL Minimal: Meets the requirements, not much more.
  • WEIRD Unusual: Not immediately recognizable; may be an adaptation or novel.

Good

  • AD Adapted Design: Uses a standard design, but in a non-standard way.
  • ANNOTE Annotation for storytelling: Callouts, reference/trend lines, highlighted subsets or labels that point the viewer to the story.
  • DET Use of details/callouts
  • NONSTD Non-Standard Design: Made a non-standard choice. This might be good or bad. In general, creativity is rewarded - but it is best to justify.
  • RATIONAL Notable rationale: Rationale goes beyond naming encodings: ties choices to the task, discusses tradeoffs or limitations.
  • RICH Data richness: Shows several variables at once in a usable way.
  • SELFCRIT Self-critique: Recognizes limitations of the design and what it gives up.
  • STORY Shows the story: The design itself makes the intended finding visible, brought out through design choices.
  • XFORM Useful transform: A derived value (change since a baseline, difference from a reference, rate per decade) that makes the comparison directly visible.

Superlative

  • NOVEL Novel design: A unique, effective design choice that goes well beyond a standard or adapted chart.

GenAI Disclosure:

Most of the grading descriptions come from codes gathered over the years, but a few codes were added by Claude. Claude wrote the grading system under my direction, including the tool for generating these explanation pages.