Peer Critique Score Sheet
20pts total (since this is 20% of the DC1 grade)
- 5 pts – Did all 3 reviews correctly
- 5 pts per critique
- 5 – Great critique. Has all required parts (acknowledgments, specifics connected to principles). Commits no bad practices. Offers interesting insight or multiple insights.
- 4 – Good critique. Has all required parts (acknowledgments, specifics connected to principles). Commits no bad practices.
- 3 – Marginal Critique. Missing a required part, or commits bad practices.
- 2 – Poor critique. Multiple problems (missing parts, multiple bad practices, …)
- 1 – Unacceptable critique – we couldn’t even show this to the author.
Parts of a critique:
- Provides an overall statement that shows that acknowledges the design.
- Provides at least 1 specific detail and connects it to principle
- Avoids bad critique practice
- prescriptive without backing up with principle and reason
- prescriptive suggesting the author do something that might address a different question
- poorly phrased prescription
- critique of the author
- incorrect use of a principle (e.g. suggesting use of non-ordered encoding for an ordinal variable)
- bonus for insightful comment or thorough criticism
Generally, good critiques get a 4. You get a 3 if something is missing, or a 5 if you do something exceptional. For each critique. In grading, rather than write down each reason speficially, we used the following standard codes:
Reason Codes (generally things that are missing – causes why you didn’t get a 4):
– Z – prescribes without goal and/or incomplete reasoning
– Y – negative comment without principle to back it up
– X – Non-specific acknowledgment (or no acknowledgment of design/goal)
– W – unfounded (or unsupported) assertion
– V – Specific change not connected to reason.
– U – No specifics.
– T – Gives alternative design without motivation.
– S – Negative not connected to goal
– R – “Is not the best. X is better.” (mixes in T,W,Y)
– Q – Questioning the authors motivation
Reason Codes (things that are good and push your score up)
A – Extensive acknowledgment
B – Extra-thorough reasoning
C – Many good detail points
It works out that if you get a 4 on each, you get 17/20 which is 85% or an AB. We don’t always write all applicable codes.
Some specific examples:
Provides an overall statement that shows that acknowledges the design:
Good: The task for this visualization is to present trends among average household income with two separate education measurements – % of adults with bachelor’s degrees or higher and % of adults with some college or associate’s degree. Specifically, the visualization raises the question if bachelor’s degrees tend to lead to higher income than just an associate’s degree.
Not Good: The design has presented the data in a very informative manner. Awesome!
Provides a specific Detail and connects it to principle:
Good: If the goal was to more directly compare the scatter plots and trend lines for the two separate education levels, then it would have been better to have the two education levels on the same scale. The bachelor’s degree attainment scale ranges from 0 to 55% while the associate’s degree scale ranges from 0 to 40%. This makes it easier for the viewer to draw incorrect conclusions from the visualization.
Less Good (only detail, no principle): One suggestion is that maybe it can change the representation of average line as it is a little distractive for me.
Bad (prescribes, without giving principle or rationale): If the purpose is to show the relationship of the factors with respect to months, it would have been better to use discrete variables for months.
Acknowledges the design thoroughly:
Good: The design shows the relation among birth rate, death rate and numeric change in resident total population in 2015 for a few states in the United States. It contains two overlapped plots: stacked area chart to show birth and death rates, and bar chart to show numeric change in resident total population. They are good choices although both stacked line chart and grouped bar chart could be options for the former one. And I understand stacked area chart as a variation of stacked line chart, and it is good at emphasizing the difference between two sets of points (i.e. birth rates and death rates for several states in this case). The bar chart shows negative numbers below x-axis, which is great to highlight negative cases. Three variables are color encoded and length encoded (i.e. height of the bar), and they all contribute to a clear story. Also, states are sorted alphabetically, which makes the viewer locate a state effortlessly.