DC2 Grading and Rubric

by Mike Gleicher on November 22, 2017

Everyone who turned in a draft should have gotten some (limited) feedback as a comment on Canvas. (note: I may not see any replies you make to those).

There is a link to a long list of common comments I made on assignments, so you can see the kinds of things I noticed on some assignments. (the list is here).

I also just posted a draft “rubric” that will give you a sense of the kinds of things we will be looking for in grading.

DC2 Rubric

by Mike Gleicher on November 22, 2017

Everyone who turned in a draft should have gotten some (limited) feedback as a comment on Canvas. (note: I may not see any replies you make to those)

To help you decide how to finalize your DC2, here is the draft of the grading rubric. It will hopefully give you some idea as to what we are looking for when we do grading.

In doing grading, most of the grade will be on the “final” project, however…

  1. You will be penalized if you did not turn in all aspects of the assignment. We will deduct 1/4 of a grade for each assignment missed (e.g., if you missed 2 parts, your A will turn into an AB).
  2. We will consider the quality and completeness of your Phase 1, 2 and 3. Doing exceptionally well or badly on a part can push a grade in either direction (so if you’re on the borderline, this will matter more). Great early parts will help more than bad early parts hurt.
  3. The early parts are per person – a “final” grade is given to a team, and then the per-person adjustments are added.

DC2 Rubric:

No project is expected to excel in all areas. A project should get into the top score in a few categories and/or middle scores in a lot of categories.

  • Quantity/Diversity of Designs:
    1. One (or multiple similar)
    2. Good (a few, reasonabe diversity)
    3. Exceptional
  • Creativity/Cleverness of Designs:
    1. Simple Tweaks to Standard Designs
    2. Interesting / Non-Obvious
    3. Exceptional / Innovative
  • Degree of Implementation:
    1. Sketch
    2. Mockup / Use of Tool / Use of Fixed Data
    3. Demoable tool
  • Visual Quality of Design Presentation
    1. Rough sketch
    2. Works enough to get idea
    3. Details thought through to be usable
  • Matching to Task
    1. Tasks not stated, or not connected
    2. Tasks described and connected
    3. Convincing task rationale and critique
  • Critique
    1. Limited
    2. Explains good/bad
    3. Thorough
  • Evaluation Plans / Sample Data
    1. Minimal
    2. Thought through description
    3. Exceptionally Complete
  • Quality of Presentation
    1. Has Basic Stuff
    2. Clear and Clearly Complete
    3. Exceptional
  • Design Documentation
    1. Hard to understand what is going on
    2. Clearly communicated
    3. Exceptional (conveys design and rationale)

Design Challenge 2 Draft Feedback

by Mike Gleicher on November 22, 2017

I wanted to get some feedback to people quickly – so the individual feedback isn’t as extensive as I may have hoped. So instead, I am giving the feedback collectively. Each draft should get a comment in the handin – at least with a link to this posting.

Note: the 4/4 means you turned something in. It’s check/no check grading for the drafts.

Overall, I am very impressed by the quick scan that I did – there is a great diversity of assignments, and people are succeeding in many different ways. There are lots of different designs, and considerations of different tasks. Many assignments show some really good, deep thinking.

While going through assignments, I made a list of comments I would make. Some of these may apply to you. Even if I specified one or more, the others may be relevant as well. Or maybe you can see some of the problems that you don’t have.

Each is written about one design – it could be one of your designs have this problem, or that many do.

Or, it could be that your work doesn’t actually have this problem, I just thought it did at a quick glance.

(some critiques may be about the set of designs)

  1. Your design is quite specific to a particular data set or use case. For example, there are map designs (which make some assumptions about what the lines are) – where each line is a map.It’s OK to make highly specialized designs. Just document it. Be clear what are the requirements for when your design is applicable. Describe how your design might generalize.
  2. Your design/scenario isn’t one that obviously fits the assignment. The assignment is about line chart data (with many lines). It’s not clear how the data you’re describing and/or the design you’re giving fits. You should better explain how your design fits the model, and compare your designs to the standard designs (since the standard designs are a good fit for the data type – they just may not be effective for some tasks).
  3. Your design you provide is really similar to the standard designs. It’s not clear what the pros/cons are relative to the corresponding standard design. You should make that comparison more explicit.
  4. Your design doesn’t appear scale well in one of the axes. Try to explain why you think the design scales (or doesn’t, as the case may be).
  5. Having an experiment design is great – but you may want to add detail. What kind of data might you use for the examples? What kinds of things might you measure? How will you define tasks with measureable correctness?
  6. I can’t quite figure out what is going on in your design. Usually, this is a sign that the design is interesting (it’s non-standard enough that it isn’t obvious) – but it needs to be described better.
  7. Be sure to describe the designs in a clear way – and have it be easy to find the descriptions.
  8. Be careful about using adjacent area encodings for things that are not part/whole relationships.
  9. Consider spacing for the lasagna plots or some other method to make the rows more distinct.
  10. Using eye tracking for evaluation can be tricky.
  11. Make clear what you have implemented, and what is a mockup.
  12. Be sure to document how to run things and what tools/libraries were used.
  13. Your design is really simple, a very minor tweak on the standard. This is OK, if it has some other good feature to go along with it.
  14. Your discussion seems to focus on a single, small example. At least discuss how your designs scale, and why this single small example isn’t well served by the traditional designs.
  15. Your designs seem very similar. You may want to contrast them, and explain why you need multiple designs.
  16. I’m not sure I quite understand the task you describe, please elaborate/clarify.
  17. Comparison in a stacked chart can be hard since there is no common baseline.
  18. Designs that show one time at a time serve very limited tasks – be sure to consider what they can and cannot do.
  19. Try to back up assertions. Saying something is good requires some explanation.
  20. Discuss tasks as part of critique/comparison.
  21. It’s not clear how the design connects to the task
  22. If you’re focused on a single task, be sure to compare/contrast your designs.
  23. Descriptions and critique are terse.
  24. Describes cons (as well as pros) for designs.
  25. Give rationale for designs.
  26. Give comparisons with baseline designs.

The Week in Vis: Week 12 (Nov 20-Nov 24)

by gleicherapi on November 18, 2017

Week 12 (Mon, Nov 20-Fri, Nov 24) – Dealing with Scale

Last week, you got by without me – hopefully making lots of progress on Design Challenge 2, and learning about text visualization, and getting some practice with a design exercise.

Last week, I was at a workshop on bridging databases and visualization. It is in some ways very appropriate to this week’s topics.

This week, we’ll talk about scalability. It’s been lurking the whole time, and we’ll discuss it head on. Monday we’ll talk about dealing with large data sets. Wednesday we’ll talk about high dimensional data sets.

No class Friday because it’s a holiday.

Learning Goals (for this week)

  1. Appreciate the kinds of challenges that arise as data and problems grow in scale.
  2. Be aware of some of the strategies available to combat scalability issues (in terms of design, not necessarily in terms of implementation)
  3. Have awareness of the pros and cons of key strategies (such as dimensionality reduction and sampling)

Design Challenge 2 Discussion

by Mike Gleicher on November 16, 2017

I had forgotten to create a Canvas discussion for Design Challenge 2 questions. Fortunately, a student did it themselves. I have answered their questions – which may hopefully help others.

The discussion is at: https://canvas.wisc.edu/courses/66251/discussion_topics/162806

I may have limited internet access until Monday, so I may not be able to respond to other questions.

The Week in Vis: Week 11 (Nov 13-Nov 17)

by gleicherapi on November 10, 2017

Week 11 (Mon, Nov 13-Fri, Nov 17) – Uncertainty

Last week, we talked about interaction and did an in class exercise about airline route maps – without interaction.

This week, we’ll continue the design exercise to expand to consider interaction, and some other things (like hints for the design challenge). On Wednesday, we’ll talk about text visualization.

Yes, the reading is not connected to lectures. Too many topics, too little time…

The reading for this week is intentionally light so you can focus on design challenge 2.

The Week in Vis: Week 10 (Nov 6-Nov 10)

by gleicherapi on November 4, 2017

Week 10 (Mon, Nov 6-Fri, Nov 10) – Interaction

This past week, we talked about human perception and color. We didn’t do any design exercises. Design Challenge 2 is well under way.

This coming week, we’ll talk about interaction. And we’ll do some design exercise in class. And you’ll keep working on Design Challenge 2.

There will be no class on Friday.

Learning Goals (for this week)

  1. Understand how interaction is used as a component of visualization design, and what problems it can be used to address.
  2. Have a sense of some of the key types of interactions one adds to visualizations, and what they are good at helping with.
  3. Appreciate the costs of adding interaction to a visualization (including the challenges of implementation).

DC2 Abstract Task Lists

by Mike Gleicher on November 2, 2017

The grader has compiled a long list of abstract tasks from your DC2 Phase 1 assignments.

For parts 2,3 and 4 you can use tasks that you came up with – or you can look through the long list for more ideas.

The long is is a page on Canvas.

Class Schedule (no class Friday)

by Mike Gleicher on November 1, 2017

As I mentioned in class today, there will be no class on Friday 11/3 (originally, the calendar had said “optional”).

We’ll also skip the other optional Friday classes later in the semester.

I had mentioned the possibility of office hours on Fridays during the class time slot: I need to make this “by appointment only.” My regular office hours on Wednesdays (2:30-3:30) stand, and I can make other times work by appointment. (except for the week I am out of town). I will be out of town 11/10-11/17 (inclusive) – but should be responsive to email.

There will probably be one guest lecture during the week I am away – stay tuned for details.

 

DC2 Clarifications

by Mike Gleicher on November 1, 2017

I made some changes to the Design Challenge 2 description. One clarification, and one change.

They are highlighted in blue on the assignment page.

Briefly:

11/1/2017 – Clarification: for phase 2, you can use the abstract tasks you turned in in phase 1. If you you want to do tasks defined by others, you can pick those as well. We will try to get a list of tasks compiled from student phase 1s out quickly so you can get more ideas beyond what you did in phase 1.

11/1/2017 – Schedule Change: we are extending the final deadline from November 19th to November 26th, although (1) the deadline will be hard (we won’t accept assignments more than a few days late), (2) you still need to turn in a “draft” for November 19th so we can get a sense of what you are turning in. I didn’t want to make an assignment due over Thanksgiving, but think of it as due before Thanksgiving, with a grace period. More details in blue below.

(if you prefer to think of it this way: … The deadline is still the 19th, we’re just giving a clearer and more lenient late policy, with the ability to “resubmit” if you want to update your assignment).

Warning: it may take a few days for the course calendar and canvas to be updated with the new deadlines.