One person sent me a P2 self evaluation, which reminded me that I didn’t give you instructions for them.

I will not look at self-evaluations until after we have graded the project (after reading it, we may adjust your grade if you were at a borderline). So, you can turn it in until Wednesday, 5/13 at 5pm.

Using the questions from P1 is fine. They are:

1. How happy are you with the outcome?
2. What went right/wrong in your project? What would you do the same/different?
3. What will you do the same/differently on the next project?
4. What advice would you give to someone else proposing to do this project in the future?
5. The cliché is to ask about what you learned from the experience. This is good self-reflection practice, but may already be described above.

However, #3 doesn’t make as much sense. Instead, I would prefer if you consider the following questions:

1. How happy are you with the outcome?
2. What went right/wrong in your project? What would you do the same/different?
3. What advice would you give to someone else proposing to do this project in the future?
4. If you worked in a group, how did you split the workload?
5. What could we have done to better connect the class content to projects? What could we (the course staff) have done to have made this project a better experience for you (or students in general)?
6. The cliché is to ask about what you learned from the experience. This is good self-reflection practice, but may already be described above.

In general, we are really interested in getting feedback on the course. It’s still a work in progress. So if you have thoughts on what we could do better or differently (or what you think works and shouldn’t be changed), please let us know. If you want to do it anonymously, please put a printed page in my mailbox.

On Friday, May 4, the “handin plan” is due. The project page says little more than “expect instructions.” Well, this is it.

There are two goals here:

1. We need to predict what you will be delivering to us, so that we can figure out how you should transmit it to us.
2. We want to get an idea of what to expect, so we can have the opportunity to catch problems where projects aren’t going to meet expectations early.

So, on Friday May 4th (preferably before noon), please send the instructor and TA a note saying:

1. What you expect to have to turn in. Will it just be a big PDF? Will it be a ZIP with code and an excutable? Is there a lot of data? How big? (it’s not practical for you to deliver more than a few MB by email). Do you have some mechanism for putting it on the web?
2. Will you have a demo to run? If so, will you be able to put it on the web so we can try it? Even if we can try it, we will probably want to schedule a time to look at it and discuss it with you. (we want to get a sense of how many demos to schedule).
3. How is the project progressing? Give us a sense of where you’ve gotten to and where you expect to get to. (a few sentences – maybe 2 paragraphs – just enough so we get a sense).

As far as when things are due: according to University policy, things must be due the last day of class. However, we will have an “email blackout” between Wednesday May 9th at noon and Monday, May 14th at 10am. Anything sent during this time will be considered as handed in at May 9th, but won’t be looked at until the 14th. In fact, we’d prefer that you didn’t send us things until Sunday the 13th (since we’ll both be traveling).

So, the deadline for us to receive your project materials is really Monday, May 14th at 10am. This is a pretty firm deadline, since we need to grade it quickly.

You will at least need to send us your written report by email (if it’s a small enough PDF, otherwise send a web link) by this time, and have some arrangement for us to get other files.

Overall, everyone who submitted something did at least a good job.

There was variance. Some are more insightful than others. Some more directly addressed the question (how does the research utilize class concepts), while others critiqued the tool itself. Still others decided to praise the tool (sometimes in ways that didn’t necessarily add). I will account for at least some of the variance by the way the question was phrased.

But, Almost all the things were (at least) good. There were a few (4) that were just OK (and one that might have been a little less than OK). But since I am not sure I can objectively qualify why things fell into these different bins, I think its more fair just to say that those who didn’t do “good” legitimately had different ideas about the assignment.

So, everyone who turned it in gets “100%.” We’ll count this as a “written participation” – but emphasize it more than the individual Piazza assignments.

To the person who did a visual critique of the paper: you are totally right! I can’t tell you how hard it was to cram everything in to 10 pages. And it really does show (the paper is way too dense).

(note: I have assigned grades as of noon, 4/28/2012 – but I have not mailed them to you yet)

Project Scoring Rubric:

N=5 or 6 (weeks)

1. 1/N – did you do all of the pieces (most people get 100%)
2. 1/N – proposal & revision (lenient scoring – most people get 100%)
3. 1/N – final material quality (clarity, completeness, … – generous scoring)
4. (N-3)/N – content quality

Note: the content quality mixes in to other parts as well. The content expectations are higher for longer projects.

Note: for 838, the "meets expectations" level for 2&3 gets you an AB. #1 is basically a freebie A. For 638, the 1&2 meets expectations level gets an A (and 3 gets an AB). We are generous on these parts so we can be honest/harsh on #4.

In terms of the 5/6 week thing: note that if you took the extra week, your actual content was more important to your grade (makes sense, since you had an extra week for that, but the same amount of time for the mechanics). Since, generally, scores are higher for 1-3 than 4, this works out.

#1 All Parts:

We counting up all the different parts. Most people turned in everything (including summary)

• initial contact/reply (a few people didn’t meet the letter of the law, was only counted against you if the next two things were problematic
• one person missed phase 2, but arguably thought their initial proposal was OK
• one person effectively missed phase 1 & 2
• everyone had the summary (but 1 was extremely late)
• 3 people were missing self-evals (note: I will not read these until after grades are assigned)

#2 Proposal

838:
    Exemplary = A (very few)
    Good enough = AB
    Minimally Acceptable or Problematic = B (or less)
638:
    Good Enough = A
    Minimally Acceptable = AB
    Problematic = B
   
note: some i gave +/- to, which basically says it might be better/worse than the grade and that should be considered in border cases

#3 Materials

This attempts to assess the quality of the write-up, how nicely things were handed in and presented, … – independently of how good the actual content was. It’s trying to be more about the form, although this is admittedly hard to separate.

638:
    Good (or better than good): A
    Good enough / acceptable : AB
    Problematic: B (or worse)
838:
    Very Good: A
    Acceptable / Good Enough: AB
    Problematic: B (or worse)

#4 Content

Hopefully, the detailed comments (from the P1 feedback) are sufficient to let you know why I gave it the score that I did. Remember, things are scored against the expectations set in the proposal (also described in the initial feedback).

Initial P1 Feedback:

The above scores were built on the P1 feedback.

Please be aware that these may come across as negative – I often was finding flaws, rather than praising the good parts on this pass. Also, in terms of grades, a portion of the grade comes from correctly following the process, and most people do well on that aspect.

I have initial feedback for all of the project 1. It isn’t totally “graded” but there is at least some commentary that may help you in planning project 2. (my not-so-good excuse for the delay is that I wanted to focus on giving feedback for P2)

I will warn people that on the first pass, I am often noting what is wrong, not necessarily counting up all that’s right. (it’s kind of like grading a test where you put an X on the 1 that’s wrong, rather than 99 checkmarks on the one that’s correct).

Today, the revised and approved project 2 proposals were due. I have gotten few of them so far (and have not responded to any of them). I’ll give people the excuse that you were waiting for P1 feedback. So, everyone can have an extension until Monday. However, remember that this is for having a project proposal approved: just sending it to us at the last minute is not an opportunity for us to iterate with you.

I am noticing from the final results of P1, that more careful scrutiny at the revised proposal and 3rd week update phase may have caught problems. So I strongly encourage people to give us description of what you are doing and where things are going (as much as possible): the more we get from you, the more we may be able to help steer you.

(due Wednesday, April 18th)

These were called seek and find, but I changed the same a little bit to make it catchier and more about the real goal.

For this assignment, you need to find an example of a visualization on the web. You need to find an example that shows relational (graph) information, but not using an explicit node/link diagram representation (as a line or curve connecting the nodes). (update – it might be hard where the links aren’t shown explicitly. if you can’t chose one that uses some interesting way to place the nodes)

In your description of it: critique the choice of not using the explicit node/link representation. Why (and why not) was this a good choice by the visualization’s author. Your critique should be more general (give a sense of what it is trying to convey, and how effective it is).

The ground rules are similar to the last seek and find exercise:

Find a visualization, preferably on the web (so you can give a link to it). Preferably one that can be conveyed well in a small picture, and preferably one that presents data of interest to you (especially if you work in some domain). Preferably one that is different than what other people have given.

Grab a snapshot of the visualization (as a picture like a PNG or JPEG – you may need to use screen capture and then edit it). In addition to linking to the visualization, we want you to post a picture (unless you can figure out how to embed without uploading). Please make sure the picture isn’t too big (if you upload a big picture, it even uses bandwidth if you show it at small size).  We want the picture by itself so we can see it without following the link to where its from, and to see it out of context).

When you attach your picture to the Piazza question, make sure that you use a filename that includes your name (enough so we know who posted what). Embed a small version of the image into your posting, and a link to where the picture came from.

Your posting should explain the network information that is being conveyed, a critique based on the concepts in class about its effectiveness, and an explicit comment on the designers choice of a non-explicit node-link diagram format.

If you are having a tough time finding something that isn’t clearly not a node-link diagram, you might also want to explain why you picked it as an example that gets away from the standard kind of display.

This assignment is due on Wednesday, April 18th. (although leniency for assignments turned in before Friday the 20th).

The Piazza page is here.

Hopefully, today was helpful in helping you refine your project ideas, and in either finding someone to work with or deciding to go at it alone.

Please send us (Michael C and I) email as you start to form project ideas, if you want project ideas suggestions, if you pick a partner, … We are happy to help provide project ideas, give you initial feedback, make suggestions as to what may or may not be appropriate, …

I will have office hours tomorrow (Thursday 4/12) and Friday (4/13) 11:00-12:00 if you want to talk about project ideas (or anything else).

Sending us email before Friday (about either your partner choice, or project ideas) is optional, but can help us help you form your project proposal. Remember, a project proposal is due on Friday.

Project 2 has been posted here

It’s basically project 1 all over again. But the first deadline is different. And its due pretty much right after break (note: a lot of other things are due this week as well).

On Tuesday, April 10th (any time, so we have it before Wednesday), you must send an email to the TA and Professor with your intention for project 2. Each person must send this (group or not). Please include:

• Whether or not you want to continue with your project 1. If so, give a sentence or two idea of what it might be to continue. (which may be taken right from your Project 1 summary posting)
• Whether or not you have an idea of something you want to do (if it’s not a follow on to project 1). It’s OK if you want help thinking of something – let us know and we will help you if we know early enough.
• Whether or not you have a project partner in mind. If you do not have a project partner, we can help you find one. We will encourage people to form groups. You are not required to work with the same partner.
• If you have any extenuating circumstances. For example:  If you really do want to work alone, let us know. If there is something about your project that precludes someone else helping out with it, let us know.
• If you have any thoughts on the mechanics of the last project that we might use to improve this one.

If you have more detail, we are happy to hear about it.

.

• Request for comments and ideas – IMMEDIATELY
• April 10 – Phase 0 – Initial Email
• April 13 – Phase 1 – Proposal
• April 20 – Phase 2 – Revised Proposal
• April 27 – Phase 3 – Progress Update
• May 4 – Phase 4 – Handin Plan
• May 11* – Phase 5 – Final Handin

Note: this is very similar to Project 1. A few details have been changed (I will try to point out what’s different in blue).

You are encouraged to work with a partner. If you would like us to help you find a partner, let us know. We’ll use these to help find people with similar interests and/or complementary skills. You are allowed to work in a group of three.

For this project, we will give you additional incentive to work as a group: the expectations for individuals are the same for pairs.

For this project, the goal is to give people enough flexibility so that they can learn the kinds of things they want to learn in the way they want to learn it. This is hard because of the wide range of backgrounds in the class.

Note: for 638 students, the project expectations are lower, but we still require you to do something for both projects. This was a slight change from the original course announcement.

Note for 838 students: at least one of your projects must have at least some implementation component.

There are several different elements for projects…

1. Literature Review – what is the literature on the visualization “science” for this problem?
2. Practice Review – what are examples of how people have dealt with the problem?
   In #2, you are finding examples (e.g. pictures in a domain paper where someone had the problem you are addressing, or examples of tools people are really using). You should discuss how well the visualizations “work” in their context.
3. Design Study – can you characterize the problem? Can you explore how to design a tool that really addresses it (or survey how existing tools really address it)?
4. Implementation and Evaluation – can you build something to address a problem / demonstrate a technique? Can you assess if it’s any good?
5. Learning about a Tool – it might be your objective to learn about a particular tool. This implies not only learning about the tool but also being able to talk about it (what its good for, …) Note that this is corrected for your level. That is, if you’ve never programmed before, learning some basic scripting to help process your data might be a big deal – if you’re an expert programmer, you need to pick something more ambitious.
6. Something totally different (er, really creative) – I am open minded to what might make a good project.

Projects might be “Data Projects” (problem-driven) – centering around a particular data set (or type of data set) that is either particularly interesting, particularly hard, or both. You might choose a hard standard data set, a data set that you particularly care about (especially if you’re a domain scientist).

Projects might be “Visualization Topic” Driven – center around a method or problem type that you are particularly interested in. For example, you might be interested in graph layout or volume visualization or applying non-linear regression.

It’s up to you to mix and match.

Some ideas…

• You might pick your pet data set and do a literature survey of the available methods for it, and a critique of the current practice.
• You might take a particular data set, and create a visualization of it. Doing this involves understanding the design choices in making an effective visualization. It may require a non-trivial amount of “data scraping” to gather the necessary data and get it into the form you need to make the visualization. Note: if you choose a design focused project, you will need to provide some discussion to let us know that there is enough analysis/design process involved.
• You might pick some interesting algorithm and try to implement it in an unfamiliar development environment. There will be some data involved, but its mainly to illustrate other things.
• You might pick some interesting visual design and do a literature survey of it.
• You might try to build tools for exploring a particular data set (which combines design study and implementation).
• You might look at typography for labels and text in diagrams from both the artistic side (what makes for good typefaces for this) as well as the practical side (how do you use typefaces that show up reliably in the web browser).

Novelty is welcomed – trying to come up with a new design for an underserved problem, or a new algorithm that is particularly suited to a particular situation, or the analysis of a new dataset. But novelty isn’t necessarily required – provided that something is new to you. Saying you want to implement a classic algorithm so that you can understand it (and maybe learn a new toolkit for implementing things) is fine too.

I am very open minded to the ways these various kinds of components may be put together. There is no requirement that a project involves programming – it might be purely a survey, or analysis exercise, or even paper designs. Of course, this just moves efforts between categories – a project with a big implementation component won’t have as much of a literature review component.

Your project does not have to be unique. I am open to (in fact would encourage) different people/groups who are not working together to work on similar projects. It’s always interesting to see how different people tackle a similar problem.

Phase 0/1 – Project Proposals

But, with this flexibility comes a responsibility for you to work out with me ahead of time what you want the project to be. You can’t just email me something on the deadline – it will probably take some iteration and dialog.

I (or really we, since Michael the TA has lots of ideas too) am happy to give you advice. Providing partial project ideas is fine – providing you do it early enough. If you tell me “I want to write a survey” or “I want to implement some algorithm in D3” or “I want to do something with this network data that I have,” I’ll try to suggest something. We are happy to do some matchmaking to help people form project teams.

We can suggest papers to read (either for topic ideas, or to seed a survey or …). We can suggest tools you might want to learn about, algorithms you might want to implement, sources of data sets to explore, … I am willing to pick a project for you – but you have to ask. And at least give me some ideas as to what you might prefer.

By Monday, February 27th, you need to have at least initiated a dialog with me. I will hold extra office hours that week to discuss project ideas. It’s best to get me some ideas early, so I can help you develop them. If you want help identifying a partner, be sure to tell us by this deadline – we’ll try to pair people up (who want to be paired).

On Tuesday, April 10th (any time, so we have it before Wednesday), you must send an email to the TA and Professor with your intention for project 2. Each person must send this (group or not). Please include:

• Whether or not you want to continue with your project 1. If so, give a sentence or two idea of what it might be to continue. (which may be taken right from your Project 1 summary posting)
  
• Whether or not you have an idea of something you want to do (if it’s not a follow on to project 1). It’s OK if you want help thinking of something – let us know and we will help you if we know early enough.
• Whether or not you have a project partner in mind. If you do not have a project partner, we can help you find one. We will encourage people to form groups. You are not required to work with the same partner.
• If you have any extenuating circumstances. For example:  If you really do want to work alone, let us know. If there is something about your project that precludes someone else helping out with it, let us know.
• If you have any thoughts on the mechanics of the last project that we might use to improve this one.

If you have more detail, we are happy to hear about it.

You must have an “accepted” project plan by Friday, April 13th. Unlike Project 1, we don’t need to have the proposal “accepted”, but we need to get it from you before Friday, so we can get you feedback on Friday. If you get us a proposal late on Friday, we may not get you feedback until Monday, which will delay you starting the project.

The format for the project proposal is the same as project 1. A project proposal should clearly identify what you are going to do – both in terms of topic as well as the actual tasks that you expect to do. It should articulate your goals – what you expect to create, the problem you intend to solve, what you hope to learn. The proposal should also have some idea of how you intend to do things, and a rough plan. Your initial plan might involve doing some literature search and design exploration in order to come up with a more detailed plan. But you should at least have some sense of where things are going.

Phase 2 – Finalized Project Proposals/Plans

Unlike project 1, we are putting together the “revised plan” and the “accepted proposal”. By Friday, April 20th, we need to have “accepted” a detailed proposal than mixes the project plan from project 1 (as described below) and the acceptance process. Make sure to send us something early enough in the week so that we can iterate if necessary.

When you do your initial proposal, you might not know enough to really flesh out the plan. For example you might decide that you are going to do a literature survey on a topic and find that there isn’t enough literature to survey, or propose to explore some data set that you find you can’t get access to, or to do something with a tool that is shown in its example documentation (so it’s too easy).

For the March 9th deadline, you need to have at least started on the major tasks of the project. Started to find the literature you intend to review, identified some designs to critique, obtained the data that you want to explore, read some of the documentation for a tool you want to learn, …

You project plan should be a written document that includes everything in your proposal, updated, and with more detail. You should have an initial reading list, initial ideas about the design, … Please send this document by email to both the TA and the Professor. When we accept your project proposal, we will give you some more specifications on what we expect in your plan.

Note: your proposal and plan are both part of how you will be evaluated.

Phase 3 – Progress Update

By the end of the Day on April 27th, you need to send us (both the TA and the instructor) a progress update – explaining what you’ve done, and how well things are going.

Part of this is to make sure that you really are working on the project, and not just waiting for the last minute. But a more important part is for us to catch any problems and try to make corrections to plans so everyone ends up with something good.

Phase 4 – Handin Plan

For project 2, we will handle the “handin plan” a little differently than project 1. It will be more of an arrangement to see how we will handle grading and archiving, since the end of the semester may be a little hectic. Details will be announced when it is due.

By the end of the Day on Sunday, March 25th, you need to send us (both the TA and the instructor) a “handin plan” – at this point, you should be able to tell us what you will turn in, and what you expect to have completed. (we were going to make the deadline Friday, but realistically we’re not going to look at them until Monday morning – we’re still listing the deadline as Friday for symmetry).

Some of this is pragmatic (we need to come up with a mechanism for you to get stuff to us – which might be challenging if you’ve build software and/or requires access to a big data set). Some of this is more project related – we want to make sure our expectations are correct (if you are doing too little or being too ambitious).

This is also a chance for us to decide how we’ll evaluate your project. Both practically (for example, if you built software we might want to arrange a live demo) and content wise.

Phase 5 – Handin

University policy says that projects must be due before reading period begins. So the project is Due on May 11th. University rules do not preclude me from giving a no-cost extension to the project deadline, so you may consider May 14th the real due date. Extensions beyond this are very difficult: we need to grade projects before the 17th so that people have their grades in time for graduation.

We will be a little bit more organized about project handin requirements this time, so it’s hopefully less confusing.

Your project is due on Friday, March 30th. The following day is Spring Break. Allowing students to work on a project over break gives an unfair advantage to people who don’t want to take a break.

I will grant extensions until Monday, April 9th. You must explicitly ask for it. Also, there will be a higher level of expectation: if you choose to do an extra week of work, you should get more done than people who don’t choose to. (you either take the whole week extension or not)

You project handin will include a written report component, as well as any “artifacts” (images, programs, …) that you create. You will also need to write a self-evaluation. All this will be described in more details closer to the deadline. Part of the idea is the right thing to turn in, and the right way for us to evaluate it, will depend a lot on what you choose to do.

For implementation projects, we might do in person demos (if they are appropriate). These will be scheduled for after break, and be for the Professor and TA. However, all materials are due before the deadline.

The deadlines are fairly strict, since we want to move on from this project to the next one.

Some Ground Rules

You must only work on data sets that you can share with the class. Things that must be kept secret should be avoided.

We would prefer if you only used tools that we have access to. If you work in a lab that has a copy of some commercial software that you want to use, ask us.

We can provide Tableau licenses.

You can write programs in any language you want. You can run things on any system that you want. With the limitation that you must be able to demo it to us. We’d prefer if you built things that we could look at over the web, or that ran on the computers in CS. But, if you want to bring a demo on your laptop, that works too.

If you want to learn/use some tool we’re not familiar with, we probably won’t be able to help you much. (But we encourage that since we’ll learn about it too).

Some resources:

Rather than start to list sources of data, papers, and ideas, I’d prefer that people start telling me what they are interested in, and I’ll steer the list towards that.

This project is evolved from the last edition of this class. The web page has a number of potential project ideas.
https://pages.graphics.cs.wisc.edu/765-10/archives/1182-visualization-projects.

This project was inspired by the class project format in Tamara Munzner’s class. That project description page has lots of good sources of project ideas:

UBC class project http://www.cs.ubc.ca/~tmm/courses/533-11/projectdesc.html

These are additional parts to Project 1. They have their own deadlines, and they are mainly to help set up for project 2.

Project Summary:

Each person (that means there will be multiple postings from groups) must post to the Piazza page with the following:

• A one paragraph description of the project and the outcomes. This paragraph should give a sense both of what you are trying to do and what you did.
• If you worked in a group: a one or two sentence description of what your role in the group was.
• If you think there should be a follow-on to this project, and whether or not you are interested in doing this as your project 2. Give a 1-2 sentence description of what the follow on project would be.

This is due on Tuesday, April 10th (e.g. before Wednesday morning).

Self-Evaluation:

Each person must send email to the TA and professor with a self-evaluation in the body of the message. No attachments allowed.

Reflecting on a project is a known good technique to materialize the lessons. Here are some questions I’d like you to think about:

1. How happy are you with the outcome?
2. What went right/wrong in your project? What would you do the same/different?\
3. What will you do the same/differently on the next project?
4. What advice would you give to someone else proposing to do this project in the future?
5. The cliché is to ask about what you learned from the experience. This is good self-reflection practice, but may already be described above.

Send the email before Friday, April 13th.