This project aims to get you to work with human motion / motion capture data. As a side effect, you’ll build up some of the basic framework code that you’d need if you want to do example-based human motion work.
This project is split into two parts:
- You’ll build some very basic infrastructure for doing motion work. To be able to read files, show motions, do some basic calculations. If you can’t do this, you can’t do something more complicated. (we’ll call this assignment 5)
- You’ll build something more interesting using what you did in #1. (we’ll call this assignment 6)
You will do part 1 in groups of 2.
Experience shows that if I don’t require #1 early, people discover too late in the game that they are in trouble. Of course, if your group is failing, it’s hard to know what to do – we might shuffle people around to more successful groups, ask that you borrow code from another group, or take other actions. I don’t want to put too much thought into it – because hopefully, it won’t happen.
The main deadlines
I will let you choose your partner for this assignment. You must tell me before noon on Friday, February 22nd who your partner is by email. If I don’t hear from you by that time, I will assign you a partner. Please pick a different partner than you worked with for Assignments 3&4.
Phase 1 is due on Friday, March 8th. This is firm: if you don’t have Phase 1 done, it’s hard to provide appropriate corrective actions for phase 2.
Phase 2 is due on Friday, March 22nd. This is the day before break. It’s unfair to expect people to work on projects over break, because some people actually take breaks, so its hard to be fair. By the same token, I don’t want this project going on after break (since I want you working on the next project!). If you choose to work over break, it’s your choice to put extra time in, so in fairness, we’ll have higher expectations on what you need to do.
The Ground Rules
For Phase 1, you are required to work in pairs. For Phase 2, we will allow the pairs to split up (and have these people join other groups) to form groups of 3.
You may choose to use any programming language and tools that you like. The only catch is that you must be able to demonstrate your program somewhere in the CS building. It’s OK if it only runs on your laptop. You also must only use tools that you have legal license to use. If you are in doubt whether something is acceptable, please ask.
You may use any utilities and libraries that you want, subject to a few caveats. They must be something that you are allowed to use (preferably open source), and you must give proper attribution. Also, you shouldn’t use libraries to do the main parts of the assignment (or, if you do, you need to make up for the challenge some other way). So, for example, you should write your own parser for a motion capture format, and build your own data structures from representing the motion data. If you use some library that has these features, you should either roll your own – or find other features to add.
If you have any doubts about the appropriateness of what you’re doing, please ask.
Phase 1
For phase 1, you need to be able to read motion capture data files, display the results, and perform some simple computations on them.
- Your program must be able to read some common hierarchical motion capture data format for which you can find a lot of examples. There’s a ton of motion in BVH format in /p/graphics/Data/Motion. There’s a ton of motion in a variety of formats at http://mocap.cs.cmu.edu/. If you look around, you can probably find tools to convert between formats. Note: these are skeletal formats, and I would like you to be able to read a skeletal format. If you want to be able to read a marker-cloud format (for example there is a ton of data at http://mocapclub.com/), that’s something different (but a nice bonus feature). You need to show that you can read a wide range of files. (more than a handfull)
- Your program must be able to display the motion on a character (in some kind of 3D view). The character can be drawn simply, but it should be clear that you have the orientations of the bones correct. So line segments connecting the joints aren’t enough – but boxes for the bones or something like that is more like it.
- Your program must let the user play the motion at the right frame rate (and do some timing to play at a set framerate). It’s a nice (and simple) addition to allow other frame rates (like “as fast as possible” or “15fps slow-motion.” You should have “scrubbing” (allow the user to control time with a slider), and buttons for single stepping through the animation.
- Your program must allow the user to control the camera. A bonus is to have the initial camera position show the whole motion (which requires computing the range of the motion, and then figuring out where the camera should go so that its all visible).
- Your program must allow for loading multiple motions – even if you only display one at a time. In the future, you will need to add features that combine motions.
- You must be able to generate marker position data from the skeletons. You should be able to generate “tracks” from the joints – like showing a streamer that follows the path of a hand.
- You must be able to splice pieces of motion together (given a set of ranges of frames, create a single long piece of motion with those pieces). Note: the not only means copying the frames, but also rigidly transforming the pieces so that the end of one meets the beginning of the next. You do not need to have an automatic way to determine what splices to make. In fact, for this part, you don’t even need to worry about the transitions. You do need to have a way to automatically figure out the rigid alignment (to put one clip after the next) – simply putting the root in the same position and orientation is good enough.
- While it’s not a requirement, it’s probably best if you use quaternions as your internal representation for rotations, since you will probably want to do some blending later.
- Another nice “optional” feature is to be able to write files back out.
The trick here is to build things in a way that will let you use the basic elements to do more complex things in the future.
Phase 2
For phase 2, you need to implement some more complicated motion synthesis thing on top of what you have already built in phase 1. You can propose something based on the readings (and I’ll help you scale it appropriately), but here are some ideas:
- Implement motion graphs – find transition points in a collection of motions, implement blend-based transitions (better than just the “cut” transitions of part 1), and some simple mechanism for control. (start with random motion, but then add something that is more purposeful). Do something simple for each piece, and then improve one (or more pieces) and show how it gets better.
- Implement more complex blending – perform time alignment (and maybe coordinate frame alignment), and create methods for identifying what motions might be blendable.
- Convert to a “Cartesian” representation (point clouds or something like MKM) and show how blending is worse/better with the different representation.
- Implement some of the simpler motion editing features (motion path editing, time alignment, simple discrete graphs) and put them together. For example, have a “fixed move tree” (say walk straight, or turn left/right) and add path editing to make a controllable character. Implementing basic skinning (including being able to read a mesh and having some ways to get skin weights) falls into this category.
- Figure out a way to take marker data (like what you find on mocapclub.com) and figure out how to apply it to drive a character mesh. Note: this requires you to figure out how to read that kind of data, as well as to figure out how to map it to a character – which is different than the usual skinning since you don’t have coordinate systems.
I am willing to talk to each group to help you pick something reasonable.
For the Phase 1 deadline, you need to have an initial plan of what you want to do for Phase 2.
Milestones
February 22nd: partner choices are due. If you don’t send me email with your partner choice by noon, I’ll pick a partner for you.
March 1st: Checkpoint. I’ll check in with each group to make sure you are making sufficient progress. By noon, you will have to have posted something to moodle send me email saying what you have working. See status emails
March 8th: Phase 1 Due. You will turn in a description of what you’ve done. There will be a peer review process (where you will need to provide evaluations of other people’s assignments). Details will be provided beforehand.
March 8th: Project 2 idea. You need to have told me about your idea for project 2 by email. Ideally, you’ll have started the conversation before March 8th. Probably there will be an email discussion as we refine topics.
One catch: I will be out of town on March 8th (and for a few days around it).
March 15th: Status update. You will need to provide information on how your Phase 2 project is going, and set some expectations about when you will turn things in, what you expect to turn in, how we will evaluate it, … Details will be announced.
March 22nd: Projects due. The form of handins TBD. We will probably do live, interactive demos.
