(this is a follow on to the “How to do Vis” post that explains why we aren’t focusing on implementation).
D3 is a Javascript toolkit that helps with making web-based visualizations. It is quite popular and gets used a lot. If you don’t know what it is, don’t worry – the rest of this posting might not make much sense.
In this class, we will not teach you to use D3. At a high level, we won’t be “teaching” you about using any particular toolkit – we’re emphasizing design, not implementation. But D3 is a specific challenge that is worth mentioning.
If you want to learn D3, you have made a good choice: it is a good way to build mark-based visualizations that run in the web browser. And building stuff to run in the web browser is the right choice: it means your stuff is portable, easily shareable, embeddable in other content, and likely to remain compatible if platforms evolve (since legacy web support is important).
However, there is a cost: part of what makes D3 great is also its biggest curse. D3 is tightly integrated with all kinds of other web stuff, so it fits in well. But that means you need to know all that web stuff in order to use it. That means its hard to learn, unless you know a lot of other stuff first. At first I thought it was just that I couldn’t wrap myself around the modern idioms, but then I saw a number of students get stumped the same ways that I did (or worse).
We will learn about D3 in class. We’ll learn about its concepts, since they are interesting and a nice way to think about how to do implementation. We’ll try to understand where its kind of approach (mark manipulation) makes sense as a strategy. But, we won’t require people to actually try it – since actually trying it requires you to have a lot of knowledge first.
If you want to learn D3, I recommend the following path. Resist the temptation to skip to step 5. Trust me.
- Know a little about HTML, and how HTML gets implemented in the browser. You should be comfortable with the concept of the DOM. Knowing about CSS is handy, but not essential. This part is easy, since you don’t need to know much. And you can’t do #2 (Javascript) without it. (well, you can put javascript right into SVG files, but that’s another story)
- Learn Javascript. Javascript is an unusual language – it evolved rather than was designed, and the forces of necessity (it’s in so many web browsers that things can’t change) cause it to become an even odder language. The better you understand its unusualness, the better the chance you have with #3. Fortunately, the tools for Javascript have gotten much better (most web browsers have good debuggers). But even before that, I was able to get a class of CS undergrads to teach themselves Javascript and do amazing things with it (see the 2012 games class and the 2011 games class). My 2012 Javascript resources page is probably woefully out of date – but I still would recommend “Javascript the Good Parts.”
- Learn the selector-based programming idiom (e.g. get exposure to JQuery). Not only is Javascript a weird language, but people use it in weird ways. Some of these “idioms” are very pervasive. Selection-based programming is one that took me a bit to catch on to – since I tried to use it in D3 before I understood what it was really doing. I recommend figuring it out before you try D3.
- Learn SVG. SVG is the graphics infrastructure you’ll probably use with D3. Certainly, all of the tutorials will use it. SVG isn’t hard – in fact, I use it for the introductory assignments in the graphics class. There are some tutorials there (see “Getting Started with SVG”, but you might want to look for the tutorials before and after it as well).
- Now, find a tutorial on D3 and go through it. Most likely, it will work through an example that uses everything above. If you skipped to this step, you might get stumped by things like the selection idiom.
- Once you’ve been through a tutorial or two, try to take an example (there are lots – because the D3 community likes to share examples) and make it do something slightly different. This is a good way to figure out how the examples work.
- Try to make something simple yourself. You will find yourself going to the web to look things up a lot.
I think that learning D3 is worth it. People build really nice things with it. And having stuff run in the browser is a total win, if it’s practical for what you’re doing. If it’s not practical (you need to consider large datasets, or complex computations as part of the visualization process, or …), it might even be worth it to think of a client-server system where some back-end does the crunching and the front end runs in the browser (this is the preferred way to do things in our research group these days).
Note that D3 is probably not the thing to use if you aren’t going to do SVG-based (2D object-oriented drawing) graphics. If you’re doing images, or volumes, or 3D, etc. you might want something else. Also, if you want something standard (e.g. just a scatterplot), you probably want something a little higher level (D3 will help you draw the scatterplot, but the Google Charts API will just make one for you). Also, if you want to do something specialized, there might be a specialized tool for it.
What about things other than D3?
I single out D3 because its the one that most people ask about. It’s also really common. And, it’s upsides (working well in the browser, and being extremely general) make it worth the steep learning curve.
Generally, in other environments (than Javascript in the browser), I’ve tended not to use visualization specific toolkits. Instead I use the basic graphics and UI functions provided. So I don’t have much experience with vis-specific toolkits. (there will be an upcoming post on what I use myself – and why it might not tell you much). As I mentioned, there are a host of toolkits for doing more specialized things – I just can’t say much about them.
There is one that I should mention, since it is probably the granddaddy of them all. The visualization toolkit VTK. VTK has been around a long time. Its focus is on “scientific” visualization (3D, geometry, large data manipulation, …). It’s a big beast. It has evolved over the past 20+ years. The project was started by some Vis pioneers, and has grown considerably over the decades. It is free and open source, but supported by a company. So many big projects rely on it, that the project gets sufficient support.
Ironically, D3 and VTK have the opposite problems that cause them to be hard to get started with. With D3, there’s so much emphasis on fitting in well with all of the other pieces and standards, that if you don’t understand them all, you can’t do anything. With VTK, they went to the extreme of doing everything themselves – including having their own build system. Which meant until you figured out a zillion pieces, you couldn’t do anything.
My experience with VTK is (at least) 12 years out of date. I just haven’t had a project where I needed to do the kinds of stuff its good at. If you need to work with a standard “scientific” data type, and/or try a standard “scientific visualizaton” algorithm on it, its probably a good place to start. Or, try to find an application that has already been built on top of it.
