For Tuesday, March 2nd, the Reading is Chapter 3 of Visual Thinking for Design.
This week, the reading is intentionally light so you can be more focused on the Design Challenge.
Before 8am on the 2nd, please make a comment about this Chapter as a comment to this posting to help structure our conversation in class.
{ 14 comments }
I was surprised by how little emphasis is put on determining depth. I understand that it is harder for the visual system to determine, but I disagree with the statement that we need to move physically to determine depth. I can view a photo someone’s taken and be able to determine with some accuracy, how far away things are to the camera. Is this because I’m not actually determining depth, but using previous knowledge and pattern matching (on shadows, size, etc.) to get an estimate of the depth?
I all ready knew somewhat about visual edge detection. In Understanding Comics, McCloud makes the assertion that humans don’t keep full faces in visual memory, just outlines, things that are like cartoons. This seems to make sense considering how strong edge detection is and with our ability to recognize cartoons of objects and faces as what they are intended to be. I am interested in going forward to understanding how pattern matching works together to combine into a recognizable object. For example, if I were to be shown a dog of a breed I had never seen before, I would still be able to recognize it as a dog, most likely from matching features that are dog specific, but these features must be formed from the underlying pattern matching my visual system is doing.
The idea of graphical code translating into semantics interested me. It seems somewhat obvious that I would have overlooked it, but it is clearly important. Some things, like a big objects translating to a larger quantity seem clear, but I would have automatically assumed any ordering of objects was from left to right, but that’s informed by the fact that I grew up with the western writing system and people who tend to read right to left might have a different semantics attached to that graphical code.
I found it very interesting that our eyes aren’t as interested in depth as I would have thought. There seems to be a big push towards 3D in entertainment and again I have to wonder if it’s going to make that big of an impact seeing as depth isn’t really that important to our interpreting pictures.
I think the big design concept to take from this chapter is that at some level, everyone has the same ability to detect certain simple patterns. Playing to this universal ability seems like it would be a good design principle as it would allow for successful use of the visualization over the broadest audience. It also brings up the point that some patterns are only recognizable by people who have a certain amount of domain knowledge and that some visualizations simply won’t be understandable by people without that knowledge.
Our ability to recognize distorted patterns is really interesting. It seems like the reason this is so difficult to do in the computer is that the computer doesn’t have the parallel hardware required to instantly discern the low level patterns present in an image. It will be interesting to see how camera technology evolves and whether or not it evolves to match the pattern matching hardware of the eye/brain. It would be kind of neat to have a camera that not only captured a picture, but also output pattern data that could be used to quickly add meta data to an image. It would at least cut down on the amount of time people spend taging face book pictures.
Visual Thinking: Chapter 3
**************************
This interesting chapter on “Pattern Binding” has many points to ponder.
1. Why neurons excite ?
Brain constructs contours and area with the help of coordinated efforts
of billions of neurons.The process of excitement of neurons, the firing process,
passing electrical energy and coordinated alignment creates a pattern
which we call “Edge Detection”. This chapter explains very nicely “How”
it happens, but doesn’t explain clearly “Why” is happens. Why the
neurons get excited and answer to this questions may throw deeper
insight into working of brain.
2. Dog detection a complex pattern searching ?
I am not fully convinced that identifying a dog with “generalized contour”
may need the involvement of V2 and V3 cortex. My guess is that human can
detect dog or cat as easily as detecting lines, points (less than 100ms).
which happens on V1 cortex ( pre-attentive stage ).
3. Are patterns hard wired and static ?
We learn patterns we as grow older and get more experience and the complex pattern searching depends on individual skills. But one question that this chapter
doesn’t explain is whether patterns moves from higher level to lower levels
as we become more experienced ? If we see Michelangelo pictures 1000 times,
shall our brain always work with V4 Cortex with that image ?
4. Preferred axis ?
What is our preferred axis ? ( page 47 )
I’ve got the older, fatter version of Ware, so my Chapter 3 was all about luminance, brightness, and contrast.
My big surprise in this chapter was that, after a long discussion of how the visual system perceives luminosity (I had no idea how huge the difference in available light was from standard interior settings to outside on a bright day — it really is remarkable how little we notice that), Ware kind of leaves things at “with all these artifacts and illusions, it’s a bad idea to encode quantitative data with luminance.”
However, I suspect that wherever our visual system wants to cause an illusion, we’ve actually got an opportunity to use that trick to cue the viewer to something. Illusions need not be bad, as long as we’re aware of where they’re going to happen.
In addition… using the visual models listed in the chapter, it should be able to model and describe optical illusions before even drawing them! Which would be SUPER SWEET.
Ware’s chapter 3 focuses on patterns as a semantic structure of visual language.
Ware starts off by noting that we see the world mostly in 2.5D instead of 3D. Image plane sampling is orders of magnitude more efficient than depth sampling. This stunted view of depth sampling is what merits less than one D. The ‘what’ pathway identifies the object. ‘Binding’ makes disconnected pieces of information into a connected whole. I was a bit disappointed that Ware didn’t provide a more extended treatment of depth. My intuitive sense is that depth would be very important for our survival because of the stereoscopic quality that it would bring to our vision.
The generalized contour mechanism enables us to identify discontinuity in edges. This is a very efficient mechanism, explaining why it is that we are able to recognize shapes represented by line drawings so effectively.
Orientation, size, contrast and color are the properties that allow us to discriminate between textures.
Motion patterns are just as important as static patterns, as they tell us about the layout of the objects in space.
Patterns discerned in V1 and V2 are common to most everyone, while patterns discerned higher up (V4 and IT) are more likely to be learned.
When a pattern can be recognized in a single, apprehendable chunk, it can be recognized easily. However, when it has to be broken up into multiple, smaller chunks, then each chunk has to be recognized and kept in visual working memory.
Our ability to recognize patterns in spite of distortions is fundamental to visual intelligence.
Spatial metaphors are fundamental to the way language works.
The first part of this chapter is quite interesting for me. I find the concept of 2.5-dimension clearly explains the world in our eyes. I did think that the picture we see are 3-dimension but it might base on much previous knowledge and information, like light, shade and experience. Therefore, to determine depth, we need one more degree, movement.
One of the ideas in this chapter is about the patterns for design. Since most designs are mixed by learned symbolic meanings, apprehended patterns can make a design visually efficient. Two types of patterns are discussed, graphic ones and relationship-defining ones. I do not quite understand why graphic patterns are believed to be better in expressing abstract ideas, since it seems to me that relationship-defining patterns can be sometimes more clear.
I like the examples about Line Graph, which expresses trend, comparison and uncertainty in a simple and understandable way. My first glance at the figures match the queries listed below and I believe such technique can be used to present data analysis with certain emphasis.
This chapter is about how we perceive the world. Ware makes the argument that we actually see most things on a mostly 2 dimensional manner. He goes on to talk about how our visual system has special machinery to detect edges, or contours as he calls them. While we are specially good at detecting these contours when they are represented by lines, even changes in textures can create very obvious boundaries. I think part of the reason that we have evolved to be so good at making those distinctions relates to the claim that we mostly see in 2 dimensions.
Since we have so much more information in the up-down and sideways dimensions, our brains have grown to interpret certain types of visual discontinuities as depth cues. If it weren’t for this, we probably would not be able to perceive things being “3D” when drawn or photographed.
He also talks about how there are different types of patterns that are easily recognized, and as the complexity goes up, the region of the brain that does the recognition moves from V1 to V2 and V4, IT. Also, we are able to recognize patterns at several different scales.
Another point is that it is important to use patterns that have semantic meanings that relate to our actual experience of the world. There is a nice list of some such semantics at the end of the chapter.
Now I get more clear with the structure of the whole textbook. Colin ware is now deconstruct “how we see the world” into elements such as pattern and color. All the chapter 3 is about the principles of pattern finding in human cognitive system, with some design recommendations according to the mechanisms.
I really like the 2.5 D space discription of the visual property of human eye. There are difinitely large difference between the “away” (in our domain, we call it ‘longitudinal’ dimension or the ‘z’ direction) dimension and the “up-down” (tranverse in our domain) and “side-way” (lateral) (or X-Y plane) dimensions. However, I think the ability of our perception for the “away” dimension is not as bad as the author declared. A good experiment might be put 3 identical simple objects (circular or rectangle boards) in different positions along the “away” dimension, and let our visual system determine the relative positions. One might argue that our visual system might determine the position by the size of the boards in the x-y plane, but we can modify the experiment as we can adjust the size of the boards so they seems the same size for the viewer even at different positions. If we can still tell the correct position relationship, our sight in the ‘z’ direction might not be so bad.
I really like that the author gives some suggestions and methods in design for proper pattern finding, especially the line graph examples. The way to use uncertainty as the shadows surrounding the center line is a very good way to show the difference and relationship between the two sets of data. However, I’m a little bit confused by how they construct the graph. The pattern of the uncertainty is obviously not identical to the weather tempareture data, so what did they do to generate the fourth graph in the bottom left figure?
When we see an object, up-down and sideways combined into 2-dimensional image and a depth is indirectly inferred using away dimension. Pattern-processing is done mostly in this 2-dimensional image and it tells us about the object and relation between objects through finding the boundaries of objects. Binding continuous contours and area generates the generalized contour and this is why line drawings are good enough. Texture difference (texture element orientation, size contrast, color) also enables us to discriminate.
For more complex patterns, what pathway is used from V1 to IT. As the processing moves up to IT cortex, individual difference becomes large. Other than finding contours and regions, we perceive a pattern by grouping objects by spatial proximity. When the pattern is complex, we see them in multiple chunks, therefore we do a series of fixations.
Overall, when designing, we can implement connecting contour, enclosing contour, common color region, proximity grouping, alignment, common texture region, common movement to make a relationship between graphical entities. We can also synchronize meanings (semantics) and graphical code.
Reading through the chapter, I found the concept of RSVP (rapid serial visual presentation) to be an example of “cool” biology. The idea that the human perceptual system is efficient enough to process large amounts of data at rapid speeds is somewhat mind-boggling. It makes me wonder what subconscious processes must be at work to facilitate such perceptual feats.
The discussion of silhouetting brought about another interesting point. A lot of current visualization literature focuses on clutter reduction: essentially removing extraneous data from a visualization. At the basis of this theory lies the idea that simpler is better. The idea that an object is most recognizable by it’s silhouette seems to provide a perceptual argument to the theory of simplicity, at least in part due to the fact that a silhouette is such an extreme abstraction from the potential complexity of the surface it represents. I am curious, however, how it is the concave features that make the silhouette most distinguishable.
This idea of silhouetting and simplicity appears to draw on the principles behind FACS and Chernoff Faces. Combining this theory on simplicity with the strength of innate human facial recognition appears to further the idea of the strongest perceptual techniques are those which will present the user with the simplest image.
On a slightly divergent note, I found it interesting that Ware discusses contour, lighting and texturing as three basic perceptual depth cues. Although I’d seen lighting in such a context before, texturing and contour are far less obvious choices. However, Ware brings up interesting examples to which both apply and make me wonder if the illusion of depth in 2D images is really possible without at least some pairing amongst these properties.
**Comments with respect to Ch 7 of Information Visualization (my best guess at the overlapping data)
An ontological discussion would have been a welcome addition to this chapter. The significance of “states of existence” within visual perception is sometimes overlooked, which can lead to misinterpretation. For example, patterns can be perceived as existing when, in actuality, there may be not enough information to sustain a pattern test for an acceptable sample size. Ware chooses to focus on the pathways that identify objects, but it helps to consider the mental association that individuals create, such as the properties of a represented object (in the symbolic sense) and identity that is applied by the viewer. When considering the process of object identification, the relationship between the signifier and signified seems almost as important as the semantics behind the spatial metaphors that Ware describes.
I was particularly impressed at how good a job the human visual system does of separating overlap from different regions by texture. (figure on p. 52) It kind of drives home that overlapping contours are something that our unconscious visual processing system does not handle, but that overlapping textures are something it does. This is a bit surprising in that textures are kind of hard to detect, relative to contours, but on the other hand contours are more localized, so when they overlap it’s harder to keep them separate.
It also seems that what makes “patterns” distinct from “features” is that patterns are invariant to some types of warping, and to a lesser extent (if you ask me) that the underlying features are interchangeable. (e.g. generalized contours.)
Here is a quote from the reading that I think pretty much sums up what information the author is trying to convey.
“The design challenge is to use each kind of design device to its greatest advantage in providing efficient access to visual queries”.
The efficiency of visual queries can be enhanced by exploiting the easily apprehended patterns. The book chapter gives enough details for understanding how brain interprets pattern and objects (what pathway) from basic features. The main challenge for a designer is now to effectively assign the semantic meanings to these patterns pertaining to its problem domain.
Basically by implying that visual depth perception requires much more cognitive energy than seeing in 2D image planes, Ware emphasizes the importance of pattern recognition based on texture and contours. This is about finding crucial contours and making meaning(‘semantics’) of what one has seen with the least information possible. It is not difficult see why this feature is so well-developed in human vision, because figure-background recognition by finding a visual pattern is a fundamental function for evolutionary survival(as a the famous skeptic Michael Shermer says, “the brain is wired to find patterns by default… prone to see a predator in the grass when there is not rather than risking not to see a predator when there is and get removed from the gene pool”). Although Ware directly mentions that cognitive processes do not work in a way that higher information processing ‘looks’ at lower processes but that several processes wrok together, it is fascinating to see how almost innate some of the pattern recognition capabilities seem.