Q: All these terms, mental models, schemas, coordination, etc. they are pretty fuzzy and not definite. I believe that to understand how InfoVis works we must understand how the brain works.
A: First, understanding a phenomenon often involves multiple levels of description and explanation (e.g. the famous AI researcher David Marr proposed three levels of analysis for information processing problems). I do not mean to completely dismiss the relevance of neuroscience for InfoVis (I seemed to do so in a previous post). Rather I just believe that a conceptual level description about the nature of representation and interaction in InfoVis can provide a more direct and relevant account. Ultimately internal representations have to be grounded at the neuro-physiological level, but neurological level explanations need to be constrained by conceptual and behavioral evidence too.
Second, what is meant by “how the brain works”? The undertone of this phrase is that the brain can be understood as a machine in isolation from the body and from the environment. This view is being increasingly questioned and criticized. Of course one can argue this is just a matter of opinion and assumption. I believe in a holistic approach of understanding InfoVis processes enabled by brains coupled with bodies embedded in a socio-cultural environment mediated by language, visual images and cultural practices.
Q: Theories of mental models, distributed cognition etc. have been around for a while, I saw no direct application or immediately usable results. Why do we care?
A: It might just be that these concepts or theories are not fully mature yet, or maybe these concepts can only be useful when combined with domain-specific investigations. We the InfoVis researchers can appreciate an incremental result in an interaction technique or a layout algorithm, but we do not have patience for the potentially slow development of theories. I believe we should play an active role in further developing existing theories that sound promising and relevant to InfoVis. We seem to be comfortable enough in considering InfoVis as an applied area and leaving theoretical questions to psychologists and cognitive science, but as Newell and Card nicely put, “nothing drives basic science better than a good applied problem”.
Q: What about perception? Isn’t it the most important aspect of InfoVis?
A: Perception is important, but perception is not a process where we take a snapshot of the visualization and then decode every bit of information inside. Empirical studies have shown how astonishingly little information we actually attend to and are aware of (e.g. the hidden gorilla). Perception, to me, is the active sampling and foraging of information, and hence must be understood in the context of dynamic, continuous loop of interaction.
There are of course qualitative aspects of visual sensation to be understood. Understanding how we perceive color difference or size difference, for example, is helpful, but when users are not certain if two colors are really that different, they often can just mouse over to get the exact values represented by the colors and do a comparison. Even in InfoVis, the foraging of information often is not purely achieved through non-textual visual means. Hence perception must also be understood in conjunction with action so that we know how they complement each other.
[Have better questions? Don’t agree with the replies? Let me know!]