- Oct 2017
We can u-;e this mode to communicate representations of how something look~
What counts as a visual mode of communication can become a bit confusing in certain cases.
For example, authors of fantasy fiction these days rely heavily on the reader to imagine visuals. They carefully describe physical objects and scenes, much like in Prownian Analysis or our descriptions of the AIDS Quilt panels, to cause the reader to "see" things that they might have never seen or imagined before. I realize that using writing would technically fall under the category of the linguistic mode of communication, but this descriptive writing exists to evoke our visual senses. These authors depend on us visualizing the worlds that they create to tell their story. If this communication uses our "visual-spatial intelligence" according to the Theory of Multiple Intelligences, should it not be considered a mix of the visual and linguistic modes?
The visual mode refers to the use of images and other characteris-tics that readers see
I think that the importance that "other characteristics" can have as a visual mode is commonly overlooked. The layout of a webpage can communicate to the reader what to emphasize on, the color of an "Open/Closed" sign can communicate the status of a restaurant from beyond a legible distance, and the size of a specific visual object on a billboard can communicate what exact product is being advertised.
A good example of the utilization of visual modes of communication without the direct use of an image is visual poetry. This type of poetry is known for having dominant visual elements. A common technique of achieving this is using the words of the poem to create the perception of a shape or image.
This is a good example of a simple visual poem. The topic of the poem is coffee, hence the writer adjusted the layout of the text to create a shape. This shape is perceived by our minds as a coffee mug, even though there is no actual picture of one.
On an interesting side note: the reason we are able to perceive this image is due to the Gestalt psychology. This is a philosophy of mind of the Berlin School of experimental psychology which studies the self-organizing tendencies of the human brain (especially with images). According to this psychological phenomenon, our mind is always trying to find patterns in the data it receives and many times organizes sets of randomness in order to perceive them as a whole. This is why the seemingly random positions of the words in the poem cause our mind to try to find a pattern in the darker areas of the poem's white background, eventually leading us to seeing a coffee mug.
modes of communication
Interestingly, these five modes of communication correlate to Howard Gardner's Theory of Multiple Intelligences. In 1983 he published the book "Frames of Mind: The Theory of Multiple Intelligences," which outlined a model in which people's intelligence was not determined solely by their general ability (also known as IQ or the g factor). Gardner believed believed that types of intelligence were required to fulfill eight criteria:
1) the potential for brain isolation by brain damage
2) its place in evolutionary history
3) the presence of core operations
4) susceptibility to encoding
5) a distinct developmental progression
6) the existence of idiot-savants, prodigies and other exceptional people
7) support from experimental psychology
8) support from psychometric findings.
He then proposed nine types of intelligence that would satisfy these criteria: musical-rhythmic, visual-spatial, verbal-linguistic, logical-mathematical, bodily-kinesthetic, interpersonal, intrapersonal, naturalistic, and existential (The last one was added after the initial eight). The chart bellow shows the types of intelligence with visual queues and a simpler wordings.
It is interesting that these modes of communication can associate to what Gardner would consider different functions of the brain (Linguistic mode of communication with verbal-linguistic intelligence, visual and spatial modes of communication with visual-spatial intelligence, aural communication with musical-rhythmic intelligence, and gestural communication with bodily-kinesthetic intelligence). Perhaps this indicated that by including multiple modes of communication in our writing for this class (or in any writing), we are able to target more parts of the brain and keep the audience more engaged. This would also mean that according to the Theory of Multiple Intelligences, a multimodal text would have a broader appeal since people lacking on a certain type of intelligence would be able to understand the text through whichever mode of communication suits their strength and preference more. For example, a heavy text with many visuals would be more easy for "picture smart" people (as the graphic above would indicate) to connect with.
Sources: Gardner, H. (1999). Intelligence reframed: Multiple intelligences for the 21st century. New York: Basic Books.
Image source: Linkedin Blog - https://www.linkedin.com/pulse/theory-multiple-intelligences-potential-applications-silva-fca-
Katie Couric is an American journalist and author who is mostly known for her television news roles. She has a been a host on all of the "Big Three" television networks (ABC, CBS, and NBC). She has also worked as a daytime talk show host. such as being a "Today" co-host. She is also famous for becoming the first woman to anchor "CBS Evening News" alone in 2006 and being a "60 Minutes" correspondent. Some of her distinctions include being inducted to the Television Hall of Fame in 2004 and having a New York TImes best-selling book. She recently worked as the Yahoo! Global News Anchor.
The gestural mode, along with the visual mode, is another mode of communication that is deeply engraved in humans instincts. Scientific evidence even proves that the recognition of basic human expressions is universal regardless of cultural background. Scientific research has proven that there generally are seven emotions that all humans recognize through facial expressions: "anger, contempt, disgust, fear, joy, sadness, and surprise" (Matsumoto, 2008).
The image above shows sample images of emotions in a study done by Dr. Matsumoto which proved the universality of their recognition.
It may take too many pictures to convey the same idea quickly
I think the author is ignoring a key aspect of the advantages of multimodal communication. Using multiple modes allows an audience that does not have access to every type of mode (ex: a user with a muted device) or can not comprehend every mode (ex: a dyslexic person who has difficulty reading) to still receive the information being communicated. For example, in our Primary Source Descriptions we included images, which were by far the most effective way of communicating the description of the Quilt panel, but still included detailed text descriptions. This would aid people who can not load the images in our posts or are blind (therefore using the text through text-to-speech or Braille to receive the information). In conclusion, people should not choose a mode simply due to its advantage for a communication, they should also consider the audience reach that their mode would have. For example, in the 1930s an animated cartoon (without sound) would have done a better job at communicating a story, but a comic book (without speech bubbles) would have been easier to access.
The information is organized in map form (the spatial mode),
I find the use spatial modes to be extremely helpful when trying to comprehend things, especially data. If we go by the Theory of Multiple Intelligences, I might be considered a person who relies on their "spatial intelligence" frequently.
For example, when analyzing historical events in a class, I appreciate having an accurate timeline that is spaced out to scale. This way I can put important time periods or the long lasting effects of an event in perspective. I believe that people can not appreciate the length of the dynastic rule in China until they see how long (for physical timelines "long" is in the literal sense) it was and how many well-known events of the Western world took place during its existence. Additionally, people seem to forget how long the planet Earth has existed until shown the tiny portion of its timeline that the entire course of human history occupies.
Even just recently I remember using spatial modes of displaying information to help my research colleagues understand an issue. Since some data from a research paper we were referencing had data that was not consistently sampled, we could not accurately replicate their experiment to test their results and input data into deep learning programs. I decided to represent the data using spaced out visuals to explain my suggested method for predicting information within those data gaps.
The visual mode might be the most universal form of communication. Psychology and biology studies have proven that humans are hard-wired to react in certain ways depending on visuals. For example, the color red instinctively draws a human's attention since it is a sign of danger. That is why stop signs and traffic lights are red, they cause drivers to instantly be alert in order to avoid accidents. I think that because the visual mode is so universal, it could be used to solve the problem presented by James Conca in his article "Talking to the Future -- Hey, There's Nuclear Waste Buried Here!". Initially I considered a red sign with the classic skull and crossbones to signify a deadly threat. However the truth is that within human history the symbolism for death has been very different for each culture. For example, the Ancient Greeks never portrayed Death as a menacing or evil character, since they knew death was inevitable and sometimes even considered it the path to an honorable sacrifice or a peaceful end of suffering. Since we do not know if future generations will use a dark hooded figure, or a dirty old woman, or a skeleton wielding a scythe as their symbol for Death, we should probably stick to simpler design. I think red should be a main color (since it will evoke the human instinct for danger) along with black (since it has proved to be a basic color to represent evil in most cultures throughout time). Perhaps the simplest solution would be a depiction that would use the human empathy to communicate the hazard of nuclear waste, such as an exaggerated illustration of a suffering human.
Sources: Conca, James. "Talking to the Future -- Hey, There's Nuclear Waste Buried Here!" Forbes, 17 Apr. 2015, https://www.forbes.com/sites/jamesconca/2015/04/17/talking-to-the-future-hey-theres-nuclear-waste-buried-here. Accessed 2 October 2017.
One of my favorite sound effects in movies is called the "Shepard's Tone" and I consider it's impact as an aural mode to be surprisingly effective.
This video explains how Hans Zimmer composed the soundtrack of the movie "Dunkirk" using the Shepard's Tone and shows examples of it in other movies directed by Christopher Nolan or soundtracks composed by Hans Zimmer. In the video, the musical mechanics behind the Shepard's Tone are explained. From 0:41 to 1:28 the video analyzes the process of using "several tones separated by an octave layered on top of each other" to create an illusion of a constant ascending tone that never ends. This effect causes viewers of the movies the soundtracks are featured in to feel a rising sensation of suspense. I consider this to be one of the most powerful aural modes for communicating intensity and suspense in a soundtrack or movie.
volume of sound
The volume of a sound plays a big role on the importance we give to it over other sounds, and tha amplitude of the mood created by it. In the real world a sound that is constantly increasing causes us to instinctively focus our attention on it because it indicates that an object is getting closer. An example of this is our instinct to look in the direction of an increasing volume of engine sound when crossing the road, as a way of avoiding collision with approaching cars. Volume can also play an affect on how we perceive it. For example, the volume of a happy comment could indicate the level of excitement the person saying it is experiencing. Another example is the volume of dark music during a horror film. The viewer is able to recognize how eminent a threat is to the character simply through the increase or decrease of the music's volume. This communication of a "threat-level" can greatly impact the fear a viewer feels even when lowering the volume, by creating a sense of safety and then suddenly switching to a "jump-scare."