“When people talk about augmented reality, they usually think it’s about visuals that are put on top of the camera image,” Breiden Bruecker explains to Gamasutra. “It’s always a visual thing. But not many people think that the same can be done with sound.”
Our ears follow two functions, hearing and listening. Hearing is the input of soundwave data and listening is the filter through which that data is passed to turn the data into information so the brain can interpret it. It is the same with our eyes where we see everything but look at specifics. Between light-waves (seeing) and sound-waves (hearing) we formulate our reality. Touch and Smell refine that reality and taste is a bit too specific to warrant inclusion here.
Hearing is one of the human body’s major senses. Beyond this, as it is so closely interactionary with the brain, hearing/ audio is one of the two main streams of learning (Mayer, 2002). The Cognitive sciences have put a great deal of research into this; referring to it as sensory memory (p17-24 .Brunning, Schraw & Norby (2011) modality (p223) or Cognitive Load Theory (p223, Paas, Renkl & Sweller (2003 ). It is often considered that telling someone as you show/ demonstrate is the most effective learning method (Mayer, 2002, 2008). In this way the media arts spectrum, where sound design and visual design have come together, has the potential to be a critical mass of communication. Thanks to computers, televisions and games the vast majority of our recreational time is spent enveloped in audio-visual fantasies. Thanks to organised sound (or what Attali, in Cox & Warner (2003), defines as music) the larger world-scape can be “consolidated’. Also in Cox & Warner is Russolo’s Manifesto where he explicitly includes non-musical sounds to create a less restrictive musical sound.
Sounds have the capacity to transport us anywhere at any time without leaving our seats (unknown). It’s been my experience that books can also transport us but the process of reading is complex, it is very difficult to meditate while reading through the guide but an audial guided meditation can get you meditating in minutes. Kastbauer (2013) discussed an experience he had while reviewing the audio-work for a game he was producing. He marvelled at its capacity to make what was visual real, to turn the sights into an experience. Turner (2015) supported this at GDC Europe when he discussed how the audio plays a role in how a game “feels”. Diamante (2007) wrote, “Recently, Kondo [(Sound Creator for Nintendo)] has been playing with music actually driving the actual gameplay.” To this end it seems that it would quantify that Audio, and Sound in general, plays a critical part in the effectiveness of Media arts; especially when in conjunction with visual elements.
Alexander, L. (2011) Dimensions; Augmented reality purely through sound. http://www.gamasutra.com/view/news/127976/Dimensions_Augments_Reality_Purely_Through_Sound.php 23/8/16)
Bruning, R. H., Schraw, G. J., & Ronning, R. R. (1999). Cognitive psychology and instruction. Prentice-Hall, Inc., One Lake Street, Upper Saddle River, NJ 07458.
Cox, C., & Warner, D. (Eds.). (2004). Audio culture: Readings in modern music. A&C Black.
Kastbauer, D. (2013) Envisioning our interactive audio future. http://www.gamasutra.com/view/feature/195324/envisioning_our_interactive_audio_.php (23/8/16)
Diamante, V. (2007) GDC: Koji Kondo’sInteractive Musical Landscapes. http://www.gamasutra.com/view/news/104002/GDC_Koji_Kondos_Interactive_Musical_Landscapes.php (23/8/16)
Mayer, R. E. (2002). Cognitive theory and the design of multimedia instruction: an example of the two‐way street between cognition and instruction. New directions for teaching and learning, 2002(89), 55-71.
Mayer, R. E (2008). Learning and Instruction (2nd Ed.). New York: Pearson.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational psychologist, 38(1), 1-4.
Seitz, A. R., Kim, R., & Shams, L. (2006). Sound facilitates visual learning. Current Biology, 16(14), 1422-1427.
Turner, J. (2015) Video: How great sound design can make your game “feel” better. Presented at the Game Developer’s Conference Europe 2015 sourced http://www.gamasutra.com/view/news/253265/Video_How_great_sound_design_can_make_your_game_feel_better.php (23/8/16)