While it is very important to acknowledge when audio designers employ new techniques to design innovative new sound systems for the purposes of player immersion and showing off the virtual worlds as something more believable and relatable, it is also worth pointing out examples of interactive games that use audio and sound in new ways to help educate and teach the player about the characteristics of audio in order to help the audience appreciate sound at much deeper levels than before.
One such educational game project is called 3 Deaf Mice, headed by David Sonnenschein (sound designer, film director and, more recently, audio consultant for Mass Effect 3) and it’s main aims according to Sonnenschein himself on the game’s webpage were to help develop the player’s listening skills and mindfulness, and to help them discover acoustic sources, waveform shapes and musical meaning within the world of sounds. Within the game, the player helps three mice – who are all deaf musicians after playing loud music too much – create and piece together their newest song by means of recognising different qualities of sound, referred to in the game as Source, Shape and Meaning (Nair, 2013) – training the player in causal, reduced and semantic listening modes respectively. The player will first identify the source of the sound and its origins. Then the player identifies the shape of the sound wave by defining the pitch, timbre, reverb and other properties of the sound. Finally, the meaning of the sound is investigated as the player manipulates the sound using various DSP (digital sound processing) transformations to create a specific result and a piece of the final puzzle. This also delves into the areas of phoneme creation, exploring the manipulation of sound in order to create recognisable phonemes which come together to create syllables, words and eventually full lyrics for the final song. In the final puzzle element, the player will place all the audio components generated into a final mix of the song’s verses and tracks. There are ten verses and ten audio tracks to unlock, with one verse and audio track unlocked whenever one of the ten levels is completed.
The basic structure of the 3 Deaf Mice music game. Identifying the sound source is first, followed by an analysis of the shape of the sound and then transforming it into an audio component used in the final creation of the song.
Looking at the aims and objectives of this interactive project one might find it hard to simply remove the labels of ‘educational game’ and ‘creative toy’ from this product. But as explained there is definitely an element of gameplay and progression in terms of unlocking each of the verses and audio tracks in the final song and in terms of the process in unlocking the verses and tracks. The player is taken through each of the three stages of identifying the source, finding out its shape and understanding its meaning at a pace that is entirely dictated by the player, giving them time to properly employ the required listening modes to solve the gradually harder sound source and shape identification puzzles. As Sonnenschein also says, one of the challenges in making this game was to make it engaging in some way, not just “a learning process, so the player is motivated to keep going and rewarded each time they accomplish a level…” (quote from interview published by Nair, 2013). One of the bigger motivational rewards in playing through 3 Deaf Mice was the ability to freely interchange and mix the final song once all the verses and audio tracks were unlocked. The player would be able to rearrange each of the individual sound components to create their own versions, allowing for free compositional thinking within the game’s interface.
REFERENCES AND FURTHER READING
Nair, V. (2013). “3 Deaf Mice: A sound-music game”. [ONLINE] Available at: http://designingsound.org/2013/04/3-deaf-mice-a-sound-music-game/. [Accessed 01 December 14].
Sonnenschein, D. (2013). “3 DEAF MICE music game with Sonic Treasure Hunts and Audio Puzzles (part 1)”. [ONLINE] Available at: http://audiocookbook.org/sonic-treasure-hunts-and-audio-puzzles-in-3-deaf-mice-game-part-1/. [Accessed 30 November 14].
Sonnenschein, D. (2013). “3 DEAF MICE – sound and story in a game – Part 2″. [ONLINE] Available at: http://audiocookbook.org/3-deaf-mice-sound-and-story-in-a-game-part-2/. [Accessed 30 November 14].
WordPress.com. (2013). “3 Deaf Mice: Open your ears, open your mind”. [ONLINE] Available at: http://3deafmice.wordpress.com/the-game/. [Accessed 30 November 14].
https://www.youtube.com/watch?v=lSH4a-0S-ng – 3 Deaf Mice Music Game/Kickstarter Video [Accessed 30 November 14]