When the Soundtrack is the Game: A Brief History of Sonic Games

In most games, sound plays second fiddle to the visuals. There is always much talk about the graphical rendering power of a computer or how beautiful and/or smooth the visuals are in a new game. There are of course games that combine sight and sound but where sound plays a larger part than normal — for example Guitar Hero, in which the user attempts to play along to popular music with a fake guitar. However, these could still not be described as “audiogames”, as without the visuals they are unusable. The emergence of the iPhone and other smartphones has led to a number of innovative mainstream games being produced which are sound only, such as the Papa Sangre series. In fact audio-only games for people with sight difficulties has a long history going back to the 70s.

1970s

Audio games as a concept have been around for many years. The first reported audiogame sold was Atari’s Touch Me game in 1974. This was a memory game where a series of tones would play, and user tried to recall the tone sequence and press appropriate buttons to mimic it. Although it was also a visual game (lights flashed in concert with the tones), the game was simple enough that it could be played by people as a pure audiogame. A similar handheld game was released called Simon in 1978 which was even more popular.

1980s

In the 1980s a number interactive fiction or adventure games were being developed. These involved users being given text descriptions of a location (“you are in a large cave” or “you are on the bridge of a spaceship”, etc.) The user could then enter text to say what action they wanted to do (“go north”, “pick up gun”, etc.) The gameplay normally involved a number of puzzles presented in descriptions and solved by entering a series of commands. By connecting these games to a text-to-speech interface, they could be made playable by sound only.

1990s

Real Sound — Kaze No Regret was a game released for the Sega Saturn and Dreamcast consoles in 1997 and 1999 respectively. Like many audiogames, it was designed for people who are blind or partially sighted. It can be thought of as an interactive radio drama. The user listens to segments of story and the then game chimes and then can make selections which decide how the plot unfolds. Audio Space Invaders was a PC game that used 3D audio called ambisonics, designed for a four speaker system, but which can apparently work adequately on a two speaker set-up. The game play consists of shooting flying invading aliens. It does not require a graphical interface to be played but, like many audiogames, provides a graphical interface in addition. This is a directional audiogame — much of the game is based on the user hearing where the enemy is: to their left, right, in front, behind. Different sounds represent different types of enemy ships, the Doppler effect indicates a ship’s direction of movement, and pitch represents closeness to the player.

2000s

The Blind Eye was released in 2000 by The Blind Eye Research project. Once again the focus of the game was on binaural 3D sound rather than speech sound. Binaural sound enables the simulation of surround sound on a pair of headphones / earbuds. The game is a timed search-and-collect task set in a city environment. The user must find a series of musical instruments hidden in locations whilst avoiding cars or colliding with walls. The city sound environment is simulated for the user. In addition the user’s footstep sounds change as they walk on different types of surface.

Shades of Doom, released in 2001, is an audiogame in the spirit of Doom, a new first-person shooter built from the ground up. It uses surround sound, and the user can hear things like the sound of wind in passages, footsteps echoing, and nearby equipment. There is also an optional voice-based environment analyzer. AudioBattleShip is also designed for people who are blind. Once again 3D sound is used to implement the game, in this case the traditional battleships board game. Players are can hear the direction of their bomb drop, together with the result. A tablet is used for input, and sonic cues can also be given to remind the user where they have placed their ships.

Drive is an audio-only “racing” game released in 2003. However rather than focusing on steering (like most racing games) it focuses on speed, which the player must maximise. The user drives their car over power-ups spread along the track which they pick up and activate with keypresses. As they speed up the power-ups get harder to pick up. The speed also causes a musical soundtrack to change tempo and a co-pilot’s voice to change behaviour. There are various other sounds in the environment designed to distract the user from their task, such as doppler-effected sirens and helicopters.

Another driving-related game was made by the company that made Shades of Doom: GMA Tank Commander. It is based on the classic arcade game. This is another surround sound game. It includes audio targeting systems, the sounds of approaching tanks, environmental ambience and audible recommendations from a tank crew.

Terraformers is a sci-fi action and adventure game which uses 3D sound to provide a sonic compass, and sonar to provide a sense of distance from objects in the direction the player is turned. There is also voice guidance. Footstep sounds indicate what the user is walking on. The player has to walk through different rooms and levels solving problems, and avoiding robots and other danger, to progress through the story. As mentioned, early audiogames were often based on text adventures. The Last Crusade is a text-based role playing game (RPG) released in 2004. The game speaks out text but also gives useable items their own name sound. They also have a unique sound when they used by the player. Similarly with mobs and non-player characters.

Some research has been done on extending non-audio games into the audio realm. An example — in the spirit of Shades of Doom — is AudioQuake, which does not build a game from the ground up, but aims to make the groundbreaking first-person shooter Quake playable without visuals. The player has certain extra objects, a radar scanner, and an item-detection scanner. These give audio outputs and alert the player to an object or event. A more obvious candidate for audiogame conversion are music games designed for sighted-people, such as dance games and games where the user plays a fake guitar to attempting to synchronize with the music.

In “Finger Dance”, a new game rather than an audio overlay, users listen to music, and try to match the rhythm with key press patterns using four keys, responding to different types of drum rolls. There is also an audio-based menu system. Another audiogame from the music/dance genre is AudiOdyssy. The player is a DJ and uses the keyboard or Nintendo Wiimote to play dynamically generated sounds. The aim is to trigger them in time with the current song playing to excite the virtual audience on the dancefloor. Stereo spatial cues tell the user how to move the Wiimote.

Speed Sonic Across the Span is a platformer audiogame that uses voice, auditory icons and earcons. Objects are represented to the left and right of the player by using stereo panning. The player can jump, walk and stand still. Jumping causes the pitch to go up and then down and the player lands with a noise. A voice announces which platform level platform the player is on (1, 2, 3 or 4). Running or jumping into a solid surface causes vocal sounds to trigger. A form of sonar helps the player find platforms in front of them. The sounds of enemies (bees and dogs) are panned so the player can tell what direction an enemy is approaching from.

Blind Hero is a music rhythm game — but unlike the earlier Finger Dance, it is an overlay on the Frets on Fire, a game similar to the well-known guitar hero. Blind Hero involves a specially-designed haptic glove that the user wears. The glove gives vibrations on different fingers to indicate that there are notes arriving that need to be played. As well as engineering the glove, much work had to be put in to design the timing of the vibrations, which indicate both how long to press the fake guitar buttons and how long to press them.

2010s

The release of the Apple iPhone had a significant impact on audiogames as it created a powerful piece of technology that was heavily headphone based, and had a very small screen. The 2010 game Papa Sangre was an audio-only released for iPhone which won or was nominated for a number of gaming awards, and fascinated both people who were blind and people who were not. It is based in a supernatural world. The player moves by tapping the screen as a footstep metaphor, and turns by turning the phone. The game uses binaural audio to create a surround effect in the headphones. The player must avoid hazards, solve problems and collect musical notes in the environment. A sequel was also released. Sound Swallower, another iPhone game, was published in 2011 and is an audio-only game that uses the device’s GPS. The player must avoid the Sound Swallower and run to collect sounds from around them before they are erased.

The Nightjar is an iPhone and uses the same engine as Papa Sangre. Rather than supernatural world it is based in a science fiction environment, on a space ship. It utilizes the voice of the ship’s computer and a separate voice guide, sometimes they deliberately contradict each other. The game received much mainstream publicity partly because the guide voice was provided by the popular actor Benedict Cumberbatch. Blindside was a post-apocalyptic themed adventure story game released in 2012. It is compatible with Windows and Mac as well as iOS. In the game the player wakes up to find that they, and most of the people in their city, have gone blind. The player must navigate the city streets avoiding monsters, and discover more about what has happened. On the iPhone screen touch allows the player to walk around and phone tilt controls other movement and direction.

Released in 2014, Audio Defence: Zombie Arena was a more fast-paced game than some of the iPhone puzzle solvers. It is a 3D sound first person zombie shooter. The user can hear the zombies approaching around them. They must turn to the face the zombie and fire a gun to destroy them. The user can select different guns with different shooting sounds and abilities, including a sonic cannon.

Moving back to the adventure/puzzler type iPhone games, A Blind Legend, is an audio only action adventure game. The touch screen is used for general movement, and for combat actions. There is also a companion who provides a voice guide, saying things like “Look out — behind you!” A related game, but developed for PC, is Legend of Iris. It is a game for children who are blind, but has a specific non-entertainment function. Its aim is to help train the children’s navigation skills. Although it is designed for audio only A Blind Legend uses Oculus VR to take advantage of the headset’s directional audio system. Thus players can move their head to change the sound direction.

The adventure game involves a number of puzzles inspired by Legend of Zelda and its goal is to get a giant out of a spirit village. Solving puzzles involves practicing skills such as locating the origin of a sound and then remembering that location, and following moving objects by sound only. Interestingly the game was designed in collaboration with a composer who lost his sight at the age of 14.

Looking over the last few decades of audiogame releases and research, some patterns can be discerned. There is often a direction-based component, and the worlds created are usually attempts to simulate (even abstractly) a real world. Another common method is the use of speech to build environments. Where music is used, it tends to be a background element, except in the music/dance games. It is interesting to note that games such as AudioQuake and AudioBattleShips were partly built to provide extra social activities for people who are blind, and that Legend of Iris aims on providing an educational tool.