By watching the video presentation of play and sound made by Yunjia and I, I have learned a lot about those talented sound artists and various forms of playing sound. Among them, I was inspired by visual deprivation game.
"It is a from of gaming that focuses on audio rather than on visual representations. And the key element of this type of game is the use of binaural audio in order to allow the player to localise items in 3D space."
A related work Blindfold Flappy is mentioned in book “Loud and Clear: The VR Game Without Visuals.” The sound of this game is adjusted so that in the left audio channel a continuous tone of varying pitches represents the vertical height of the bird while the pitch in the right channel represents the height of the gap, which should be matched in order for the player to progress.
The theoretical basis of visual deprivation game is lies in sensory compensation-- "The process of neural and behavioral reorganization following sensory loss is known as sensory compensation. Typically, it is believed that sensory loss is followed by increased acuity of the intact modalities." For example, Blind people have more developed hearing than normal people. Therefore, when the game lacks of visuals, players will be more sensitive to sound, and it is easier to detect subtle changes in sound, such as changes in sound volume and changes in binaural audio.
The idea of this theme is to combine sound localization and physical scene to make a sound-based AR interaction puzzle solving project.
Use Pr and Au to change the left and right channels of audio.
Create QR code through QR code making website.
Use Procreate to draw AR scenes.
This is my first time to explore an AR work, and to my surprise it is not as difficult as I imagined. However, because I am not familiar with it, I spent a lot of time in searching tutorials and tools. And I have encountered many difficulties, for example, some tools cannot add sound to an AR animation, and the effect of it is too poor to satisfy me. I also tried to use Unity to make thie project, but since the system of my laptop and mobile phone are different, it cannot run smoothly on my phone.
At the end, by using QR code website, I successfully made the code. By scanning these codes with phone, the sound and animation will be played.
In addition, the name of each audio is a piece of code that represents the distance between the current position and the target position. So even if player cannot tell the direction by listening to the sound, the player can still locate the target by solving the puzzle of names.
This project was a successful attempt in the AR and interactive field. The experience of production gives me a basic understanding of how to make an AR work, what tools to make it, and what way to show it. If this project will be further developed, some aspects need to be improved:
-a large scene is needed. The site of the project needs to be further expanded. Large malls or garden maze are good places.
- multiple targets. More targets can be set, but the style of QR code is the same. Players need to listen carefully and distinguish the connections between those audios.
-QR code is everywhere. It can be pasted ont the ceiling or in the corner, and the size of QR code can be as big as iPad or as small as a fingernail.
Baas, Berend et al. “Loud and Clear: The VR Game Without Visuals.” Games and Learning Alliance. Cham: Springer International Publishing, 2019. 180–190. Web.
Pieniak M, Lachowicz-Tabaczek K, Karwowski M, Oleszkiewicz A. Sensory compensation beliefs among blind and sighted individuals. Scand J Psychol. 2022 Feb;63(1):72-82. doi: 10.1111/sjop.12781. Epub 2021 Oct 28. PMID: 34708412.
The power of sound as an art form. in New York Times Company [database online]. New York, 2013 (accessedApril 26, 2022).
Blindfold Flappy
By Wanting Xiao
Email Wanting Xiao
Published On: 27/04/2022