APD Week 9 - Rock Opera

This response was made in collaboration with Helen Kwok and Benjamin Mansur.

Theme

Materiality

Method

Ben, Helen and I will be creating an interactive piece that plays specific sounds when the user touches physical objects.

Context

The strongest influence on this response was Mengyang Wang’s OutsideIn Capsule. Helen was particularly taken by this concept and made a connection between it and the meditative soundscapes she listens to with the app Calm. The tactile interactions in Wang’s OutsideIn Capsule really inspired our response a lot.

Through tabletop role-playing games I have had some experience with websites such as A Soft Murmur which allow the user to create their own atmospheric soundscapes with different toggles and switches to play audio loops. We drew on these ideas to create our own physical keyboard to mix the material and the immaterial.

Staying mindful of Costello and Edmonds’s pleasure frameworks we wanted to attempt to explore sensation, through tactile interactions; discovery, by having the player learn what sounds correspond to each object; exploration, by having the player combine these different sound elements together.

Response

We began with the idea of using specific objects matched to specific sounds, such as seashells making ocean sounds.

For our prototyping we found some rocks outside and used those. The rocks also lended an element of exploration and discovery to this piece, as it invited the user to touch them to find out what sounds relate to each object. Helen coded up a prototype with the MaKey MaKey that allowed the player to complete a circuit by touching two wire components on each rock, which would then play a sound and an image of the location it represented. When another rock was touched the first sound and image would stop and the new one would begin.

After playtesting we received some feedback that confirmed some ideas we had been considering. We added the ability to overlay the sounds over each other, which really brought things together nicely. It also prompted us to include shorter sounds to allow the player to make the atmosphere more their own. We found that using the sounds we had already sourced and edited we had made the building blocks to a rainforest soundscape, which was really quite beautiful to listen to all at once and have some semblance of control over.

Reflection

Reflecting on this response I am really excited about the potential future of this idea. I can imagine it working quite nicely in a meditative environment, where a user would make their own space that is unique to them. I think that is a rather nice idea and would like to explore it further.

The relatively small number of alligator clips in the MaKey MaKey kit did limit the scope of this response, however that might be for the best in that it kept our prototype very focused. With a larger amount of materials available we could expand on the framework of discovery that currently exists in a limited fashion in the current piece.

We were also limited by the objects available to us, and through playtesting received feedback that the piece may feel stronger if the objects corresponded more directly with the sounds available. I could also foresee a sort of puzzle game where the objects are mismatched and need to be rearranged to fit their sounds. This might be fun to explore further in the future.

Collaborating with Helen and Ben was great fun. It felt really good to be able to each bring our own skills to the table. For one example, we had an issue with audio clips we had sourced clipping and taking ages to load, and one of them was overly loud in comparison with the others. Having a lot of audio editing experience I was able to very very quickly resolve these issues and shoot them back to Helen, who was able to then make changes to the code to reflect these updates.

The way in which this response touches on previous themes is really interesting to me. As well as materiality it connects with observation, objects, and some elements of place. In our play testing it was interesting to see how users would try and play it like a keyboard. It was really fun adding more and more variations to the piece based on player feedback.