Master Of Animation, Games & Interactivity
Master Of Animation, Games & Interactivity

Context:

I want to express the purpose of visualizing sound in this prototype. This involves synesthesia, which is a state of perception with a neural basis. One sensory stimulus will cause people to involuntarily induce another perception. This is unconscious. There are various forms of synesthesia, and I want to design a prototype that can reflect synesthesia. This should be a combined tambourine. When the drum head is struck, there will be a short performance reminiscent of the picture.  Richard Cytowic believes that the color and other visual synesthesia caused by specific sounds are like fireworks, sounds, music and environmental sounds induce colors and shapes of fireworks to rise everywhere.  Sound can change the human network's perception of tonal brightness flickering and directional movement. When you are listening to a symphony, it may be like being caught in a vortex of colors. Everyone has different color synesthesia for sound, just like I would think that B-flat major is blue. With color, I also want to integrate sound with more elements, such as the phenomenon of different sounds in nature or the atmosphere.

Method:

Cooperating with the characteristics of the sound, visitors will have a sense of substitution and association. For example, in the flute performance I added in the video, the tune is tactful, and when I first heard it, it was a wave shape. Then start to imagine the environment that conforms to the shape of the wave, as well as the weather and atmosphere. Finally confirmed the first scene.  At the same time, in the performance of the second section, the tune was more twists and turns, and the frequency was tighter and faster than the first section, so I chose a denser composition and weather—rainy. It was as rapid as raindrops falling on the ground, and the waves undulated as the wind blew over a reed.  At the same time, I also added a ripple effect to the text to make it closer to my synesthesia.

Reflection:

Synesthesia is a very personal thing, and everyone's synesthesia is different. I just choose examples to express my feelings. Different people have different imaginations when they hear the same piece of music. If I want to continue to develop this theme, I tend to focus on using cross-border means to embody synesthesia. For example, let the model of the tambourine be completed and then hit the tambourine to add objects of different colors to the scene on the screen. The density arrangement is determined according to the combination of frequencies, so that players can paint by themselves. The tambourine in the middle can be used to switch the type of sound, from drums to other different instruments. Greatly improve the playability.

About This Work

By Raivis Su
Email Raivis Su
Published On: 23/09/2020