APD - Week 7 - Play and Sound
THEME
Play and Sound
Please feel free to play with force yourself~ https://hiyokaygames.itch.io/help-sing (pwd: magi)
Progress video: https://youtu.be/9VGcNZXYRHM
CONTEXT
As an artist and animator, I have a pretty limited grasp on sound design. The closest thing to a musical education I get is from watching TwoSetViolin on YouTube. A while back they had a video where they tried to make a lo-fi track as classical musicians. One of the things I noticed is that Lo-Fi is usually a steady beat, supported by other relaxing sounds, such as rain, library, cafe ambiance. I wanted to make an interactive make your own Lo-Fi type game, but instead of the usual set of relaxing sounds to sleep or study to, instead, it’s random sounds that are out of place.
While I sketched out ideas of possible sounds, such as a toy duck and a cat, I wondered if my own voice could be used, perhaps recorded in different pitches and attached to different buttons. This reminded me of Vocaloid, which is a singing voice synthesizer software where users create music by putting in a backing track and giving the Vocaloid program a set of lyrics to sing. The program will change the pitch and tone of pre-recorded sounds to create these lyrics.
When Jordi Bonada was trying to make a singing-synthesizer program, the decision was made to record a set of vocal exercises with a great phonetics range, and build a model capable of singing any song, as mentioned in the article ‘The Making of Vocaloid’.
I wanted to use my own voice to make something similar to this. I’m not too sure how the pitch would work on Unity or even if I could record and replay the sounds on Unity, but I decided to try my best.
METHOD
First I made sketchy illustrations for the random noises because I am an artist and there’s nothing more demotivating to me that working on a project that doesn’t look aesthetically pleasing. The idea was to have the elements as black and white lineart when it’s not clicked and not producing sound and when it’s clicked it switches into color.
I decided to have a constant humming coming from the girl because I wanted to have her in colour the entire time.
I’ve never worked with sound in Unity alongside Fungus before which was a bit of a challenge. I got to choose between ‘Music’, ‘Ambiance Sound’, ‘Random Sound’ and ‘Sound’, that all play sound but in slightly different ways. ‘Music’ and ‘Ambiance’ could be looped but I couldn’t play multiple overlapping versions of each, so I used Music for the character’s humming and decided to save Ambiance for more background noises. And used Sounds and Random Sounds for the other elements like the cat and guitar as they allowed for overlap.
Then I went through the painful process of recording and editing myself singing each vocaloid like sound. Vocaloids are a Japanese concept so they use the Japanese gojūon, which uses phonemes instead of letters. So instead of M and A, E, I O, U there’s MA, MI, Mu, Me, Mo. Using this system is easier to make words so I recorded myself singing these sounds in short bursts. I tried to make it as similar to vocaloid sounds as possible, but it was difficult to keep everything at a consistent length and similar sounding.
I then posted the link to play it on the MAGI unity club. I got feedback from Margaret on how it would be better to have a Do Re Mi type vocal progression so that the player can make a melody instead of lyrics. I was inspired by vocaloid software where users get to make their own lyrics is why I used the Japanese gojūon , but sadly there are a lot more fine details that go into making something like that. I also got more feedback from Sahaj about mapping the buttons to the keyboard and the option to disable the background music.
So I made a new version that incorporated all the feedback and more with bgm toggle, keyboard toggle and even pitch variation.
RESPONSE / REFLECTION
I was quite happy with how this project turned out, even though it changed its form from a lo-fi maker to more of a vocal keyboard. I wish I had controlled my pitch better and done some more takes so that the final sounds would have been more polished. The fact that the things I could do depended on my level of unity knowledge was a bit disappointing too. For example; Fungus would only allow the pitch to be changed lower but not higher.
As Bronson Roseboro puts in, in their research thesis ‘The Vocaloid Phenomenon’, making songs takes “both the composition of lyrics and of music, each being a deeply rich discipline all its own.” I attempted to do this through trying to incorporate lyrics and tone together, but ended up with a song creator that had both lyrics and music but was a bit of a mess. By scaling down and focusing on just the notes and limited sounds, I was able to create a much more playable song maker.
Getting the feedback from my friends and Unity club members and my decision to actively take the feedback on board was honestly what set this week apart from the rest of the APD weeks.
REFERENCES
Roseboro, B., 2019. The Vocaloid Phenomenon: A Glimpse Into The Future Of Songwriting, Community-Created Content, Art, And Humanity. DePauw University.
https://scholarship.depauw.edu/cgi/viewcontent.cgi?article=1125&context=studentresearch
St. Michel, P., 2014. The Making Of Vocaloid. [online] Daily.redbullmusicacademy.com. Available at: <https://daily.redbullmusicacademy.com/2014/11/vocaloid-feature>
#kaykay #kkapdwk7
