APD Week 3- Play and Tinkering: Floral Beats [][]

By Carlo Tolentino

Theme: Play and Tinkering

Context:

During this week’s lecture, we were shown various ways of tinkering with input controller based projects. In the early phases of my studio 2 project, I ideated on combinations for projection mapping and music inputs such as a piano, a guitar or unconventional percussion-like instruments. I wanted to tinker with this idea, as well as give me the opportunity to tinker with music and controller inputs.

I wanted to experiment with an old RockBand Drums kit that was lying around the house and see if I can get it to work properly on my pc as a form of controller input that one hits with a hand or a stick.

Dark Souls played with a Rock Band Drums kit: https://www.youtube.com/watch?v=02my_zhX4Bs
it was motivating to find other people using the Rock Band Drums kit in ways it wasn’t initially intended for, such as playing an entirely different genre of game.

The eventual idea of making flowers and plants move and react to inputs and music is from a research suggesting that sound, music, and its vibrations stimulate plant growth: https://sciencing.com/does-music-affect-plant-growth-4596442.html I wanted to play with the information on these findings, and see how I can translate it through my process of rhythmic music created by a person in real-time.

In the realm of aesthetics, my response was inspired by a beautiful illustrated and animated music video that had meditative floral elements: https://www.reddit.com/r/AfterEffects/comments/cm6vlc/any_thoughtscritique_on_this_music_video_i_created/ This also gave me the opportunity to experiment with an art style that I haven’t tried but keen on developing, hand drawn floral designs.

Method: Without a fixed objective, tinker with a Rock Band Drums kit to see how I can use it as both a musical input, as well as a controller input for an interactive experience involving a projection-mapping software.

Response: Flower Beats is an audio-visual interactive animated flowers and plants growing prototype using a Rock Band Drum Kit (input controller), Resolume Arena (animation, interactions), rb2midi (Rock Band Drums kit bridge to pc), loopbe1(music output as midi inputs).  

Reflection:

I initially wanted to tinker with an Xbox Kinect for this week’s theme, but finding the proper adapters and their unavailability quickly halted that intention. I’ll definitely come back to tinkering with it in a future theme or process.

The Rock Band Drums kit, was fortunately just lying around the house gathering dust. I find it exciting to repurpose these old controllers, that are often used only in a specific genre of games that they come with. The idea of using various traditional or music game controllers as interactive projection-based instruments also appeals to me and might explore these implementations in the future. I also think that developing this in Unity would provide better results, but at the cost of more development time. Resolume provided the bare minimum functionality I needed to create the prototype/s I had in mind.    

During the process of tinkering and getting the Rock Band Drum Kit to work as both a customized musical instrument and a midi-device, I thought of ideas that I could develop with the physical hitting of the drums. A few of these were: a first-person game where you had to do a quick drumroll to attack, a platformer where hitting on the drums would raise level design platforms depending on which note you play, and lastly, a musical floral arrangement or plant. I chose the last one as it is the one that I thought I could plan out to be within feasibility considering a quick iteration prototype.

Flower Beats is intentionally not projection-mapped, as my focus was to figure out a process and to demonstrate the combination of the Rock Band Drums kit, its communication with various software, syphoning everything to Resolume, then the human interaction. Additionally, I combined various drum-hit sounds, chimes, and cymbals to the inputs slightly deviate from what one would expect from hitting on a drum kit. Despite being familiar with Resolume as a VJ software, I still encountered technical challenges in making it communicate with the drum kit, its midi inputs, as well as the parameters of each input to be translated into basic transform animations. I could have also made all the animations simply react to sound in Resolume, but then it would be a chaotic cascade of simultaneous animations instead of each floral element having a corresponding drum hit input. I’m very pleased with my end-result, and I can pinpoint areas in my process in which I could develop further, as well as translate the concepts that emerged into a more resolved project. In the end, I felt that I tinkered and grappled with a combination of various software, to produce an output that allowed for further tinkering or even more so, the act of play. Everything in the video is when I let my wife, nephew, and niece tinker and play with Floral Beats. Admittedly, I had to cut parts of my nephew and niece playing with it too, as it didn't take long for them to quarrel taking turns on it. I suppose they surprisingly had more fun than I thought they would.

Royalty-free BGM 80's Pop Synthwave from: https://elements.envato.com/