APD-Week 5

Video link: https://youtu.be/dJxSe7ajDro

Theme: Play and Force

Context:

When Matt presented this topic in class, my first thought was to show how objects are affected by gravity. However, after reflection, I feel that using gravity to reflect this topic is too superficial. It cannot express the rich content and deep meaning of the theme. Then I was inspired by the book Generative Art Guide: Examples, Software and Tools to Make Algorithm Art:

“Generative artists leverage modern processing power to invent new aesthetics – instructing programs to run within a set of artistic constraints, and guiding the process to a desired result.”

I also think of Newton's third law:

“If an object A exerts a force on object B, then object B must exert a force of equal magnitude and opposite direction back on object A. This law represents a certain symmetry in nature: forces always occur in pairs, and one body cannot exert a force on another without experiencing a force itself.”

When slapping the table, it can be seen as a force by the hand to it, and the table will deform and have a reaction force. And this force showed in many ways, such as people will feel pain, sounds will be made, and other objects on table will also be affected. In this process, input is hand slaps table, and the output is felling, sound and other objects. I want to take this output sound as a new input and visualize it with modern processing power.

The key rationale of my project is that the harder the table is hit, the louder the sound will be. There will be a glass in the scene, when the player hit the table, the glass will be added the same force to jump. And computer will capture the volume of sound to calculate the strength of force.

Using sound as input of an interactive project is not new. Holland Kerr’s project is to use sound as input to blow the virtual windmills on screen. However, in this work, the input is not just sound, or the sound is the restricted medium, to guide the process to a desired result. To be more specific, project includes 2 input-output behaviors:

Hitting table(input-1) → making sounds(output-1/input-2) →computer capture and analysis→ visuals(output-2).

Also these two behaviors affect each other, depend on each other, and output 2 is the result of both input 1and 2.

Method:

Scripting in Unity to capture sound data and slice it, and let the computer analyze the max volume of it. Then, the volume of the sound, also, the strength of force, is expressed by the height of the cup.

Response:

It was fun to test this project because I couldn't predict what the output would be, each input leads to a unique output. And this result is both predictable and unpredictable.

Predicted: the cup will bounce according to the volume of sound. The louder, the higher.

Unpredicted: The sound is too low (between 1 and 1.5), making it difficult to visually distinguish the difference each time.

For the better experience, I multiply the sound data by a fixed variable. In this way, the gap between the volumes is magnifies to make is clear for player to tell the difference.

Reflection:

By exploring and setting limits, I tested more interactive possibilities and managed to make a simple interactive work. I have always avoided making an interactive project in the past because I am unfamiliar with this field and I cannot find a way to step into. Fortunately, by reflecting on play&force, I boarded my horizons and successfully took the first step into the field of human-computer interaction, and I am no longer limited to the traditional way of playing. As mentioned in About alternative controllers

“All these games are designed to make us play differently, not only with our fingers but with our whole body, to make us experience new feelings regular controllers can’t provide.”

Through reflection, I realized that some parts of this project can be further developed:

-More interactive objects: There is only one cup in the scene. Different kinds of objects can be set to show the different reactions under the same force.

-Showing input values in real time: Due to my poor coding skill, the real-time data can only be viewed in background. And I will continue to improve my coding skill to achieve it.

Reference:

Marnie Benney and Pete Kistler. 2021. “Generative Art Guide: Examples, Software and Tools to Make Algorithm Art.”

Shake That Button. 2022. “About Alternative Controllers.”

Games and Rules : Game Mechanics for the »Magic Circle. transcript Verlag, 2019. Print.