Pure Data — recent_blog — Simon Lysander Overstall

A project I am currently working on is "I Am Afraid" — a multiplayer VR interactive artwork that involves playing with sounds and words, with others, in a virtual space. The artwork provides an opportunity for people to play, be curious, and explore as a counter to fear, either personal or general. The project is the work of Dr. Maria Lantin from the S3D Centre at Emily Carr University, Vancouver, BC.

In the VR space the participants can record any sound (usually produced with their voice) and then interact with the sound through gesture, scrubbing through the sounds or looping playback. Gestures can be recorded and set to loop, so patterns and composition can be improvised. Spoken words are recorded and converted to text using Watson so that they can be rendered in scene and interacted with. My role has been to contribute to the design of the sound interaction and implement the audio technology, in particular the live granular processing in Unity that allows for the scrubbing and other effects.

The project is built for Google's Daydream VR system, in Unity and can work with a motion capture system to allow in game mobility. It is very complex with the multiplayer (think networking), motion capture, and live audio processing. Although the components built with their service are not currently part o the project I would like to thank Enzien Audio for their tremendous support in trying to get things working on the Pixel phone. One version of the granular audio processing was built in Pure Data and converted to a Unity plugin with their Heavy Audio tool. Currently we are using Unitly's Native Audio Plugin SDK demo which we managed to build for Android.

More news when it happens.