Sound Canvas

October 2018 | By Kearney-Volpe / Miele / Phillips / Pereira

A simple drawing tool that works through both sight and sound.

Collections:

A black line drawing of a smiley face and the word "Hi". A blue vertical line indicates where the user is exploring the horizontal pixels of the drawing

Animated gif of blue cursor tracking a woman's nose. As she moves, she draws lines with the cursor demonstrating drawing with camera tracking on the nose.

Screen capture of the Sound Canvas Experiment. On the lefthand the menu displays presets and controls. In the main play area, a black squiggly, horizontal line is drawn from across the canvas. There is a blue vertical line in the center of the canvas indicating where the user is exploring the drawing with sound

This experiment is a simple drawing tool that works through both sight and sound. You can draw with your mouse or keyboard. Or, turn on your webcam and track a point on your body, like your nose. Note that your images are not being stored or sent to any Google servers while interacting with the experiment.

The whole canvas space will be turned into sound. For example, a rising line makes a rising sound. A line drawn from left to right will be heard moving from your left to right ear. To explore the drawing you’ve made through sound, select Keyboard input mode and use your arrow keys. Read this blog post to learn more.

This experiment was made by Claire Kearney-Volpe, Josh Miele, Kyle Phillips, Luisa Pereira, Yotam, Mann, and Google Creative Lab. Special thanks to Chancey Fleet, James Maxson, and friends at Henry Viscardi School at The Viscardi Center, Tech Kids Unlimited, and ADAPT Community Network.

Built with:

Tensorflow.js, Tone.js, WebRTC