Keyboard

October 2018 | By Use All Five & Google Creative Lab

A simple musical keyboard you can play with your face, body, mouse, or keys.

Collection:

Black and white web camera feed of woman with a cursor mapped to her nose. Colorful musical keys with their notes displayed is overlaid

Animated Gif of Black and white web camera feed of woman with a blue cursor mapped to her nose. Colorful musical keys with their notes displayed is overlaid. As she rotates her head, the cursor moves over notes that animate and become brighter.

Screen capture of Keyboard interface. The lefthand menu displays a range of presets and controls for the experiment. In the main play area are colorful rectangular keys with their musical notes written on them

This experiment lets you play a musical keyboard in different ways. You can use your mouse and keyboard. Or, turn on your webcam and track a point on your body, like your nose. Note that your images are not being stored or sent to any Google servers while interacting with the experiment.

You can customize the scale and number of notes on your screen. And you can control other instruments using MIDI. Just connect a MIDI device and select it in the MIDI Out dropdown.

This experiment was made by Use All Five, Kyle Phillips, Claire Kearney-Volpe, Luisa Pereira, Yotam Mann, and Google Creative Lab. Special thanks to friends at Henry Viscardi School at The Viscardi Center, Tech Kids Unlimited, and ADAPT Community Network.

Built with:

Tensorflow.js, Tone.js, WebRTC