This new device uses computer vision to teach the piano

Four out of five parents reportedly want their children to learn a musical instrument. A much smaller number of kids actually do, and even fever stick with it.

“It’s intimidating, it’s expensive, it’s hard,” acknowledges Roland Lamb, whose company ROLI has been trying to reinvent music education, and instruments themselves, for 15 years. Now, Lamb believes that he has found a way to make piano lessons a lot more accessible: ROLI’s new Airwave product uses special infrared cameras for hand tracking, which makes it possible to take lessons on an iPad that monitor a player’s posture—just like a teacher would, but for a fraction of the price.

ROLI’s $299 Airwave, which looks a bit like a designer reading lamp or a fancy iPad stand, is meant to be combined with one of the company’s keyboards, as well as a tablet running the company’s Learn app that offers access to a number of different learning modes via a $14.99 per month subscription service.

At the most basic level, ROLI’s app offers aspiring learners the ability to pick from a wide range of compositions, have a song displayed as sheet music, and then play it note by note. Corresponding keys light on the ROLI keyboard when it is time to play them, and the computer vision camera keeps track of the student’s hands. “If I play with the correct finger, it will progress,” Lamb explains. “And if I play with the wrong finger, it won’t progress.”

Afterwards, the app gives feedback on positional and postural accuracy, meaning the way a student uses their fine motor skills to press individual keys, and the way they hold their arms and wrists while doing so–kind of like a piano teacher would do.

However, Lamb insists that ROLI doesn’t want to make traditional piano lessons obsolete. “If you can afford a human teacher, you should 100% study with a human teacher,” he says. Students who struggle to keep up with practice between their once-weekly lessons can get some reinforcement from the device, Lamb argues. “And if you can’t afford a human teacher, this can also help you.”

But Airwave isn’t just for beginners. ROLI is also pitching its computer vision system to musicians and producers, thanks to its ability to track hand movements in 3D, even when those hands don’t touch the keyboard at all. Essentially, musicians can turn Airwave into a kind of modern-day Theremin to orchestrate and manipulate complex soundscapes simply by moving their fingers through the air—and just as seamlessly switch back to hitting a few keys.

“You can have the ethereal beautiful quality of a Theremin, but you can also play it like a keyboard,” Lamb says.

ROLI isn’t the first company to use cutting-edge technology to create a new kind of instrument, or rethink music education, for that matter. Creative developers have built VR headset apps to teach the piano, and turned smart watch-like wristbands and Bluetooth-connected bouncy balls into music-making machines.  “Almost all of those types of inventions are doomed to fail because the cultural shift is too great for people to make,” says Lamb, who wrote his PhD thesis on the history of musical instruments.

That’s why ROLI is launching Airwave with a keyboard at its core, and focuses on teaching an instrument as well-established as the piano first. Over time, Lamb wants to also bring the technology to other instruments, and perhaps entirely new ways to learn and make music, by opening it up to third-party developers. “We believe that this is a platform,” he says. “Many different apps will be able to utilize Airwave to create different kinds of interactions.”

“This is almost like a new category of spatial music,” Lamb says.

https://www.fastcompany.com/91204742/this-new-device-uses-computer-vision-to-teach-the-piano?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss