John Kustin and Vangelis Dikopoulos win Qualcomm Innovation Fellowship to support work on intelligent audio systems

Imagine driving along, listening to a podcast through your car speakers while your road trip companion simultaneously takes a nap in silence. You can blast your podcast as loud as you’d like, but the sound isn’t detectable from the passenger seat, just feet away.

This is one example of a scenario that ECE PhD students John Kustin and Vangelis Dikopoulos hope to achieve with their research. They won the 2024 Qualcomm Innovation Fellowship (North America) with their proposal, “Towards Intelligent and Adaptive Low-Power IoT Audio Systems.” The fellowship will provide the team with $100,000 of research funds and professional mentorship, as a means to “empower graduate students to take giant steps toward achieving their research goals.”

“Our proposal addresses the fundamental challenges of parametric sound in IoT devices through the design of an integrated circuit suitable for mass production,” Dikopoulos said. “Our goal is to design a high-performance system that is very low-power, versatile, and capable of operating in a variety of acoustic environments.”

This system will significantly improve the quality and responsiveness of audio interactions in various IoT applications​​.

Vangelis Dikopoulos

The audio Internet of Things (IoT) includes any device that interacts with users through voice commands or sound delivery. For example, smart speakers like Alexa and Google Home, smart earbuds, hearing aids, public announcement systems, and the Bluetooth sound systems in many cars are all part of the audio IoT.

Currently, even smart speakers widely broadcast the sound throughout a space––such as the background music at a dinner party or the surround sound in a movie theater. With today’s technology, this is easy to do. 

However, as audio IoT devices become more widespread, users may look for more private or personalized listening experiences. Kustin and Dikopoulos are designing an integrated circuit that will enable parametric, or focused, sound for individual users in shared spaces. To achieve this, they use a technique called beamforming, which enables directional signal transmission.

In their proposed system, beamforming will be performed by shifting the timing of the signals broadcasted from the speaker in different directions. In most directions, the signals will cancel out––but in one intended direction, the signal will be additive. In this way, the sound is directed to one location in space. However, the sound will not be immediately audible.

“Parametric audio systems intentionally beam out ultrasonic sound,” Kustin explained, “Humans can’t hear audio in the ultrasonic range. The audio becomes audible through demodulation in the air as the waves progress further away from the source––the audible sound is generated right next to your ears. Combining the parametric audio format with beamforming techniques allows us to steer the highly focused ultrasonic audio beam.”

The small power budget of their system will allow for long-term use in mobile IoT devices, as well as devices set up in remote or difficult-to-reach locations. They envision targeted audio delivery in public places like museums, supermarkets, and airports, such that a personalized broadcasted message can reach a specific individual, in addition to private listening in confined spaces such as cars.

“Overall,” Dikopoulos said, “this system will significantly improve the quality and responsiveness of audio interactions in various IoT applications​​.”

Kustin and Dikopoulos are both advised by Michael Flynn, Fawwaz T Ulaby Collegiate Professor of Electrical and Computer Engineering, who will be supervising this work in his research group.