Keynotes

Title: Personalized Hearing Loss Compensation for the Next-generation Hearables and Hearing Aids

Abstract: With recent advances in precision hearing diagnostics and machine-learning methods for audio signal processing, fully personalized audio processing for early-onset and standard hearing impairments become within reach. However, this innovation comes with several challenges: embedding biophysical (hearing-impaired) models within closed-loop systems for algorithm design remains computationally challenging, and neural-network-based audio processing comes with entirely different sound-quality challenges than encountered in standard hearing aids. In this keynote, I will introduce how novel diagnostic methods can be used to individualize the parameters of biophysical and NN-based models of human auditory signal processing, and how these models (CoNNear) can be embedded within differentiable closed-loop systems to develop end-to-end signal processing for hearing aids and hearables. I will show examples of the restoration quality of the resulting systems and how they can be further optimized for real-time processing and excellent sound quality. These translational steps will facilitate their integration with the next-generation embedded systems for hearables and hearing-aids.

---

Title: Next-Generation Speech Enhancement: Generative Diffusion Models and End-to-End Multichannel Filtering**

Abstract: In today’s digital age, devices such as telephones, video conferencing systems, and assistive listening devices are integral to daily communication, necessitating advanced speech enhancement and restoration algorithms to counter acoustic and transmission artifacts. Recent advances in machine learning have led to significant enhancements in both single-channel and multichannel speech enhancement techniques.  In this presentation, we share our latest research on generative diffusion models for single-channel speech enhancement that enhance the naturalness of sound as perceived by listeners and provide superior generalization capabilities compared to traditional predictive methods. These models are effectively utilized across various restoration tasks including noise reduction, dereverberation, and bandwidth extension. We will also address ongoing challenges such as hallucinations at negative SNRs and the high computational demands of these models. Furthermore, we will present our investigation into how relatively small deep neural networks can substantially enhance speech quality when multiple microphones are available. Moving beyond conventional methods of separate spatial and spectral filtering, we have developed a joint nonlinear spatial-spectral filter trained in an end-to-end manner. Our results demonstrate that employing multiple microphones along with compact neural networks for this joint spatial-spectral filtering results in exceptional quality. This approach particularly excels in complex scenarios where the number of sound sources is larger than the number of microphones, considerably outperforming conventional methods. At the same time, the joint filter consists of a small neural architecture that can be implemented causally and is thus readily capable of real-time operation.

---

Title: New tools for Spatial Acoustic Signal Processing for Applications with Low SNR

Abstract: In this talk, I will discuss two recently introduced signal processing concepts and their example applications in problems involving propagation of signals over space with low SNR conditions. The first concept is the Relative Transfer Matrix (ReTM) which is a generalization of the Relative Transfer Function (ReTF) for multiple simultaneously active sound sources. We allocate receivers into two multichannel groups and formulate the Relative Transfer Matrix to describe the spatial acoustic channel between them.  The second concept is called the Point Neuron Learning, a new physics-informed neural network (PINN) architecture that embeds the fundamental solution of the wave equation into the network architecture to strictly satisfy the wave equation.  The point neuron learning method can model an arbitrary sound field based on microphone observations without any dataset, directly processes complex numbers and offers better interpretability and generalizability. I will illustrate the applications of ReTM and Point Neuron Learning in drone audition and speech enhancement in noisy reverberant rooms.