Socially Pertinent Robots in Gerontological Healthcare - SPRING

H2020 consortium. Project duration 2020-2023. Details can be found in  https://spring-h2020.eu/

Scientific objective
Develop a novel paradigm and novel concept of socially-aware robots, and to conceive innovative methods and algorithms for computer vision, audio processing, sensor-based control, and spoken dialog systems based on modern statistical- and deep-learning to ground the required social robot skills.
Technological objective
Create and launch a brand new generation of robots that are flexible enough to adapt to the needs of the users, and not the other way around.
Experimental objective
Validate the technology based on HRI experiments in a gerontology hospital, and to assess its acceptability by patients and medical staff.

Combined Neural Interface and Deep Learning Methods for Multi-Microphone Assisted Listening and Selective Attention Devices

Ministry of Science. Project duration 2020-2022. This is a joint project with Elana Zion-Golumbic Jacob Goldberger.

Modern day environments are laden with rich stimuli all competing for our attention, a reality that poses substantial challenges for the perceptual system. Focusing attention exclusively on one important speaker and avoiding distraction is a major feat, in particular for individuals with hearing or attentional impairment. 
In this project, we propose a unique combination of methodologies from signal processing, machine learning and brain research disciplines that can jointly develop novel algorithms and evaluation procedures capable of extracting and enhancing a desired speaker in adverse acoustic scenarios. Specifically, we harness the power of deep neural networks (DNNs) for audio-processing classification tasks, to develop approaches for training a DNN with multi-microphone speech recordings, and to overcome the complex nature of dealing with natural speech data with its inherent dynamic nature. Moreover, determining which speaker should be selectively enhanced will be guided automatically by the user’s momentary internal preferences using a real-time EEG-based neural interface. The novel and unique neuro-engineering approach is critical for developing stable technological solutions meant for human use and that need to adhere to real-life behavioural and environmental constraints, and can be applied more broadly in the design of new-generation “attentive” hearing devices, “hearables” and teleconferencing systems.
Contact: sharon.gannot@biu.ac.il