Noise-aware Speech Separation with Contrastive Learning

הרקע לפרויקט:

Recently, the speech separation (SS) task has made significant advancements due to deep learning techniques. However, separating target signals from noisy mixtures remains challenging as neural models can mistakenly assign background noise to each speaker. This project propose a noise-aware SS method called NASS, which aims to enhance the speech quality of separated signals in noisy conditions.

Specifically, NASS treats background noise as an independent speaker and predicts it alongside other speakers using a mask-based approach. Patch-wise contrastive learning is employed at the feature level to minimize the mutual information between the predicted noise-speaker and other speakers. This allows for the suppression of noise information in the separated signals.

מטרת הפרויקט:

The purpose of this project is to enhance the speech quality of separated signals in noisy conditions.

תכולת הפרויקט:

1. Watch the lectures in youtube - Stanford University CS231n, Spring 2017
2. Read the paper
3. Download the dataset
4. Build the model
5. Train the model
6. Expect to satisfactory results :))
The project will be implemented in Pytorch

קורסי קדם:

Deep Learing, Python and Pytorch

דרישות נוספות:

Watching related videos on YouTube

מקורות:

We will implement the following paper:
Noise-aware Speech Separation with Contrastive Learning