ºìÐÓÊÓƵ

Topic

Multi-Channel Source Separation with Video Data

Department of Electrical and Computer Engineering

Date & location

• Monday, December 16, 2024

• 11:00 A.M.

• Virtual Defence

Reviewers

Supervisory Committee

• Dr. T. Aaron Gulliver, Department of Electrical and Computer Engineering, University of Victoria (Supervisor)

• Dr. Panajotis Agathoklis, Department of Electrical and Computer Engineering, UVic (Member) 

External Examiner

• Dr. George Tzanetakis, Department of Computer Science, University of Victoria 

Chair of Oral Examination

• Dr. Afzal Suleman, Department of Mechanical Engineering, UVic

   

Abstract

This research introduces a supervised multi-channel audio source separation system that integrates a video-based face detection system. The face detector identifies the nose position, aiding the multi-channel processing in isolating the primary speaker while suppressing environmental background noise and distracting secondary speakers. It is demonstrated that in far-field applications, multi-channel processing struggles with distracting secondary speakers when the primary speaker position is unknown. Utilizing video data provides valuable insights to identify the target speaker and assists the audio source separation system in directing its focus towards this speaker. Furthermore, it is shown that multi-channel processing benefits from speaker position information to improve noise reduction in noisy reverberant environments.