Efficient and Robust Deep Learning architectures for Real-World problems

Next BIU Engineering Colloquium,
Dr. Chaim Baskin, Thursday, 28.12.23 @16:00 
BIU Engineering Building 1103, Room 329 
Also, via Zoom https://biu-ac-il.zoom.us/j/5498468036

====================================

We are delighted to host

Dr. Chaim Baskin

Dr. Baskin will give a talk on the subject:

Efficient and Robust Deep Learning architectures for Real-World problems

The advancements in deep learning models and their ability to excel in various fields are awe-inspiring, but practical applications still face several challenges. From a data-centric perspective, Deep Neural Networks (DNNs) require a vast amount of precisely labeled data. From a model-centric perspective, DNNs tend to be amenable to malicious perturbations, have limited throughput, and struggle to process irregular data. Unfortunately, these limitations restrict the ability of deep learning to solve a wide range of real-world problems in domains such as Biology, Chemistry, Physics, 3D geometry, social networks, and recommendation systems.

In my talk, I will discuss four critical challenges in real-life deep learning.

Firstly, I will discuss a method for reducing the bandwidth of read/write memory interactions during model deployment while taking into account communication complexity constraints. Prominent applications include large-language models, transformer-based foundation models, and large-graph architectures.  

Secondly,  I will introduce innovative approaches that enable learning with noisy and limited annotations. The first approach facilitates self-supervised pre-training to detect noisy samples better. The second approach takes advantage of a small calibration set to train a teacher model in a bi-level optimization framework implicitly. In addition, I will describe how to use a small number of annotated labels while efficiently merging between modalities to handle deep learning's necessity for clean and large amounts of annotated data.

Thirdly,  I will describe adversarial attacks that can efficiently mislead any navigation algorithm. These attacks are a significant safety concern that disables deep learning models from being deployed in real-world platforms, such as autonomous vehicles.

Lastly, I will introduce the geometric deep learning paradigm and focus on learning graph data in the context of various real-world problems. I will delve into the importance of the adversarial robustness of these models and relate to their expressivity.

I will also discuss future directions on combining the presented approaches to design novel deep learning models that will efficiently merge between different modalities under relaxed assumptions on the quality and amount of annotated data, safe for use in real-world platforms, and meet the specifications of modern AI accelerators.

Short-bio 

Chaim Baskin is a Senior Research Associate at the VISTA laboratory within the Center for Intelligent Systems in the Computer Science Department and a Visiting Assistant Professor in the Faculty of Data and Decision Science at Technion. In addition, Chaim holds a Visiting Scholar position at Czech Technical University in Prague. Chaim's research focuses on representation learning, geometric deep learning, and optimization of neural networks for efficiency. He obtained his Ph.D. from the Computer Science Department at Technion in 2021. Chaim held a post- doctoral position at the same department from 2021 to 2022. Moreover, he is a member of Technion's TechAI and TASP research hubs.