Dr. Ofir Lindenbaum Develops a Foundation Model for Scientific Tabular Data

Foundation models are neural networks trained on large, diverse datasets and can be used for a wide range of tasks. In recent years, such models have revolutionized the fields of text and image processing. For example, language models such as ChatGPT can perform many tasks, including answering questions, writing, translating, and summarizing, without requiring a separate model for each task. A similar revolution is now beginning in the field of tabular data, one of the most common and important forms of information in science, medicine, and industry.

Dr. Lindenbaum's research, recently awarded the ISF grant, focuses on developing a foundation model specifically for scientific tabular data, such as gene expression tables, electronic health records, and measurements from experiments and sensors. "These datasets are often complex and noisy, contain a large number of variables, and differ substantially across research domains and measurement systems," explains Dr. Lindenbaum. "The goal is to develop a model that can learn from a broad range of datasets, adapt to new scientific problems, and perform tasks such as prediction, identification of important variables, and discovery of hidden groups and structures in the data."

A central component of the research is the development of an interpretable-by-design model. "In scientific research, producing an accurate prediction is not enough; it is also important to understand why the model reached that prediction, which variables influenced the result, and whether it identified a genuine scientific relationship or relied on a coincidental pattern. In a medical setting, for example, we would want not only to predict a patient's risk of developing a disease, but also to identify the medical or biological measurements that contributed to that prediction," says Dr. Lindenbaum. "A model that is both accurate and interpretable could become a powerful tool for discovering new knowledge in fields such as biology, medicine, and physics."