Dr. Ethan Fetaya Teaches Computers How to See

“If you look at it from an algorithm perspective, an image is a collection of pixels. Deciding which set of pixels is the face, for example, is a highly complex one, and the best way to try and crack it is to learn from an example: the computer receives a collection of images with faces, others without faces, and tries to distinguish between them,” explains Dr. Fetaya, who specializes in machine learning and vision apps.

Per Fetaya, several fascinating questions arise once we approach principles of learning in computer vision. “A classic one is a generalization: let’s say that the computer has seen a thousand examples. If we’re looking for a very complex model, we might find one that works great with these examples, but not with others. On the other hand, if the model is too simple, its performance would be abysmal, both with images we’ve seen and with those that we haven’t. For instance, if I’m trying to distinguish between men and women, and only have three images of men, all of them green-eyed – the computer might conclude that a man is anyone with green eyes. The goal then is to find a balance and build models that are complex enough on the one hand but still manage to find the pattern we’re looking for. This is called generalization: does your model grasp the real pattern and not just a random pattern.”

“Neuron networks are a type of learning system – a model that roughly imitates the brain. They have phenomenal success, particularly in vision, but they are not flawless. One of the problems is that they can be easily manipulated. You can take an image of a cat, change it slightly, almost undetectably, and the model would now think that it’s a dog. This kind of disruption could have ramifications on fields such as autonomous driving, a field that is going to have an immense impact on our lives but still faces serious hurdles. The levels of precision and reliability have to be extremely high. A system that’s in charge of human lives cannot be so vulnerable and prone to error.”

Fetaya, 37, married+2, has been researching computer vision since his doctoral studies at Weizmann Institute (“I wanted to do something practical after a BSc and MSc in math”). After completing his PhD, he worked at General Motors’ autonomous driving department, then went on to pursue a post doctorate in the field at the University of Toronto in Canada. He returned to Israel at the beginning of this academic year, directly to the Faculty of Engineering. In the past semester, he taught an advanced course on deep generative models, and now he is establishing his group and is seeking “students with a strong mathematical background and programming skills, creativity and curiosity.”

In addition to computer vision, Fetaya also explores additional issues in computer learning. One of these is few-shot learning. “Humans can learn from a limited number of examples, even as children, unlike computer systems which require plenty of information,” he says. Models relating to symmetry are another issue on his agenda. “One of the qualities of a neuron network is a universal approximation – if we have enough computing power, and given enough parameters, we can approximate anything. This begs the question, does that still hold true it we limit the model? Do I weaken the model when I restrict it? This is where symmetry comes in: if we have a collection of things with no specific order, such as players on a basketball team, and my model is supposed to make a prediction that takes all of these objects into account – I want to get the same result, no matter what order their order of input was. What’s the right way to model this situation? How do we accomplish what we want without over-limiting it? If we force it to be symmetrical, can we approximate every function that upholds these symmetries? Despite significant successes in recent years,” concludes Fetaya, “the gap between human learning and machine learning is still great, and there are many interesting questions to study.”