My Computer and I Went to the Movies

Seven years of work went into Prof. Zvi Lotker’s book, Analyzing Narratives in Social Networks: Taking Turing to the Arts, published last spring by Springer, one of the world’s leading scientific literature publications. The topic seems to be taken straight out of a sci-fi novel: analyzing artwork such as stories, plays, and movies via a series of dialogues between a man and a machine. “In 2014 and 2015 I published two articles on human/machine dialogues, one exploring the dimension of time, the other – space,” shares Prof. Lotker. “They taught me that I wanted to delve deeper and construct an entire theory on such narrative analysis, between man and machine. It’s a unique research topic that hasn’t been studied by anyone else in the world. No one ever imagined that a computer could go to a movie or a play with you, then discuss them from a different perspective than the one you held.”

Because it’s something that’s pretty hard to grasp, having a conversation with a computer.

“Canonical works such as Hamlet, The Godfather, even the story of Moses, they’re different from science. We know them well and tell and retell them over and over in different versions. When we talk about Romeo and Juliet, we know that they die in the end. Even children at the Seder table come prepared – they hear the story in kindergarten, they are prepared. So the goal of a canonical story is not to surprise, that’s not its essence; the essence is our interpretation, and that can change with age, time period, and all sorts of things we go through in life. The story is not what’s written, but what isn’t, and that’s something that’s really hard to handle when you try to write algorithms or mathematics of stories. Math tends to be precise, while stories have important parts that are metaphoric, things that can be interpreted in different ways each time.”

So how do you bridge that gap?

“The way to cope with this hurdle is to generate containers. Definitions that are like cups or pots, things you can pour different liquids into at different times. You can pour coffee, tea, milk, or water – but once you’ve poured water into the cup, it’s a cup of water. In other words, the goal is to create concepts that are not fully defined, which leaves you with the freedom of interpretation. Math’s alleged tendency is to be defined all the way through but, in reality, all of the math’s great theorems can be described in a variety of ways. That’s why matrices are so great – they can be interpreted in so many ways, and they can be anything that generates links between two elements, whether it’s people or physical elements. That’s the general idea of the book: generate a dialogue between a person and a machine, where each party has something to say, and create a shared language that both sides can understand.”

Still, how do you have a dialogue with a machine? It has no consciousness.

“The question is not whether the machine has consciousness, but whether we are willing to listen to what it has to say. People who work in science like to see proof, but you can prove here: that it’s beyond math. It’s the intersection of math, philosophy, and literature, and it’s an extreme and difficult interaction because these fields are practically opposites. I enjoyed talking to my computer because it made me look at things differently. We don’t always agree, but the result is usually interesting.”

So are we headed towards a world where machines replace humans even when it comes to the most basic human skills, like a conversation?

Not at all. We can ask the computer questions and it will answer, compose linear equations, say: ‘that right there is an important point.’ But what these things mean, whether we’re looking at a critical point in the story’s rhythm, and if so, why – that’s all left for human interpretation. I can say, ‘wow, that’s an interesting point I didn’t consider,’ or I could say, ‘you’re wrong, let’s move on.’ the hard part, the artistic, thoughtful part, no one is taking that away from us. In fact, this book harshly criticizes AI. The prominent example in contemporary AI says, let’s imitate. But why? There are some things that computers do better than humans, and others that humans do better than computers, so why imitate? Why not collaborate? It’s not the right approach and in a sense, I consider it destructive. I love humans, but if we don’t leave enough room for ourselves, computers will eventually replace us.”

Data has form

Lotker, who last month won the Rector’s Prize for Scientific Innovation, presents the reader with a three-part book. The first is about the geometry of the story (Graph 1): how to tell a story mathematically, how to identify conflict and key points of interest, and how to make room for a human/computer dialogue. Parts two and three explore the two main axes of narrative as defined by Lotker: time and space, respectively.

The axis of time measures the subjective time of the characters and the story. It traces works from a chronological perspective, placing subjective clocks throughout, and demonstrates how they can be compared and measured. “Working on a time axis with the computer made me realize that we don’t always understand time correctly. We look at it objectively, as something permanent that doesn’t change, but time is actually subjective. Everyone experiences it differently. We have a lot of information about the points that matter to us, even if that information is not always accurate; the points that aren’t important are forgotten,” says Prof. Lotker. “So the key to our soul, in a way, is understanding how time works subjectively. And so I initially defined the concept of the subjective clock and sought out ways and tools for measuring it. With the help of the subjective clock, the computer was able to identify critical points in the play, moments where something significant is happening. What that significant thing is, and why it’s so critical – that’s left for human interpretation.”

The third section of the book explores space – interpretation on the axis of time and issues for which time and space conflict or collaborate. “The axis of space essentially allows us, after reading the story chronologically, to start analyzing it as a single unit,” explains Prof. Lotker. “It was Gunnar Carlsson who said: ‘data has shape and shape is matter.’ Data has shape, and shape matters. When I set out to analyze a story or a script or a play, I first generate its geometry: a graph in which the characters are the vertices, and if two share a dialogue, I connect them. That’s how I build the story’s social network. It allows me to see the story as a geometric, mathematical object, and that’s something a computer can comprehend – as opposed to human language, which is far more complex.”

Graph 1: Hamlet – Who spoke after whom

And what can form tell us that words possibly can’t?

“In Richard III, for instance (Graph 2), whose protagonist is the epitome of evil on earth, the graph is completely hollow; there’s no social network because the characters don’t speak to each other – the protagonist murdered them all. That’s the visual representation of human evil: a large circle with no triangles, where the characters don’t speak to each other, where there’s no network. It says something about how we perceive evil: the destruction of the social fabric in human society.”

Graph 2: Richard III – Who spoke after whom. A graph without triangles represents evil

To be or not to be, an algorithm

One of the most famous plays that Prof. Lotker analyzes in his book is Hamlet, the tragic Shakespearean tale of the Prince of Denmark who is forced to avenge the death of his father. “When I analyzed the play on the axis of time, the computer marked down the first murder as a critical turning point: the murder of Polonius, the king’s Chief Counselor and father of Hamlet’s beloved Ophelia. This painted this seemingly minor character in a whole new light for me, and when I sat down to analyze it, I suddenly realized that not only is he the main character – he’s the main villain,” says Prof. Lotker. “Polonius, according to the play, is murdered by accident: Hamlet stabs him through a tapestry, mistaking him for his uncle, Claudius. Seemingly, an unfortunate case of misidentification. But Polonius does several unforgivable things: After the murder of Hamlet’s father, when it appears as though Hamlet will not be the next King of Denmark, Polonius cancels his daughter Ophelia’s engagement with Hamlet and sends her off to join the nunnery. In an instant he turns her from a queen to a nun, preventing her from ever having children. I can’t think of a worse father. What’s more, Polonius was the king’s Chief Counselor; once the king is murdered, he becomes Claudius’ Chief Counselor. At the beginning of the play, the ghost of Hamlet's father informs Hamlet that he was murdered via poison to the ear, which is, metaphorically speaking, to say that he was given ill advice. And who whispers advice in the ear of the king? Why, his counselor, Polonius, of course. Which leads us to the second axis – space.”

Graph 3: Comparison between King Claudius and Lord Polonius: On the left, we can see that Polonius has more direct power. On the right, we can see that King Claudius has more indirect power.

Once the story gets a graphic representation following a timeline, I was able to start asking where lies the conflict, or what power each character possesses. I’ve got algorithms that describe direct and indirect power. Then, when I compared all the main male characters – Hamlet, his father, Claudius, and Polonius (Graph 3), I discovered that the main character controlling the play is Polonius. He has formal power by definition of his position, and he directly controls the social network, contrary to Claudius, whose control is indirect. One example of this is found in the play within a play: Hamlet, who is trying to find out who murdered his father, puts on a play about the murder of a king and watches the audience to see how they react. Not wanting to watch, Claudius gets up to leave. When Polonius catches him, he stops the show. An indirect action vs. a direct one. These links between Claudius and Polonius are something the computer opened my eyes to, and that’s where human interpretation comes into play: the fact that Claudius controls the indirect network and Polonius controls the informal one shows that there’s a distribution of labor, which can explain Hamlet’s struggle and why it takes him so long to take vengeance: he is dealing with a two-headed monster. His hesitation, the famous “to be or not to be,” reveals that this is not simple revenge; there was a cold-hearted, calculated plot involving plenty of people, and untangling such a web is far from easy.”

You could reach the same conclusion without a computer.

That’s right, but the computer can hone it and help me with countless incisions. I can ask it to split up the network, for example, into people who support Claudius vs. those who support Polonius, focus on Ophelia or Queen Gertrude, or unite women against men. If, for example, I take the relationships between Claudius and Polonius and position them against the relationships between Ophelia and Gertrude, then ask the same question about influence, I get something incredible: Men have direct control, but women have indirect influence (Graph 4).

Graph 4: Comparison between Queen Gertrude and Ophelia, and King Claudius and Polonius. On the right – direct influence. On the left – indirect influence.

One of them drank poison, the other jumped into the river. Doesn’t seem like they’re in control of anything…

“Again, it’s a matter of interpretation. You have to take into account the cultural norms of Shakespeare’s time, women had very little control over their own destiny. So you could say that Gertrude and Ophelia did in fact take control over their destiny, tragic though it may be – you could also tell the computer that it’s wrong. I make room for the algorithm to ask or get clues from humans because humans care about the story, they have an opinion and they understand the story differently from the computer.”

Love, morality, life, and death

For the past five years, Prof. Lotker has been teaching a mini-course based on his book, Advanced Topics in Social Networks. This year it became a full-fledged course for fourth-year and postgraduate students. “Students love this course, and it’s a great way to wrap up their degree because they get to use all of the tools they acquired during their engineering studies and talk about things humans truly care about: love, morality, life, and death,” Prof. Lotker smiles. The course includes highly advanced math, but we use it to analyze fascinating things they didn’t even think about, and it’s a fantastic experience for the students: what fun it is, after four years of studying, to suddenly discuss stories.”