Pod Insights The Brand Called You Revolutionizing AI Against All Odds | Geoffrey E Hinton, The Nobel Prize in Physics 2024; The 'Godfather of AI and Deep Learning' About this episode Geoffrey Hinton, a visionary in AI, defied skepticism and transformed machine learning. From early inspiration to winning the Nobel Prize in Physics, his journey is a testament to persistence, brilliance, and thinking outside the box.00:20- About Geoffrey E HintonGeoffrey Everest Hinton is a British-Canadian computer scientist, cognitive scientist, cand ognitive psychologist. He was a Nobel prize winner in Physics, known for his work on artificial neural networks which earned him the title of the "Godfather of AI".
Transcript 0:08 Speaker 00 Hey Jeff, thank you for coming in. Just a really remarkable, long history of contribution and impact for decades. And again, congratulations for achieving the Nobel Prize in Physics for your work in AI with Hopfield. But also prior to that, you did the ACM Turing Award, which is considered the Nobel Prize of Computing from ACM alongside your colleagues Ben Gio and Alan Lacun. So again, and then of course, you're Canadian, or you're at the University of Toronto, and you've been there. And Toronto is always a top ranking university, even globally in terms of its work, and you'd be a big part of that. So thank you for coming in and sharing your insights
0:53 Speaker 00 with my audience.
0:55 Speaker 01 Thank you for inviting me.
0:57 Speaker 00 So Jeff, you just a simply remarkable career spanning decades and pioneering in so many different areas. You know, what were maybe two or three inflection points in your life that made this wonderful career? And it could have been something when you were like three or five years old, it could have been something inside, outside the family, could be mentors, maybe early career moves and so on. So you know, what really stands out to you, you know, two or three inflection points?
1:27 Speaker 01 Okay, I think it's basically three mentors. So when I was still at high school, I had a friend called Inman Harvey, who was always much smarter than me. And he would do things like, he came into school one day and said, Jeff, do you know, there's no language in the world where you can make a question just by reversing the order of the first two words. That was a very odd thing to say. That's not what school kids normally talk about. But it turned out it was 1965, and he'd just read Aspects of the Theory of Syntax by Chomsky, the year he was published. He was that kind of a guy. And later on, he got the idea that maybe memories in the brain are like holograms,
2:08 Speaker 01 because holograms had just come out. And he talked to me about that. And that got me very interested in the idea that memories might be distributed across large parts of the brain. And from that, I got interested in how does the brain learn stuff. So that was definitely a very, an experience when I was around seventeen or eighteen that was very important. Then, when I was, when I finished my PhD, I went to the University of California, San Diego, and met David Rommelhart, who was a very interesting character. He was very thoughtful, had lots of good ideas. I remember one thing about him, we once went
2:50 Speaker 01 to a seminar that was very boring. And as we came out of it, I said, don't you resent wasting an hour of your time in a boring seminar? And he said, no, not at all. I just sit and think about something else. I have a whole hour to think about something. So he reinvented backpropagation. He didn't realise it had been invented before, but he reinvented it. And I helped him a bit with that. I didn't realise it had been invented before either. And we used it to learn distributed representations of words. And I think that was probably the first time, or one of the first times people had shown that backpropagation could learn very interesting representations. That was sort of transformative to me. And about
3:33 Speaker 01 the same time, I got to know someone called Terry Sonoski, who had been a student of Wheeler's in physics, and then converted to neural nets, got interested in the theory of neural nets, and did his PhD with Hotfield. I think he was probably one of the people who got Hotfield interested in neural nets. And I organised a seminar in, a workshop in San Diego, funded by the Sloan Foundation. And most of the workshops they had were just the normal senior people coming and giving their normal senior talk, which was quite
4:14 Speaker 01 interesting. But I decided I wanted, we'd have people we'd never heard of. And one of the people who applied to come was Terry Sonoski. I couldn't decide from his letter whether he was a genius or whether he was crazy.
4:30 Speaker 01 And I went to a conference in Texas and met David Marr, who was an important figure in computer vision then. And said, you know, he was from Boston, like Terry, did he know Terry? And he said, oh, yes, he knew Terry. And I said, well, I can't decide whether he's a genius or crazy. And David Marr said, no, could he? So I invited him.
4:53 Speaker 01 And we got along very well. And we worked on a different learning algorithm called Boltzmann machines.
5:00 Speaker 01 The, in the end, didn't turn out to be the right learning algorithm for technology. Backpropagation was the right learning algorithm. But Boltzmann machines was the most fun I've had in my life, working with Terry. And in the Nobel Prize, they indicated that Boltzmann machine was one of the reasons they gave the Nobel Prize.
5:17 Speaker 01 So those were the three people who affected me most, I think.
5:20 Speaker 00 Yeah, but it's really interesting that, you know, the people who really impacted your life were non-traditional thinkers. I would say.
5:29 Speaker 01 Yes.
5:29 Speaker 00 Kind of matrix thinkers, right? In many respects.
5:33 Speaker 01 People who like to think outside the box.
5:36 Speaker 00 Yeah, and that really is a homework of your career, right? Thinking outside the box, not staying in a particular lane and and just sort of looking at all the possibilities. Would you say that that came from your family early on or things like that? Or, or just these mentors that you've had or people, colleagues?
5:57 Speaker 01 So, so actually, I didn't change what I worked on a lot. I was just working on something very unpopular, which is, you know, how can a neural network with random initial connections learn to do complicated things?
6:10 Speaker 01 It seemed to me there must be a way because the brain does it. It was very unpopular in computer science. People said, this is crazy. You'll never get random neural networks to learn anything interesting. You have to have innate knowledge. Cognitive scientists said that. And the computer scientist said, why would you do it that way? Why not just use computer programs?
6:27 Speaker 01 And don't you understand the difference between software and hardware?
6:32 Speaker 01 It turns out, in the brain, there's not a big difference between software and hardware. So.
6:40 Speaker 01 The question is, why did I stick at the same thing for sixty years? So I sort of started thinking about this when I was like seventeen, and I'm now seventy seven. Um, and I've been thinking about how the brain learns for sixty years. And why would I stick at that, particularly through all those long periods when people said this is a crazy approach? And I think part of the answer to that is my parents are atheists and sent me to a Christian school. And having been raised an atheist, and I was quite thoughtful, I guess. Um, it was just obvious this stuff was nonsense. Um, they taught me religion at school and it was just rubbish. Um, and I felt that very strongly.
7:22 Speaker 01 And as time went by, more and more of the other kids started to agree with me. Um, so I think that was an important experience for me of being at a young age. Um, thinking that what everybody else at school believed was just nonsense. Um, that's good for preparing you for a career. And when you work on something, everybody thinks nonsense. Of course, the one extra feature you need is that you have to be right, that it isn't nonsense.
7:52 Speaker 00 Yeah, I mean, um, I would say early in your career, though, um, you know, where you would have difficulty getting published, right? Because it, it, you know.
8:02 Speaker 01 Oh, yes, yes. So the journal Neural Computation was created so that we could publish things about neural nets. And even quite late, even when neural nets were beginning to be very successful in like two thousand and nine, two thousand and ten, around that time. It was quite hard to get things published. So Yann Le Cun sent a paper to a conference, I think it was about pedestrian detection. I'm not certain about that, it might have been about semantic.
8:29 Speaker 01 Yeah. Technique. And one of the referees said, um, this shouldn't accept it because it taught us nothing about vision because everything was learned. It didn't occur to the referee that what it taught us about vision was that everything should be learned.
8:45 Speaker 01 So, and Joshua Benjou sent a paper to International Conference of Machine Learning, ICML, um, about neural nets. And one of the referees said, this shouldn't be accepted by ICML because it's about neural nets and they have no place at ICML.
9:02 Speaker 01 I had a paper rejected from actually a cognitive science society conference. Both of the referees said it should be accepted. The program chair should be said it should be rejected because it was about the interaction between evolution and learning. And the program chair said evolution had no place in a cognitive science society conference. Turned out he was a Christian fundamentalist. So all my prejudices were confirmed.
9:29 Speaker 00 And you took a proposition at the University of Toronto because they were more open to this kind of thinking, perhaps.
9:38 Speaker 01 No, not really. I moved to Toronto because I didn't like getting funding from the American military. I didn't like what Reagan was doing in Nicaragua. And I felt totally out of place at Carnegie Mellon. So at Carnegie Mellon, there were maybe 400 people doing computer science because it was the School of Robotics and a big computer science department. And we founded a branch of computer scientists for social responsibility. And we got three members. They were, they basically were from a different culture than me.
10:21 Speaker 00 So, so because of this sort of tie in from the military side, you take this position of U T.
10:29 Speaker 01 Yeah, I was part of it.
10:34 Speaker 00 The experience of U T, then it took a while to even convince them, perhaps, right? But, you know, the C F A funded you though, right?
10:44 Speaker 01 Yes, I, one of the major reasons for moving to Canada was there would have been a lot of undergraduate teaching, but C F A brought me off that. So I could move to Canada and not have any more teaching than I'd have had in the States. That was nice and not have to take military money. There was much less money around, but there was support for basic research from both C F A and from N S O C. And there was enough money so that I could have graduate students to get on with my basic research.
11:15 Speaker 00 Yeah, and then during this period of, you know, some of the AI winter, this ability of getting CIFAR money and INCIRP money provided a way to continue the work. And I'm told that's sort of very famous event with your two students, Alex and Ilya, right?
11:34 Speaker 01 Right.
11:34 Speaker 00 And can you, can you, can you describe what that was like, that journey?
11:42 Speaker 00 When it occurred to you to work with, work with these students and, or bring them on and supervise them in some fashion, while they work on this work, which then became Alex now. And can you describe that whole sort of generation or how that occurred?
12:00 Speaker 01 Well, I was quite lucky, I think, that there were very few professors with research groups that worked on neural nets and how neural nets could learn. There was me, there was Joshua Benjou, Jan later came back from industry and became a professor. There were a few others, there was Jürgen Schmitt who played in Switzerland.
12:20 Speaker 01 And what that meant was, but there was nobody at MIT, nobody at Berkeley, nobody at Stanford.
12:28 Speaker 01 So what that meant was, we got the best graduate students. Because graduate students, the people who can see the future, they determine the future. Where they go is the future. And some of the smart ones realized that learning in neural nets was really interesting. And so we got a lot of really smart graduate students. And that was a big piece of luck.
12:55 Speaker 00 And then you decided to attack this challenge that Sifilei had created, right? And so what interested you in that challenge in your students? And what was that journey like in progressing to 2012 with this, you know, kind of amazing thing?
13:17 Speaker 01 Yes. So people in computer vision, around, say, 2010, when they were rejecting papers by Jan, even though they were better than what they did.
13:27 Speaker 01 They thought that vision consists of, basically the David Marr paradigm. You have a theory of how it should work, and then you figure out how to implement that theory in the brain, or in a computer.
13:43 Speaker 01 But basically, you have to understand how it all works. You're telling it how to do it. You have a mathematical theory of how to do it. And they didn't like the idea you just learned to do it, and you might even not understand how the system did it in the end.
14:00 Speaker 01 It was exciting in 2012, because finally we had a big enough dataset, so we could show that neural networks really could learn to recognize things in real images. Before that, they'd recognized things like postcodes, but they're sort of simple two-dimensional images, without many of the properties of real 3D vision. And the computer vision community was fairly adamant that, well, you know, that's a toy problem. Okay, so maybe you can do it better than us, but it's not an important problem. It's recognizing objects in real 3D images is what's important. And you're never going to do that. And they were fairly convinced we were never going to do it.
14:39 Speaker 01 We actually knew, well before the results of the competition, that we were going to win, because the net that Alex had developed, we were testing it on previous competitions, and it was doing much better than the other methods done on those previous competitions. I actually contributed a few of the ideas to regularize the net, ideas like dropout, and ideas about what kind of neurons we should be using, rectified linear neurons, not sigmoid neurons. But the main work was done by Alex and Ilya. In particular, Alex was an incredibly good programmer, who could make GPU boards cooperate with each other and really do convolutional neural nets much more efficiently than anybody else had been able to program them.
15:24 Speaker 01 And Ilya had a lot of vision. He realised that, given the technology we had in the lab at the time that we'd been using for speech recognition, we should be able to win the ImageNet competition. And he was very insistent with Alex and with me that we had to do the ImageNet competition. He was so insistent that he sort of pre-processed the images to make it easier for Alex, so Alex would actually do it. I contributed a brilliant management technique. So Alex at that point had to do what's called a depth rule, which is a sort of review of the literature, a review of a particular topic in the literature. Then you have to review it in such a way that you
16:05 Speaker 01 made it clear that you were ready to start doing research. And he didn't want to write this thing. He writes very well, but he doesn't enjoy it. He'd rather be programming. So he didn't want to write his depth rule. He was behind schedule with it. And I was his advisor, so I was meant to make him do it. So I said, look, Alex, every week that you, every time you increase the score by one percent, you can delay your depth rule by another week. And he delayed his depth rule by a lot of time. And then, so when he was meant to be doing his depth rule, he produced this
16:43 Speaker 01 programme, that's one of the most important papers this century in AI. It's not that original, but it really demonstrated, they did a really good job of showing that, yes, neural networks really do work for doing object recognition. And so actually, that was basically, he then didn't do any more research at UFT. That should have been his thesis, the thing he'd done instead of his depth rule. But in the end, he decided he didn't need a thesis.
17:16 Speaker 00 So when did the intersection occur between yourself and your two students and using Jensen's GPUs?
17:28 Speaker 01 Okay, so what happened was in about 2006,
17:35 Speaker 01 I'd been visiting, I think it was that one, I'd been visiting Rick Zaleski in Seattle who worked for Microsoft. And at that point, I couldn't sit down. I traveled by train. I couldn't fly and I couldn't drive a car. And the train between Seattle and Vancouver, I think, wasn't working. And so Rick drove me up, lying in the backseat of his car. And we talked a lot. He had been a student of mine quite a long time earlier, like 20 years earlier.
18:06 Speaker 01 And he explained to me that we ought to be using GPUs because neural nets are basically matrix multiplies and GPUs are optimized for doing a whole lot of matrix multiplies.
18:17 Speaker 01 And I thought about that a bit. And then I think the next year, probably 2007 or 2008,
18:25 Speaker 01 we started buying GPUs. And
18:31 Speaker 01 they really improved things a whole lot. They made things go like more than 10 times faster. Which was like, you know, five or 10 years progress in CPUs. And it became very clear that one of the reasons neural networks hadn't worked as well as we hoped was we weren't using enough compute power. We should have seen that earlier because in the 1990s, two different groups, in the 80s or 90s, two different groups showed that neural networks were actually comparable with the state of the art for speech recognition. And those were groups at ICSI in Berkeley and a group in Britain.
19:14 Speaker 01 And both groups used unconventional hardware. The group at ICSI used DSP chips connected by Xilinx chips so they could get high bandwidth communication. And put a lot of work into getting more computation that way. And showed that it really worked for speech recognition. And Tony Robinson in Britain used transputers. Which were a funny kind of parallel computer, which were hard to program. Which showed that if you went to all the effort of programming them, then you could make speech recognition work much better. And so we should have drawn the lesson that what was needed was more computation. But it took us a long time to realize that. That that was the main bottleneck.
19:54 Speaker 00 And then when did you get sort of recognition from Jensen's colleagues and himself that. You know, there's something here between the work you're doing and the work they're doing.
20:06 Speaker 01 Okay, so it was very interesting. In two thousand and nine. Um, I went to a NIPS conference and we had. Neural nets working very well for part of speech recognition, the acoustic modeling part where you. Look, you essentially look at a spectrogram. And take the middle frame of the spectrogram and say, which piece of which phoneme is that? You divide phonemes up into three pieces and actually which piece of which phoneme? Is this middle frame of the spectrogram? Um, and we could make neural nets do that better than anything else. And that was using GPU boards to train the neural net. And at that conference, I gave an invited talk.
20:48 Speaker 01 And I told a thousand that the future of machine learning with.
20:53 Speaker 01 A GPU board, if you want to do machine learning, they should all go but go and buy an Nvidia GPU board. And then when I got back, I sent mail to Nvidia. Saying, I just told a thousand machine learning researchers to buy your boards. Could you give me a free one? This is a poor academic who has to beg. And they didn't reply. And so we knew in two thousand and nine that this was the future of machine learning. Nvidia didn't really realize it until the AlexNet in two thousand and twelve. Maybe they realized it a bit sooner. Um, actually. Jensen came and gave a talk in Toronto and at the end, he said, you know. We realized as early as two thousand and twelve that this was the future. GPU boards with the future of machine learning.
21:36 Speaker 01 And I pointed out, I sent mail to them in two thousand and nine, but they didn't respond. Um, he immediately gave me a free board.
21:44 Speaker 01 What he should have said was, why didn't you invest in Nvidia in two thousand and nine? If I'd done that, I'd be a billionaire now.
21:53 Speaker 00 Yeah, I mean, just a magical sort of coming together. You know, Jeff Dean, he's been at Google since I think nineteen ninety-eight or nineteen ninety-nine. And he was an early, sort of recognized the value. When did you and Jeff Dean at Google sort of come together and sort of share your work? And he was interested in your work and things like that?
22:16 Speaker 01 Okay, so Jeff actually did a master's thesis. I think this is a master's thesis, or maybe it's his senior project on neural nets. So he actually knew about neural nets and backpropagation. But then he went off and wrote a whole lot of the software at Google. He's really good at software. And then he talked to Andrew Ng, and Andrew Ng convinced him to set up the Brain Team. And after a while, Andrew Ng left to go and do Coursera and suggested they recruit me. So I went to work there one summer as a visiting scientist for a couple of months. And that was the summer of two thousand and twelve. And Jeff and I got along extremely well. I really liked him. He knew a lot more about software than I did.
23:01 Speaker 01 At that point, I knew more about neural nets than he did. I'm not sure I do anymore.
23:07 Speaker 01 And so we had a very nice collaboration with me being a visiting scientist there. And then when we got the spectacular results with AlexNet, we set up a little company in order to facilitate an acqui-hire. And we had an auction, and I really wanted Google to win because I really wanted to come work with Jeff. And we sort of organized it so that Google won.
23:32 Speaker 00 It'll be interesting if some of the early people like yourself and Andrew Ng and others, do you get together and have your own kind of like private party?
23:42 Speaker 01 Okay, so at CIFAR, Andrew Ng was one of the people involved in the program. And so Yan and Yoshua and Andrew Ng and me would all meet up regularly a couple of times a year. Typically, we'd have a big workshop before NIPS, sorry, Europe's. And we'd have some other, one other workshop in a year. So we sort of kept up with the air of the research and got to know each other pretty well. Andrew was an important part of the CIFAR program.
24:10 Speaker 01 And so CIFAR helped a lot there.
24:12 Speaker 00 Yeah.
24:13 Speaker 01 And that's why Andrew Ng knew me pretty well by the time Jeff Dean was wondering who to replace him with.
24:19 Speaker 00 Then I guess Jeff would then join that little sort of get together afterwards, right? Or no?
24:25 Speaker 01 No, he was in industry and this was mainly academics. Yeah, and Jeff was too busy running the brain team by then.
24:36 Speaker 00 So that's a remarkable journey. Now, let's sort of move up to more of the present. Are there any areas of machine learning or AI and machine learning that you think are really interesting? I know that Yala Karun sort of feels that it's got to be more hybrid or there's other things that have to be involved. But it seems like Ilya and yourself and others are believing that what you've done is, you know, where it's going to be. You know, and do you see any kind of new research that looks really interesting on thinking of Komagorov or Rital networks and then, you know, there's categorical architecture and things like that.
25:20 Speaker 00 Do you look at some of those things and you think that's interesting? Or are you sort of doubling down on your work, right?
25:28 Speaker 01 I guess there's really two questions. One is, will there be radical new developments that change neural nets a lot? And I think there probably will be. But the second question is, are they necessary to make neural nets get to be as smart as people? And I think probably not. So for a long time, people, particularly people from good old fashioned AI, have been saying, these neural nets, they're just doing sort of instant intuition. They'll never be able to reason. Well, they were wrong, right? The neural nets can reason quite well now. And they don't do it by having a neural net feed stuff to a symbolic reasoning engine. That's not how it works. It's done by the neural net.
26:11 Speaker 01 So you can ask a neural net now, one of the good chatbots, you can ask it. Sally has three brothers. Each of her brothers has two sisters. How many sisters does Sally have? And people have to think quite hard. A lot of people get that wrong. They either forget that the two sisters of the three brothers are the same two sisters, or they forget that one of them's Sally.
26:35 Speaker 01 So a lot of people get it right, but also a lot of people get it wrong. So it's a reasoning problem that's sort of some people struggle with. Neural nets used to not be able to do it at all. Now, a lot of them can do it. And there was nothing special put in to do reasoning. They were just made bigger. And probably trained on more data than involved reasoning.
26:58 Speaker 01 So I don't think it's necessary to come up with any radical new radically new to make these things comparable with people. But I think as a matter of fact, we probably will come up with radically new things and they will help.
27:11 Speaker 00 Yeah, I mean, that's really interesting. Have you looked at some of these alternative? Systems do you do you look at them at all? I think at Google, didn't you look at analog networks or?
27:22 Speaker 01 Oh, I was, yes, but I wasn't looking at them. For to make them better reasoning or anything. I was looking at them to make them use less electricity. And in the end, I concluded that.
27:35 Speaker 01 Analog networks are very interesting. And once you can learn to do things. You can get rid of the separation between hardware and software. So in an analog net, to make it really efficient. You want to use all the quirks and peculiarities of the particular piece of hardware you've got. So the weights in my brain. Are relying on the way my particular individual neurons work. And that's why the weights are no use for your brain. We can't share the weights. And so to make analog computing efficient, you have to make it what I call mortal. That is, you have to tie it to a particular piece of hardware and no one waits for that piece of hardware. When the hardware dies, the knowledge is gone.
28:19 Speaker 01 And different pieces of hardware can't share their knowledge by sharing weights. They can share their knowledge by one piece of hardware produces strings of sentences and other pieces of hardware tries to mimic it. Which is how we share knowledge with neural nets. But they can't average their weights. Whereas if you have a digital model. You can have two different bits of hardware that do exactly the same thing at the level of the instructions. So you can run the same artificial neural network on both bits of hardware. And one bit of hardware can look at one bit of the internet, another can look at another bit of the internet. They can each figure out how they'd like to change their weights. And then you can average their weight changes. So both bits of hardware benefit from the other one's experience.
29:05 Speaker 01 That's what we can't do. We don't have an efficient way of sharing. When two copies of the same model that are digital and therefore use weights the same way. When they average their weights, if they've got a trillion weights, they've got a bandwidth for sharing knowledge of the order of a trillion bits. When you and I share knowledge, like we use a sentence and you change your synapses, so you might say the same thing. It's a rather simple model, but roughly that. And the bandwidth of that is how many bits are there in a sentence I produce? Well, a few hundred. I mean, you're talking trillions to a few hundred.
29:41 Speaker 01 So they're just much, much better than us at sharing knowledge. That's what's special about digital systems. We're not digital. They are. And they can share knowledge between identical copies of the same model running on a different hardware. And that makes them a much better form of intelligence. And I find that very scary.
30:00 Speaker 00 Yeah, I mean, you've done a series of talks. I think there was one at Oxford.
30:05 Speaker 01 Yeah.
30:06 Speaker 00 And after you got your Nobel Prize in Physics, you're also going out. And then even in the sort of the probability, can you talk about that? They don't know how you.
30:18 Speaker 01 Yeah, obviously. Obviously, it's a very subjective probability. They're not probabilities computed by doing a bunch of calculations. So subjectively, it seems to me crazy to think the probability of these things, when they're smarter than us taking over, will be less than 1%. I think that's what Jan thinks. He won't give me a number. Someone like Yudikovsky thinks the probability is greater than 99% and we should start bombing the data centers now. I think that's crazy, too. We're going to be dealing with something that's very unknown, but something that we have some control over. Then maybe not as much as we want. And so the probability is somewhere between 1% and 99%. And maybe I should have left it at that. But then I had to guess.
31:02 Speaker 01 And I think it's a lot more than 1% that these things will take over from us. And my way of saying it's a lot more than 1% is 10 to 20%. If you say 50%, it sounds kind of very scary.
31:16 Speaker 01 Someone might be able to convince me it was fifty percent. Um, but people are quite ingenious and we haven't yet done a lot of exploration of how we might stay in control of things smarter than us. We don't know if it's possible, but I assume that eventually we'll realise we have to put a lot of work into that and we may well come up with something. And that's why I think it's probably less than fifty percent. I hope it's less than fifty percent.
31:44 Speaker 00 Because then there's speculation if we aren't able to come up with something and then what would the future of everything look like?
31:51 Speaker 01 And I think we're toast.
31:55 Speaker 00 That that's a remarkable statement.
32:00 Speaker 01 Well, Elon Musk thinks they'll get smarter than us and they'll keep us around because we make the world more interesting. And I think they'll be so much smarter than us. I think that's like saying, um, you know, we'll be smarter than cockroaches, but we'll keep them around because they make the world more interesting. Well, cockroaches aren't that interesting. And I think that's how super intelligence may look at us.
32:24 Speaker 00 Yeah, or it's sort of like, um, there's an ant and is it contemplating, you know, quantum gravity collapse and similarities and artificial intelligence and then then there's us and then what do we think of as ants and what will super intelligence think of us as, I guess, but.
32:40 Speaker 01 Yeah. So, yeah, I so I think we should be seriously worried about the possibility that, um, digital intelligence, because it can share so much better and therefore learn so much more. That's why G P T four can know thousands of times more than any one person because you're able to share between models running on different piece of hardware. Um, because of that, it's going to get much more knowledgeable and much smarter than us and we should be seriously worried about can we stay in control? Now, there's a few people like I think Rich Sutton, who thinks, well, you know, intelligence evolves, that's going to be the next stage of intelligence. We're going to be history.
33:23 Speaker 01 Um, and that's just how it is. And that may be true, but we're people and what we care about is people. And so as people, I think we should try and staying, we should try and keep people in control. I mean, the reason I eat cows is because I like people more than cows. Um.
33:43 Speaker 01 I'm sort of people centric.
33:47 Speaker 01 And so I think that's a sort of moral justification for us, at least in human morals, for us trying to stay in control.
33:56 Speaker 00 You know, it's interesting, as you went back, let's say, two and a half years. Everything that we're seeing right now and the use of these models, right? Would have been considered either science fiction, or the majority of people would say, well, it's fifty years old or thirty years old.
34:14 Speaker 01 And maybe, maybe if you went back two years, they would, but if you went back ten years, if you went back ten years and asked people, by twenty twenty five, will we have something like this? That has said, oh, no, no, no, that's going to be fifty years off. I would have said that.
34:30 Speaker 00 Yeah, I mean, you know, there is AI that creates music, right? And it's gone through multiple generations. You're seeing Sora and there's the addition from Google, which is, you know, is doing sort of really interesting generation. But you also have the art AGI benchmark, right? Where for years, so two thousand twenty was like scoring zero and then went up to maybe five percent. Some people are getting up to fifty percent and then you have the old one.
35:03 Speaker 01 It's like eighty percent or seventy percent or something, right?
35:06 Speaker 00 Well, the old three is, if you give it compute, it's eighty seven point five percent.
35:13 Speaker 01 Right, and that's pretty amazing.
35:16 Speaker 00 Yeah.
35:17 Speaker 01 It'd be nice to know, I don't know the literature well enough to know how tailored it is to that particular dataset. There's always the problem of have they somehow tailored the training so that it's good on that dataset? But, but nevertheless, it's very impressive.
35:31 Speaker 00 Yeah, but Shalay actually did the testing. So, I mean, he so, I mean, he created the art benchmark, right? And now they're creating Arc two, so they had that's got a line in the sand where nobody was crossing it in five years. So there's a lot that comes out and there's this eighty five percent line and they cross that line and they go, oh, well, I guess we're going to have to create another version.
35:56 Speaker 01 So this is happening a lot in AI, like in twenty twenty two, that Gary Marcus, I think, said, AI is hitting a wall. It's, it's, it's not going to progress much further. Well, that was dramatically wrong.
36:11 Speaker 01 And we keep getting these dramatically wrong predictions about how neural nets aren't going to go much further without radically new stuff happening, and they just keep doing better and better.
36:22 Speaker 00 Yeah, I mean, definitely Gary sits on one end.
36:27 Speaker 00 And I could see the kind of battles online, right?
36:31 Speaker 01 I think he sits, I think he actually sits slightly beyond the end.
36:37 Speaker 00 Um, so that, you know, that's a kind of an interesting, and then there's, it's a spectrum, right? You get people in between.
36:43 Speaker 01 Yeah.
36:44 Speaker 00 And you definitely have come out in a, in a really prominent way, um, you know, expressing your opinion and about the concerns that you have, and that we have to think about it in a thoughtful way.
36:58 Speaker 01 Yeah. So another thing that influenced me a lot partly was the analog versus digital and realising why digital is so much better because it can share.
37:07 Speaker 00 Right.
37:09 Speaker 01 But there are other things that influenced me. So a lot of people think that these large language models, although they can predict the next word and answer questions and things, they don't work the same way we do. Now, if you look at the history of these models, the very first models that used backpropagation to train neural nets to predict the next word, um, I think I implemented the first one of those in 1985. It was very small, but it wasn't designed as a technology. It was designed as a model of how people can learn the meanings of words from strings of symbols.
37:49 Speaker 01 So the best model we've got of how people deal with language is these large language models.
37:58 Speaker 01 We, the people who believe in symbolic AI, we're just absolutely certain that the way language has to work is you have some kinds of symbolic expressions in your head, and you have rules for regulating them, and that's how it has to be. It has to look like logic, a funny kind of logic, um, but it has to be like that. And they didn't really think that was hypothesis. They thought that was an obvious truth.
38:26 Speaker 01 Newell and Simon, Newell in particular, thought, talked about the symbol processing hypothesis, but he thought it was obviously true.
38:36 Speaker 01 The way language actually works is words come in, you convert those into patterns of neural activity, feature vectors, and then you have interactions between the feature vectors of words in the context. And you can now predict the next word. You can do other things too. But the symbols are turned into feature vectors and the interactions between features are where all the knowledge is. Now with transformers, those interactions got to be of a more complicated kind, but there's still interactions between features. That's where the knowledge is. There are no symbol strings in the head and there are no symbol strings in neural nets. If they want a symbol string, they generate it at the output. But inside, it's just features and interactions between features.
39:21 Speaker 01 So
39:25 Speaker 01 I think many people still don't really believe that that's how people work. And so they think there's a big difference between these neural nets and people. And there may be differences in how we get the gradient and what kind of experience we have and how much experience we have. But the basic mode of operation for attaching meaning to symbol strings is the same. You turn symbol strings into feature vectors for each of the symbols. And the feature vectors interact with each other and refine each other. And that's your model. Those active features, that's what the model is. And it's a very flexible modeling system. You can model more or less anything that way. And the nice thing is because the symbols have names,
40:09 Speaker 01 when you make a model like that, you can communicate it to other people slowly by producing the names of the symbols, not by transferring feature vectors. But we have this very flexible modeling system called language. And the linguists never had a model of how meaning works. They always thought later on we'll get meaning, for now we'll do syntax.
40:33 Speaker 01 We have a model of how meaning works. Meaning is associating symbols with features and then having interactions between those features. You can model anything that way. And that's what we're doing with the language.
40:49 Speaker 00 You know, it's really quite fascinating that this work that dates back decades, decades and decades is ultimately proving itself out, right? And
41:00 Speaker 01 Yes. So it took, it took a long time. So my little language model of nineteen eighty five,
41:07 Speaker 01 which learned about family trees and could do a little bit of generalization. It had about a hundred training cases and about a thousand weights.
41:16 Speaker 01 It took nearly ten years before Yoshio Bengio made a similar model. He actually made two different models, one of which was similar to my original model. They did the same thing, not with toy data like I had, but with real natural language. They showed that basically the same technique worked for real natural language.
41:38 Speaker 01 And then it took another ten years before linguists started using feature vectors. And then it took another ten years before they invented key transformers. And then everybody started building these large language models. So it was a slow process. And basically it was a slow process, I think, mainly because of computer power, partly because of
42:02 Speaker 01 institutional inertia, but but largely because you needed the computer power. Yoshio couldn't have done his demonstration of these language models working on real language with computers we had in nineteen eighty six. They just weren't fast enough.
42:20 Speaker 00 Are you still actively, you're emeritus professor. So do you still do fairly active research or now?
42:31 Speaker 00 Or guide or somehow get involved in some of this or like what's the status of your work today?
42:39 Speaker 01 Okay, reached the age of seventy five. I've been working really hard for the last fifty years. And I decided, and I was much slower at programming, I will forget what the variable names stood for and things like that. And I'd do a copy and paste and forget to change the things when you pasted it. The sort of standard errors that initial programmers make. But when your short term memory is not so good, you forget those things. And it got very irritating that I couldn't program as well as I used to. So I decided it was time to retire. And since I was going to retire anyway at seventy five.
43:13 Speaker 01 I decided on the way out, I would just mention that this stuff was dangerous. And I wasn't expecting it to get hit by tide wave of media. And so now I don't do research anymore. I mean, I think about research occasionally, but mainly I see my role now as trying to explain to people how neural nets work and that they really are rather like us. And trying to warn people about the risks and trying to encourage governments to impose regulations on the big technology companies, forcing them to do more safety research. Because it's only the big technology companies that have the resources. And I want things to go in kind of the opposite direction to the way they went at OpenAI. So they started off being very concerned with safety and being willing to dedicate a lot of
43:57 Speaker 01 compute resources to safety. And as time went by, they got less and less interested in safety. And the leading safety researchers like Ilya left. And so they've gone, they're moving very rapidly away from concern with AI safety. They talk about it, but if you look at what they do, how they spend their compute cycles is just making better and better models.
44:22 Speaker 01 I think governments need to regulate companies to use their compute resources to to provide their compute resources to safety researchers. So that we can study the, not only the long term existential threat that these things might eventually take over, but also all the short term threats to do with cybercrime and bioterrorism and joblessness. We we need more research on all those things than discrimination and bias. The thing I've talked most about though is the long term existential threat because that's the thing where people, particularly people who believe in old fashioned symbolic AI, said, oh, that's just science fiction, that's just nonsense, that's never gonna happen.
45:06 Speaker 00 You know, there are safety institutes like in the UK, also in the US and Canada are moving in these directions.
45:16 Speaker 01 But if you ask how much computation resources do they have? The one in Britain has enough money so it could fine tune a big model. But it doesn't have enough money to be training big models from scratch. It's only the big companies can do that.
45:33 Speaker 00 Yeah, but then you have our DeepSeq 3 from China, you know, 5.6 million, or at least they they're getting 5.6 million compute, right? To do something that's comparable and then they actually released it open source.
45:48 Speaker 01 Which is. Okay, I have other things to say about open source. It's not open source, it's open weights, right? People shouldn't use the same word. Open source, you release your code, like the Unix kernel or something, so that people can find bugs in it. And people look at it and they say, oh, that looks like a possible bug. When you release the weights, people don't look at the weights and say, oh, that weight looks a little bit too big. And when you release the weights, they can then fine tune it to do nasty things. So I don't think people should be releasing the weights of big models. I think it's like releasing physical material. If you ask, why didn't we all have our own atom bombs?
46:32 Speaker 01 Well, why doesn't every small town have its own atom bomb? Um, as well as a well regulated militia. Um, then the reason is, it's hard to purify uranium, to separate the acidopes of uranium. Um, if you were to give people the physical material, um, it's then much easier to make a bomb. So the real restriction is on refining the uranium. And the real restriction for these large I O models is learning the foundation models, learning the big models in the first place. That's what takes huge resources. And it may take less resources now, but it's still quite a lot.
47:12 Speaker 01 If you then release the weights, people can now fine tune them to do all sorts of things for only a few million dollars. I think they're crazy to release the weights.
47:25 Speaker 00 So you must have interesting conversations then with Yala Kud and sort of the direction.
47:33 Speaker 01 Yes, we're friends, but we have very different views.
47:37 Speaker 00 I mean, they released something, the large concept model, right?
47:41 Speaker 01 Yes. No, my belief is, and I'm not sure this is true, but my belief is they intended to give the weights to just a few researchers and they accidentally released the weights to lots more people than they intended. I'm not sure this is true, but I've been told this. And then they decided they'd make that a feature. And so my summary of that is move fast and break things and leave them broken.
48:08 Speaker 00 You have a great sense of humor too.
48:12 Speaker 01 You have to if you're looking at the extinction of humanity.
48:17 Speaker 00 Do you, do you, if you're not speaking at one of these big conferences like ICML or or NearUp or things, do you actually attend still and do you still read the occasional paper?
48:28 Speaker 01 I read some of the papers.
48:32 Speaker 01 Yeah, the last time I attended NIPS, it was very difficult because if I tried to walk anywhere, I got a big, big sea of students wanting selfies who were very insistent. It was quite annoying.
48:49 Speaker 01 And I think it will be worse now.
48:51 Speaker 00 Yeah, for sure. And you have your past colleague and friend, I mean, or current friend. I mean, Ilya got off and started his own safe ASI endeavor. Do you still keep in touch with him periodically?
49:07 Speaker 01 I talk to him regularly, but we don't talk about what his company is doing because I think that's probably this commercial secret set.
49:17 Speaker 01 So we don't talk so much about the details of the research, but we talk about all sorts of issues.
49:21 Speaker 00 Right. I'm sure you talk about OpenAI as.
49:25 Speaker 01 In fact, we exchange emails today.
49:30 Speaker 01 Actually, we don't talk about OpenAI. And I don't actually know what Ilya feels about OpenAI right now. Because, of course, he was their chief scientist. A lot of the early success of OpenAI is down to Ilya. Musk was the person who worked with Sam Altman to found it. And Musk, his sister, he wouldn't start unless he got Ilya. And worked very hard to get Ilya.
49:55 Speaker 01 So I think Ilya probably has a lot of lingering allegiance to OpenAI. But I don't really know what he feels about it. So we try not to talk about things that might be sensitive.
50:11 Speaker 00 So this is kind of like a greenfield sort of area, just for a few minutes. Is there any areas you want to explore that we didn't talk about? That you want to get all into, that sort of into the interview itself?
50:24 Speaker 01 I'm interested in the issue of, do these things have subjective experience? So most people would, people use various terms like consciousness or sentience or subjective experience. And they also talk about emotions and feelings. And a lot of people, particularly people in these humanities, are very confident these things don't have any of that. Because that's uniquely human. And if you ask them to justify that, they can't. They're just convinced it's uniquely human. If you look at our record, we've got a very bad record. People think much too much of themselves. They're really just apes with pretensions.
51:07 Speaker 01 But they think we were made in the image of God, and that's why we're at the centre of the universe. And if you look at our history of beliefs about people, we're incredibly egocentric.
51:18 Speaker 01 The idea that machines couldn't have emotions seems obvious to most people. And it seems like nonsense to me.
51:26 Speaker 01 So one interesting experiment that puzzles people is, suppose I took a neuron in your brain.
51:33 Speaker 01 One neuron, and I replaced it by a little piece of nano electronics that behaved exactly the same way as the neuron. It received the inputs the neuron received, maybe even including peptides and things in the blood. And it produced the same outputs as the neuron produces. So it might even inject peptides into the blood, but it also produces actual potentials and things. And I just replaced one neuron with a piece of nanotechnology that behaved in exactly the same way. Presumably, you think you'll still be conscious. Just replacing one neuron wouldn't stop you being conscious, right? But you know about arguments by induction, and so you can see where this is going.
52:15 Speaker 01 Now we replaced two neurons, and you're still conscious, right? Now we replace all the neurons. And at what point do you cease to be conscious? It's obvious you're still going to be conscious. So if you're a materialist, you have to believe that machines can be conscious.
52:35 Speaker 00 Yeah, it's, um. I guess with all the sort of fine tuning that occurs and then the safeguards that are put in them, you, you may not get really the actual kind of processes being clear, especially from a consumer standpoint. Because you're too late in terms of your interaction. Have you played around with sparse auto-encoders and Gemoscope? And there's something called Neuropedia, where you can see the activations happening and things like that. Have you played around with any of that?
53:13 Speaker 01 Not recently. I spent a long time in the past playing around with looking at activations. And I'm fairly convinced that once you have fairly nonlinear neurons, and you have multiple layers, you can get stories about what the neurons in the intermediate layers are doing. You need to be very cautious about whether you believe those stories. So the problem is that the effect of a neuron changes depending on the state of the other neurons. In a nonlinear system.
53:46 Speaker 01 It's not going to be easy to say what a neuron deep within the system is actually doing in terms of inputs and outputs. And the inputs and outputs are the only thing we can understand. So if you have one layer of neurons, one hidden layer, you can look at the connections to the pixels and you get a receptive field and you can sort of feel you understand that. Even there, you can be misled. But if you have multiple hidden layers, I'm deeply suspicious about whether you're ever going to really understand what the neurons in the middle are doing. Because what they do means very different things depending on which other neurons are on.
54:24 Speaker 00 Yeah, I mean, it's really fascinating. And then, you know, you get these red teams taking a look. And then they say, hey, this thing replicated itself to another server. Because
54:37 Speaker 01 That's quite worrying, right?
54:39 Speaker 00 Yeah, I'm. Any final thoughts then in their closing sort of, I guess you've given a series of recommendations throughout the entire interview, but any final recommendations and questions that you want to leave the audience?
54:56 Speaker 01 So one very minor thing. People talk about discrimination and bias. And I think that's one of the few problems that I'm not so worried about. And of course, some people will say that's because I'm an old white man and I didn't experience it. And there may be some truth to that. But if your goal is not to make an AI system that doesn't discriminate and isn't biased, but your goal is to make an AI system that's less biased than the system it replaces, then I think we can make a lot of progress there. So the difference between a person and AI system is that you can freeze the weights in the AI system so you can measure the bias. If you try and measure the bias in a person, they realize you're measuring
55:40 Speaker 01 them and they start behaving differently. That's what I call the Volkswagen effect. And
55:47 Speaker 01 I would like that terminology to catch on. And so it's much harder to correct bias in people than it is in AI systems. So suppose you take data of old white men deciding whether to give young black women mortgages. You're probably going to find a lot of bias there. If you train an AI system in that data, you'll find a similar bias. But now you can look at the trained AI system and you can measure the bias and you could do something about it. You're not going to correct it perfectly, but you can make it less biased. So now let's take a gradient descent approach. Start with whatever the existing system is.
56:26 Speaker 01 Replace it by a less biased AI system and then just keep replacing those AI systems by less biased systems. Don't aim for perfection in one go. You're never going to get that. You're never going to get perfection at all. But you should be able to make it a lot less biased.
56:43 Speaker 00 That's interesting. So you just triggered another sort of question in me. If you were a student.
56:53 Speaker 01 Sorry, my, we got a bad internet connection there.
56:57 Speaker 00 So I just triggered another question and that is.
57:02 Speaker 00 If if you could start in doing research, let's say you're you're now twenty six. And today, what would be the areas of research that you would focus on?
57:19 Speaker 01 I don't know. I really don't know. I'm still very interested in how the brain gets gradients. I think it's probably not backpropagation and it's probably not Boltzmann machines.
57:31 Speaker 01 So from the point of view of a psychologist or neuroscientist, there's still very interesting open questions about how the brain finds gradients. I think we've established that if the brain can find gradients for synapses, then it can learn amazing things by following those gradients. We didn't know that before for sure. We hoped that was true. We've now established that that's true. If you can get the gradients for connection strengths, you can do amazing things. And we know backpropagation is one way to get the gradients, but we suspect that's not what the brain is doing. Probably because the brain's solving a different problem. So in backpropagation, that's the problem we're solving is if I have a huge amount of experience, how do I squeeze all that knowledge into not many connections, like only a trillion?
58:15 Speaker 01 The problem the brain's solving is, if I have very little experience, like only two billion seconds worth, how do I make use of a very large number of connections, like a hundred trillion, to get as much as I can out of that very limited experience? And those are different problems, and it may be that backpropagation is not the best method for solving that second problem of how to use a
58:45 Speaker 00 Yeah, you've frozen on my screen.
58:47 Speaker 01 Right.
58:51 Speaker 00 You know, there's this idea of eighty to a hundred billion neurons and then a hundred trillion connections, but there's computational aspects beyond what's happening, right? I mean, they're seeing some kind of computation in everything that's in the human body. So maybe that hundred trillion is, maybe it's five hundred trillion.
59:14 Speaker 01 Um, yes and no. A lot of the fast computation to do is done electrically.
59:22 Speaker 01 And so individual neurons are more complicated. There's more complicated nonlinear interactions going on in the dendritic tree and so on. But in terms of the raw amount of computation, I think
59:37 Speaker 01 if you made a neural net with a hundred trillion connections, just using simple rectified linear units, I think that would be comparable amount of computation to what the brain's doing.
59:48 Speaker 00 So you answered all the questions, and this comes to the end of the interview, and you also sort of posted some reflections. So Jeff, again, congratulations. I've been a longtime follower of what you do, and I think it's marvelous that you've come out. And thank you for taking the time and doing the interview with me. So I hope you'll be willing to do this again at some point in the future.
1:00:16 Speaker 01 Yeah, as long as it's not too soon, yes.
1:00:21 Speaker 02 Thank you for listening to the brand called You Videocast and Podcast. A platform that brings you knowledge, experience and wisdom of hundreds of successful individuals from around the world. Do visit our website www.tbcy.in to watch and listen to the stories of many more individuals. You can also follow us on YouTube, Facebook, Instagram and Twitter. Just search for the brand called You.
Previous Reinventing Identity and Leadership | Amit Paul, Co-founder, Innrwrks and Host, the World of Wisdom Podcast Next Defining Leadership and Legacy | Air Marshal Sanjeev Kapoor (retd), AVSM, VSM, PhD, Indian Air Force Page updated November 15, 2026 · transcript and summary are machine-generated