Pod Insights a16z Podcast a16z Podcast | Artificial Intelligence and the 'Space of Possible Minds' About this episode What is A. I. or artificial intelligence but the 'space of possible minds', argues Murray Shanahan, scientific advisor on the movie Ex Machina and Professor of Cognitive Robotics at Imperial College London.
In this special episode of the a16z Podcast brought to you on the ground from London, Shanahan -- along with journalist-turned-entrepreneur Azeem Azhar (who also curates The Exponential View newsletter on AI and more) and The Economist Deputy Editor Tom Standage (the author of several tech history books) -- we discuss the past, present, and future of A. I. ... as well as how it fits (or doesn't fit) with machine learning and deep learning.
But where are we now in the A. I. evolution? What players do we think will lead, if not win, the current race? And how should we think about issues such as ethics and automation of jobs without descending into obvious extremes? All this and more, including a surprise easter egg in Ex Machina shared by Shanahan, whose work influenced the movie.
Transcript 0:00 Speaker 3 Hi everyone, welcome to the A6 and Z podcast. I'm Sonol, and today we have another episode of the A6 and Z podcast on the Road, a special edition coming from the UK. We're in the heart of London right now. I'm here with Marie Shanahan, who is a professor of cognitive robotics at Imperial College in London, and he also consulted on the movie Ex Machina. And so if you didn't like the way that movie turned out, we don't want to put any spoiler alerts, you can blame him. And then I'm here with Tom Stanish, who's the deputy editor at The Economist and also the author of a few books.
0:30 Speaker 2 Yeah, six books. The most recent one was writing on the wall, which was uh the history of social media going back to the Romans, and probably the best known one in this context is the Victorian Internet, which is about telegraph networks in the 19th century being like the internet.
0:42 Speaker 3 That's great. And I'm here with Azia Mazar, who publishes an incredibly interesting and compelling newsletter that I'm subscribed to, the exponential view. He used to be it the Guardian and the Economist. And then most recently founded and sold a company that used um machine learning heavily. So welcome everyone. Thank you. Hello. So today we're going to talk about a very grandiose theme, which is AI, artificial intelligence and just sort of its impact and movements. This is really meant to be a conversation between the three of you guys. But Murray, just to kick things off, like you um you consulted on the movie Ex Machina. Like what was that like?
1:14 Speaker 0 Oh, it was tremendous fun, actually. Uh so I got an email out of the blue from Alex Garland, famous author. So that was very exciting to get this email, and he the email said, uh, oh, my name's Alex Garland. I've written a few books and stuff. And um uh and I read your book on um on consciousness, embodiment in the inner life, and uh I'm working on a film about artificial intelligence and consciousness, and would you like to kind of get together and and uh and talk about it? So of course I jumped at the chance and we uh met and had lunch and I read through the script and I uh he wanted a bit of feedback on the on the script as well, whether it hung together from the standpoint of somebody working in the field. Um and then I'm then we met up several times while the movie was being filmed, and and I have a little Easter egg in the Oh you do.
2:02 Speaker 3 What what was your Easter egg? I've seen that movie three times in the theater, so I
2:05 Speaker 0 will remember it. Oh, fantastic. So there's a so there's a point uh in the film where Caleb is typing into a screen to try and crack the uh the security, and then some code flashes up on the screen at that point, and that code was actually written by me.
2:18 Speaker 3 Oh, yeah. And it's
2:19 Speaker 0 real code, it's not the usual rubbish booth. And if it just sort of flashes up. But what it actually does is if you actually um type it in uh to a Python interpreter, it will uh print out ISBN equals and the ISBN of my book.
2:32 Speaker 2 Oh, that's so like it as well. I'm even I'm even more
2:35 Speaker 0 thrilled.
2:36 Speaker 3 I think it's fascinating that you say that the part that you didn't go into detail about is the name of the second part of the title of your book, embodied consciousness.
2:44 Speaker 0 Yeah, there's a long subtitle, which is uh cognition and consciousness in the space of possible minds. And I very much like that phrase, the space of possible minds. I think if you were to kind of pin me down on what I think is my most fundamental and deepest interest, it's this idea that what uh constitutes possible minds is much larger than just humans or even the animals that we find on this earth, but also encompasses the AI that we might create in the future, either robots or disembodied AI. There's a Hamiltonian
3:15 Speaker 1 space of possible minds. That's beautiful. Yeah,
3:17 Speaker 0 a huge kind of space of possibilities.
3:18 Speaker 1 So I mean it's a really interesting idea, and it's something that comes across in a couple of your uh your other uh books uh as well, which is this notion that um we think of intelligence as uh the quite often the artificial intelligence, that that plastic white mask that you see on the cover of many, many a film uh or book cover. Uh but of course, as we start to develop these new AI systems, they might take very, very different shape. They may be embodied in different ways, or they may be networked intelligence. So one of the areas I think is uh interesting is um what's happening with Tesla and the Tesla cars that learn from the road, but they all learn from each other.
3:53 Speaker 3 Now,
3:54 Speaker 1 where is that intelligence located and what was it, what would it look like, and where will it sit in your space of possible minds.
4:00 Speaker 0 Yeah, absolutely. It's a completely distributed uh intelligence and and it's not embodied in quite the sense of the of course a car is a kind of robot in a way, if it's a self-driving car, but it's not really an embodied intelligence, it's sort of dis disseminated or distributed throughout the the internet, and it's a kind of presence. So I can I can imagine that that in the future, rather than uh uh the AI necessarily being the stereotype of a robot standing in front of us, it's going to be something that sort of uh is is hidden away on the internet and is a kind of ambient presence that goes with us wherever we have.
4:33 Speaker 2 That's another sci-fi um stereotype there, isn't it? That's the uh the Univac or the Star Trek computer. But um, as I understand it, your work uh uh starts from the presumption that um embodiment is a crucial aspect of understanding uh intelligence and uh that which is why you're interested in both the robotic side and the uh intelligence side.
4:52 Speaker 0 So certainly I would have taken a stance uh um, you know, if you'd asked me ten years ago that that uh cognition and intelligence is inherently embodied because what our brains are really for is to help us get around in this world of three-dimensional space and complex objects that are move around in that three-dimensional space, and that everything else about our intelligence, our language, our problem-solving abilities built on on top of that. Um now uh now I'm not totally sure that it's not impossible uh that it's not possible to build uh AI that is kind of disembodied. Maybe I in in my latest book um uh I use the phrase vicarious embodiment, or I should say vicarious embodiment for a US audience.
5:33 Speaker 2 So it can kind of embody itself temporarily in a thing and then go somewhere else.
5:36 Speaker 0 Oh, well, that's a that's a that's another thing. You can have sort of avatars, but uh what I mean what I mean by vicarious embodiment is um is that it it uses the embodiment of others and to gather data. For example, the enormous repository of videos there are on the internet. There are zillions of of uh videos of people picking up objects and putting things down and moving around in the world. And so potentially it can learn in that vic vicarious way, everything that it needed to learn by being actually embodied it's it's
6:05 Speaker 2 this goes right back to the um neurological basis, I think, of some of your research. Because you started off doing symbolic AI and then moved over as kind of the whole field has to more of this um uh neurological approach. Um
6:15 Speaker 3 by neurological approach time, you mean more like in the deep learning sense.
6:18 Speaker 2 Well, exactly. As I understand it, part of your uh approach there was the idea that the brain itself can rehearse uh motor neuro um sort of uh combinations, uh and that that's how we kind of predict how the world will behave. We kind of say, what would happen if I did this? Which is very much like what the um deep mind AI is doing when it plays breakout or whatever, this kind of deep cue networks where you which is all about feedback based on predicted um actions and remembering how things worked out in the past. Certainly
6:45 Speaker 0 um I've I've uh always thought that this idea of inner rehearsal is very, very important, our ability to imagine um uh different possibilities. So watching
6:53 Speaker 2 YouTube videos of people doing things is a form of it, it can function as in a rehearsal at all. Well,
6:57 Speaker 0 or it or it if it if you have a system that can learn from that, the the sort of dynamics of the world and the statistics of actions and their effects and so on, then it can use that. So it's sort of builds a model of how the world works, and then it can use that model to uh to construct imaginary scenarios and rehearse imaginary scenarios. Actually, just going going back very quickly to to the to the deep mind DQN. So, in fact, one of the the for the um uh bit of work that they actually published, uh I think one of its shortcomings actually is that in fact it although it has done all that that learning about um what the right action to do in in the right circumstance is, it doesn't actually do inner rehearsal. It doesn't actually work through uh scenarios. It just
7:39 Speaker 2 remembering how things worked out in the past.
7:41 Speaker 0 Yeah.
7:42 Speaker 3 Murray, what act actually, what exactly then is inner rehearsal? Because I think I'm actually confused at we're describing three different things. There's sort of a predictive aspect, there's sort of this decision-making framework, and then there's also sort of a um something that reacts to the world in a dynamic environment that's constantly changing and reacting to that information in a very um proactive and intentional way. Those are all different qualities. So what is inner rehearsal exactly? So
8:05 Speaker 0 I I think that that that um the sort of architecture of in of intelligence is is putting all of those aspects together, really. So the so the inner rehearsal is is is um when we close our eyes, of course, we're not really necessarily gonna close our eyes, especially if we're on the underground in there. But uh it's when we close our eyes and imagine going through some particular scenario, imagine doing an action and and inwardly uh realizing that this would be a good or this would have a bad outcome. It's like a planning
8:32 Speaker 3 scenario and model-based reasoning. It's like planning. Some of the
8:34 Speaker 2 same bits of our brains light up, and if I imagine punching you, then actually the parts of my brain that will be involved in punching you are partly fortunately, uh, they're not actually punching you, but they're just envisioning that there is a similar thing. But the point is, there's more to it than just sort of thinking of the scenario. In some sense, the brain does rehearse the scenario in other ways, doesn't it?
8:53 Speaker 0 There's there's quite a bit of evidence that the way the brain does it, as you say, is to actually use the very same uh bits of neurological apparatus that it uses to do things for real.
9:03 Speaker 3 So the planning is almost interchangeable on the other.
9:11 Speaker 2 Um basically modified so that um in rather so that the part that normally suppresses the uh intention to act things out that you're rehearsing actually was taken away. And so the cat was imagining swiping mice and this sort of thing while being asleep.
9:24 Speaker 3 So that's actually kind of fascinating because it's a reversal of how I've always thought of the human brain, which is you're basically saying almost that there's always a bunch of scenarios and actions that can play out in any given moment in the brain, and that we're actually already acting on in essence by the neurological impulses that are being fired in the brain. But in reality, what's holding it back is some kind of control that's stopping something from happening as opposed to saying I'm gonna do X, Y, or Z and then acting on something intentionally. So it's more of a negative space thing than a positive. Yeah, or a kind of
9:50 Speaker 0 veto mechanism. Right. Yeah, I mean in fact, uh I think the I think that you've actually proposed there two very good rival hypotheses for what's actually what's going on. And I wouldn't want to uh to venture what I think is the uh is the is the answer uh there and it's the kind of thing that neuroscientists starting to.
10:06 Speaker 1 But it it doesn't feel like um current uh AI, uh certainly the stuff that's in in uh implemented commercially or even that's published at a research level is really bridging into this area that we're talking about, these rehearsal mechanisms for the other. I think it's
10:20 Speaker 0 actually one of the one of the uh potentially hot topics to incorporate into machine learning in the in the not too distant future. And people so re-in-so one of the the fundamental techniques in in deep lines work is reinforcement learning. Um developmental psychology. It has its origins really in in uh uh in in things like classical conditions, right?
10:41 Speaker 3 Pavlavian classes, rebels, signals.
10:44 Speaker 0 So within the field of reinforcement learning, there's a whole little subfield called model-based reinforcement learning, which is all about trying to do it by building a model, which you can then potentially use in this rehearsal sort of way. But but uh although Rich Sutton, who is the who's the sort of father of reinforcement learning, in his uh in his book way back in 1997, he proposes architectures in which these things are blended together very nicely. But I don't think anybody's really built that in a very satisfactory way quite yet.
11:13 Speaker 3 So just to help us come along with this, concretely, where are we right now in this evolution? And there's schools of thoughts that can disagree with this, but just simplify things. Machine learning, deep learning as a deeper evolution of machine learning, and then sort of like a full AI on a continuum. Is that sort of a fair way to start looking at and where do we kind of stand on that continuum?
11:31 Speaker 1 So I I have a I have a model which says that um you know AI and machine learning are really quite distinct things. Um AI is all about uh building systems that can in some way uh replicate human intelligence or th explore the spaces of possible minds, uh Mari's phrase, whereas machine learning is a very specific uh technique about uh building a system that can make predictions and learn from the data itself. So there are AI efforts that have no machine learning in them. I mean, psych, uh CYC is a great example. You know, you try to uh catalogue all the knowledge in the world. And I think it's you know it's it's the the the mindset of the market uh to combine the two because it it might give give something more attention.
12:14 Speaker 0 Yeah, I mean I I very much agree with that. I see machine learning as a kind of subfield of of artificial intelligence, and it's a subfield that's had tremendously uh a tremendous amount of success in recent years and is gonna go very, very far. But uh but uh ultimately the machine learning components have to be uh embedded in a larger architecture, as indeed they already are, you know, in some ways in things like deep minds working. We've had we've had this sort of
12:39 Speaker 2 thing before, though, in the history of AI, haven't we, where particular approaches have been flavor of the month and have done so expert systems for a while. I mean, there's the early, the early neural nets, which were much smaller neural nets, and now um bigger neural nets uh and deep learning based on that and sort of these um systems that are sort of self-guided learning seem to be flavor of the month. But um, given that you've been in the field so long, do you see this as you know, something that is likely to run its course and then we'll then we'll move on to something else. Is it the end
13:04 Speaker 1 of
13:04 Speaker 2 history?
13:05 Speaker 0 Um so I I think there might be something special this time. Um and uh and one of the indicators of that is the fact that there's so much commercial and industrial uh in uh interest in uh in in AI and in machine learning.
13:17 Speaker 2 Well that reflects the fact that it's been making a lot more progress than any of those previous attempts. Isn't there a problem there that with the expert-based systems um that you could ask them why they reach particular conclusions and with a self-driving car based on an expert system and it you know it decides and the classic you know, trolleology dilemma of does it you know run over the case? Oh, the
13:33 Speaker 3 school birds of the children.
13:35 Speaker 2 Exactly. Um, all of those sorts of things. I mean, um uh which I think are very interesting because even now you have sort of implicit ethical standards in automatic breaking systems. You know, is it small enough, it's it's that small, it's probably a dog. If it's this big, it's probably a child.
13:46 Speaker 1 So I think the I think that the trolley problem is is definitely worth looking at uh and and and talking about because uh we're we we as humans don't even agree on what the correct outcome should be.
13:57 Speaker 2 So if we're thinking about the trolley problem and um and some one of these scenarios comes to pass, with an expert-based system, extra system-based, you know, rule-based system, you could say, why did you do this? And the system will be able to say, well, but basically this rule fired and da-da-da. And with a with these more elaborate systems where it's more like gardening than engineering, the way we build them. Um it's much, much harder to get any of that kind of thing out of them. And isn't that going to make them harder to regulate it makes them much more capable, but isn't that going to be problematic potentially? So
14:24 Speaker 1 I I I think there are still there are still objectives that we we we understand, right? So the the way that you build a system that predicts using machine learning is is very utilitarian, right? You say there's some cost function you want to minimize or some objective function we want to target. And then you then you train it. And you don't really worry about the reasoning because the ends in a way justify the means.
14:45 Speaker 2 But the ends are going to vary.
14:45 Speaker 1 I
14:46 Speaker 2 think we're gonna see um uh you know get into a car and you can like adjust the ethics dial. Right. Because that that recent research that suggested that people are totally fine about cars making utilitarian decisions, as long as it's not them that you know is in there. But but in a
14:58 Speaker 1 way, we've lived we've we've lived in this world for a long time. We just haven't had to um we haven't had to ask the difficult questions. So any time you pick up the the phone in the UK, it's to your utilities provider in the US I understand it's to Comcast cur can uh customer service, you're forced through an algorithm, you're forced through uh a non-expert expert system where the human at the other end has no uh discretion and has to just ply their way through a script, and we know how frustrating it is to live in that.
15:24 Speaker 3 Very much so.
15:25 Speaker 1 Now, as we embed these uh these AI-based systems or or or machine learning-based systems into our everyday lives, we're gonna face exactly the same issues, which is my car didn't do what I wanted it to do, my toaster didn't do what I wanted it to do, and I have no way of changing that. Um and and so this this question about where is the utility function and what is a trade-off, has been designed into systems for 30, 50, 100 years or more. Um it's just becoming explicit now. It's becoming much more explicit because it's happening everywhere.
15:54 Speaker 0 Well, also there's the I think there's a big issue with these kinds of systems which may work just the way we want them to work statistically. So if you're if you're a if you're a company, then you know that it makes the right decision for for 99% of the people who are. If you are the one percent person who's phoned up and got a decision which is that which is not one that you like. You don't want to be told, well, it was
16:14 Speaker 2 right statistically. Yeah,
16:16 Speaker 0 just or just computer says no, you know, you you want to s to have reasons. And also if you're or more seriously, if you're a government government um uh if you're in government and you're making some big decision about uh about something or in a company and making a big decision about something, you don't want the computer to just say, just trust me, it's statistics, man. Yeah. Uh you know, you want to do you want to train of reasons.
16:37 Speaker 2 That brings up another aspect of this, which I find quite amusing, which is that there are quite a lot of sci-fi futures. So Ian Banks' future and the Star Trek future, where you basically have a post-capitalist society because you can have a perfect planned economy because an AI can plan the economy perfectly. But you know, there is this question of how how uh how plausible that is. But I wonder, you know, to the extent to which to which you think AIs will start to be used in in policy making and those sorts of decisions. Aaron Powell
17:02 Speaker 0 Well, I I suspect that they that they will be. And I think that's why, in fact, uh this this whole question that you're raising of of uh trying to make the decision-making process more transparent, even though it's based on statistics and so on. I think that's a very important research area. Um I
17:17 Speaker 1 think I'd separate out the the two areas of transparency. So one is the um is the black box nature, right? Can we look inside the box and see why it got to the conclusion it got to? The other side part that's important is to actually say this is the conclusion we were we were aiming for. And and the the within policymaking, what becomes interesting then is forcing policymakers to go off and say that extra million pounds we could have put into heart research, we didn't, because even though it cost four lives, uh and we put it into something else because we need we needed to.
17:48 Speaker 3 The kind of analysis we're talking about, this actual airline analysis, we're doing it every day already with insurance, which is just distributed risk. So we don't mind
17:54 Speaker 0 if humans do it, but we might mind if machines do. I think that's the big issue is uh would we be happy to to hand over those decision-making processes to machines, even if they made exactly the same decisions on exactly the same basis, you know, it will society accept that being done in this automated way.
18:09 Speaker 1 It's called Excel. It's not even so much that we trust whether it works. Let's assume it works. What we're doing is we're allowing uh using Excel, we're allowing ourselves to manipulate much larger data sets that we could have than we could have done just with pen and paper.
18:20 Speaker 3 We're sort of organizing the cells in our mind into cells and a spreadsheet. In
18:24 Speaker 1 cells spreadsheet, and you instead of having a hundred data samples, you just look at 16 million and what you know that the Excel can handle. So we've already started to explore the space of decisions using these tools, right, to extend human reach. Aaron Powell
18:37 Speaker 3 There's some of the examples that kind of approximate where we can go. Because the examples that come to mind, I think historically of Doug Inglebart's notion of augmented cognition and augmented intelligence. Um then I'm even thinking of current examples like Stephen Hawking, um, Helene Miale wrote a beautiful book about called Hawking Incorporated about how he's essentially a collective, because I I mean I don't agree with this turner phrase, but describing him almost as a brain in a vat, surrounded by this collective of a group of people who are anticipating his every need. And it's not just like Obama's crew who's helping him get elected and his support team. It's actually people who are who understand him so well that they know exactly what's the information.
19:17 Speaker 2 In your space of possible minds, um, we have a whole bunch of minds that we could be, you know, and some people are trying to figure out already, which are animals, and then you've got the sort of social animals, the the group minds there. And uh this kind of brings us to another ethical question um from the previous one we were talking about, which is uh yeah, the whole question of uh it the evidence that octopuses are very octopodies, we should say, are extremely intelligent, um, has you know has made some people change their mind about whether they want to eat octopus. Yeah, so I don't eat octopus. You don't have octopus or anymore. Really? I'm
19:46 Speaker 3 vegetarian, so I don't know. And as of today, I don't eat
19:49 Speaker 2 crab either. Anyway, so there's the there's the point of the extent to which a creature with a mind that we kind of uh we recognize is is uh cleverer than we thought, uh whether it's right for us to boss it around. But We're going to get this with AIs as well, aren't we? Because the usual scenario people worry about is we are enslaved by the AIs. But I'm much more interested in the opposite scenario, which is that if the AIs are smart enough to be useful, they will demand personhood and rights. At which point we will be enslaving them.
20:12 Speaker 1 Well, so let me give you a practical example of that.
20:26 Speaker 1 CC Amy, and then Amy will have a natural language conversation with you, and you think you're dealing with my assistant. One of the things that I found was I started to treat her very nicely because the way she's been designed as a product from a product manager perspective is very thoughtful. So you didn't say organized lunch slave. Exactly. I didn't do that. And I was, I was quite nice to her. And then I had a couple of people who are incredibly busy write very long emails to her saying, I could try this, I could try this. If it's not convenient, I could do this. And I thought this is just not right. There is that there is a uh misrepresentation on my part. So I then started to to um create a slightly apartheid system with Amy, which is it if you're very important, and Murray, you fell into that category, uh, you'll get an email directly from me. Uh and other people start. We'll get we'll get uh an Amy invite. And it does start to uh raise some of the issues that you that are very present day, right?
21:15 Speaker 1 They're very present day because right now we have these systems. And I think one of the ethical considerations is that we need to think about our own attention as individuals and as people, you know, and and and as we start to interface with systems that are trying to be a bit like the Turk, the chess playing device that present pretended to be human. We're giving attention to something that can't appreciate the fact that we're giving it attention. And so I'm now using a bit of computer code to impose a cost on you. Well, actually,
21:42 Speaker 2 it's like when I speak to an automated voice response system, you know, I speak in a much more precise way when it says read out your policy number. I know it's a I've got to help the algorithm right now.
21:50 Speaker 3 We're shaping our behaviors as well. So I think
21:53 Speaker 2 we already do that when we type questions into Google. We miss out the stop words and we know that we're just basically helping them. We
21:58 Speaker 3 don't ask questions anymore. We peck things out in keywords.
22:00 Speaker 0 Google will expect us to do that less and less as time goes by and expect the uh interactions to be in more and more in natural language. But but but can I come so I think that but between the two of you, you've you've raised the two kind of opposing sides of this deeply important ethical question about the relationship between consciousness and intelligence and consciousness, consciousness and artificial intelligence. Because on the one hand, there's the prospect of us failing to treat uh as conscious, something that really is that's that's that's very intelligence, and that raises an ethical issue for how we treat them. Then on the other side of the coin, there's the possibility of us inappropriately treating us conscious, something that is not conscious and is and is in is and is uh uh you know is perhaps not as as intelligent. So both of those things are possible. We can go wrong in both of those ways. And I and I think this is this is really um one of the big questions we have to think about here is and and the first I think the first really important point to be made is that there's a difference between consciousness and intelligence.
22:55 Speaker 0 And just because something is intelligent doesn't necessarily mean that it's conscious in the sense of capable of suffering, and just because something is capable of suffering and conscious doesn't necessarily mean that it's terribly bright. Um so we have to separate out those two things for a start before we kind of have this conversation.
23:12 Speaker 1 We seem to care about is is consciousness from an ethical perspective because we we care a lot about the uh 28-week pre preterm baby, uh, which is not very intelligent. Oh, we care about dogs and cats as well.
23:24 Speaker 3 So wait, where are we then in when people have expressed fears? Because one of the things I think has compelled me to invite all three of you into this in this discussion is none of you fall into one of these extremes of like completely, you know, cheerleading, like the future is dead, and you know, we're gonna be attacked and taken over, or the other extreme, which is sort of dismissive, like this will never happen ever. Where where are
23:47 Speaker 0 we? We're all in the sensible middle, aren't we? I guess that you're asking where are we, you know, historically speaking, in the internet. In this evolution in this moment. And um and I think the answer is we just don't know. But again, there's this is no there's a very, very important distinction to be made. This is the trouble with academics, or is it just want to make distinctions, you know?
24:03 Speaker 2 That's crazy important. They are kind of important.
24:07 Speaker 0 And this is a case where it's really important to distinguish between the short-term uh specialist AI, the kind of tools and techniques that are becoming very, very useful and very economically significant, and general intelligence, artificial general intelligence or human level AI. And we really don't know how to make that yet. And we don't know when we're gonna know how to make that.
24:29 Speaker 2 You don't sound like a believer in the kind of uh takeoff theory that you know the AI is able to develop a better AI in exponential, you know, less time, and so you get this sort of runaway. And I think that's a very unconvincing argument. It assumes all sorts of things about how things scale. So I think the
24:43 Speaker 1 takeoff argument, it has a sense of plausibility, it's a timing that's the issue. Uh that so I I I can't deny the possibility that we could build systems that could program better systems and that could start to program better systems. But the point is
24:57 Speaker 2 that a system that's twice as good, if it's say an order, you know, an order, it might it might scale um non-linearly. So it might be um 256 times harder to build a system that's twice as good. And so every incremental improvement is going to take longer, and it's going to take a lot longer. And um improvements in other areas like Moore's law and so on are gonna are not fast enough to allow each incremental generation of better intelligence to be um arrived sooner than the previous one. So there's a simple scaling argument that uh that this need not be linear. Well, but there's
25:26 Speaker 3 also a classic complexity break argument. I mean, there's so many different things.
25:29 Speaker 1 A lot there are a lot, and you know, as we start to peel apart the uh the the brain and our understanding of the neurological basis for how kind of cognition functions work, we learn more and more and we see more and more complexity uh as we as we dig into it. So, in a sense, it's a case of we don't know what we don't know, and we uh but we've been here before, uh before we had understood this idea of there being a magnetic field and needing to you know represent physical con uh quantities with tensors rather than with with scalars or vectors. Uh we didn't see magnetic fields, we didn't understand them, we didn't have mechanisms for manipulating them because we couldn't measure them. We couldn't, and therefore we couldn't affect them. And there would have been this whole set of physical uh crystals and and and and rocks that were useless because we didn't know that they had these magnetic properties, and we didn't know we could use them. Silicon dioxide being a great example, totally useless in the 17th century, quite useful now.
26:17 Speaker 1 Um and so at some point we might we might say that the reason we think this is looks very hard where it's not possible is because we're actually just not seeing these physical quantities. And when we touch on this idea of uh uh of consciousness, you know, there is this idea of integrated information theory, which is this theory that you know consciousness is actually an emergent property of the way in which systems integrate information and it's it's almost a physical property that we can measure.
26:41 Speaker 2 Yeah, or or we could be like, I suppose like Babbage saying, um, I can't imagine how you could ever build a general purpose um system using this architecture, because he can't imagine a non-mechanical architecture for computing. Right.
26:51 Speaker 3 Murray, where do you fall in this in this uh singularity debate? And you're now allowed to say that you make any distinctions. Yeah, well,
26:56 Speaker 0 without making any distinctions, but I'm I'm still gonna be boringly academic because I want to remain kind of neutral because I just don't I just think we just don't know. I I think these arguments in terms of recursive self-improvement, the idea that if you did build human-level AI, then it could self-improve. I think there's a case to be answered there. I think it's a very good argument. And certainly I do think that once if we do build human-level AI, then that human-level AI will be able to improve itself. But I kind of agree uh with uh Tom's argument that it's that it's uh uh not necessarily it doesn't necessarily entail. It's gonna be exponential. Yeah,
27:30 Speaker 3 right. So actually, to pause there for a moment. You started off very early on talking about some of the drivers for why you're excited about this time, why this time might be different. What are some of those more specifically? Like Moore's law, we've talked about it. Because that's obviously one of the scalars that sort of helps.
27:44 Speaker 0 Yeah, yeah. So basically there are what's driving the whole machine learning revolution, if we can call it that, uh, is uh, I mean, there are three things. And one is Moore's laws, so the availability of of uh a huge amount of computation, and in particular uh the development of uh of GPUs or the application of GPUs to this uh to this whole space has been terrifically important. So that's one. Uh two is big data or just the availability of of very, very large quantities of data, because we have found that algorithms that didn't really work terribly well on what seemed like a lot of data, you know, 10,000 examples actually work uh much better if you have 10 million examples. They work extremely well. So the un the unreasonable effectiveness of data is as some Google researchers called it. And um uh so that's two, and then the third one is some improvements in the algorithms.
28:33 Speaker 0 So there have been quite a number of little tweaks and improvements to uh ways of using backpropagation and the way and the way the kind of neural network architectures and so on.
28:43 Speaker 1 So I add three more to that list. So one is the um in practical software architectures, we're starting starting to see the rise of microservices. Uh what's nice about microservices is a very, very cleanly defined system. So you don't need generalized intelligence, you just need very specialized optimizations. And as our software moves from these hideous spaghetti to these uh API-driven microservice architectures, you can apply machine learning or AI-based optimizations to improve those single interfaces. Right, that's
29:10 Speaker 3 actually closely tied to the containerization of code at the server level. There's so many connect connected things like that. So it's
29:15 Speaker 1 much easier to insert a bit of intelligence into a process. And then the other two are um so there's this um this phrase, which I'm sure um Andrew Snohorovitz is familiar with, which is software is eating the world. Uh and um as uh software eats the world, there are many more places where AI can actually be relevant and useful. Uh so you can start to use AI in a food delivery service because it's now a software coordination platform, not chefs in a kitchen, um, and and therefore more places for it to play. And that this is a commercial argument. And and and so Murray's explained some of the technical reasons. The third commercial argument is accelerating returns. So as soon as you start within a particular industry category to use AI and get benefit from it, the increased profits you get, you reinvest into more AI, which means your competitors have to follow suit. So you can't now Build an Xbox video game without tons of AI.
30:03 Speaker 0 You
30:03 Speaker 1 can't build a user interface without using natural language processing and natural language understanding. So that forces uh the allocation of capital into these sectors, because that's the only way that you can compete.
30:15 Speaker 3 So given those six drivers, not three, who are the entities that are going to win in this game? Like is it startups? Is it the big companies? Is it government universities?
30:25 Speaker 0 Well, if you were to ask me to place a bet at the moment, I would place it on uh on the big corporations like Google and the other. Basically they have
30:33 Speaker 2 access to the data and everything else you can buy, but that you can't, right?
30:36 Speaker 0 Right. And also they have the resources to uh buy whoever they want. Um an interesting phenomenon we're seeing in academia uh these days is that it used to be the case uh that uh that the people who had you know were very interested in ideas and intellectual things, they they wouldn't necessarily be tempted away to the financial sector. But you'd still we'd still retain a good chunk of them in in in universities to do PhDs and going on. But now companies like Google and Facebook can hoover up quite a few of those people as well, because they can offer intellectual satisfaction as well as a decent salary.
31:10 Speaker 2 But also they are getting the um, you know, the Silicon Valley is the new Wall Street argument. They are getting the people who used to go into financial services. Which is a good thing. I remember the uh head of a Chinese sovereign wealth fund saying a few years ago, you know, you you Westerners are crazy. You educate your people in these fantastic universities, and then you take the best people and you send them into investment banks where they invent things that blow up your economy. Why don't you have to do something useful? We used to say that too. And the whole of you know, the Chinese Politburo is they're all engineers and they you know they they really value sort of engineering cultural and engineering skills, and they can't believe that we've sort of wasted it in this way. So I think it's fantastic that that you know now there's less money to be made at Wall Street than maybe than there is in Silicon Valley and people are getting West. I think that's only got to be a good thing.
31:48 Speaker 1 Coming back to who the winners, um, the winners might be. I mean, I think there is a strong argument to say that having the the data uh makes a lot of the difference. Yeah, no, I think that's the crucial distinction. You'd be I think you'd be hard pushed um to say let look at um voice interfaces, you know, between uh Apple, Microsoft, Google, um uh Baidu uh and nuance, uh that's quite a crowded field uh field already. So it does feel like there are a lot of AI startups who are uh going to run up against this problem of of both data and distribution. But that said, there are there are particular um niche applications where you can imagine uh uh a startup being able to compete because it's just not of interest to a large company now, and they may then be able to take a path to becoming uh you know independent and you know. Well
32:32 Speaker 2 look at say Boston Dynamics, because uh one of the ways you train animal uh machines to walk like animals is not to use a massive internet data set of how cats walk. Right. Um so that in that case, not having access to that data uh is not an impediment, and you can develop amazing things, and they have done. And then if that's a good example, been acquired by Google, right?
32:48 Speaker 0 Actually, the DeepMind are another example of the same thing because it because if you want to apply reinforcement learning to to games and make and that's enabled them to make some quite fundamental uh sort of progress, you don't need vast amounts of data either. Which is that
33:03 Speaker 1 Google's gonna
33:04 Speaker 0 buy all these companies. Well, yeah. But uh well, I I I ought to put in a little pitch for academia. Because the one thing that you do retain by staying in academia is is a great deal of freedom and the ability to disseminate your ideas to whoever you want. So you're not in in any kind of silo. And some of these companies are very generous in making things uh making stuff available. Right. So TensorFlow.
33:26 Speaker 1 TensorFlow is
33:27 Speaker 0 a great example of that that we've just seen Google uh Google release. Um but nevertheless, you know, all of these companies are ultimately driven by by uh by a profit motive and and are gonna hold things back. We've just seen, for example,
33:39 Speaker 2 Uber has snapped the entire robotics department for Carnegie Mellon. Presumably the uh motivation of the people there is that you know, finally the work that they've been doing on uh self-driving vehicles and so they actually get out into the world.
33:50 Speaker 1 And you can actually
33:51 Speaker 2 make a difference. And yeah, I'm sure they get much better pay. But I mean the main thing is that rather than doing all of this in a theoretical way, here is a company that's prepared to fund you to do what you want to do. In the real world in the next decade, and that must be amazingly
34:03 Speaker 0 attractive. It is incredibly attractive. And of course, many, many people, you know, um will will go into industry in that way. But there's also something attractive for a certain kind of mind in staying in academia, where also you can explore maybe some larger and deeper issues that you I mean, for example, uh, you know, Google aren't going to hire me to think about consciousness. No.
34:21 Speaker 1 Or they might. You never know. I mean, there's also that there's also this question about the kind of questions that you will look at uh as an academic. So the trolley problem being a good one, but there are all sorts of ethical questions that don't necessarily naturally play a part in your thinking when you think about your wall street, your courting walls walls.
34:38 Speaker 3 And corporate entities aren't set up to think about that. Like Patrick Lynn studies the ethics of robotics and AI and that entire work is funded by government contracts and just distributed through universities. So okay, so the elephant in the room, AI and jobs. What are our thoughts on that?
34:52 Speaker 1 Well, I think look at where we are uh today, which is that we're quite far away from a generalized intelligence. Um and uh you know, McKinsey just looked at this question about the automation of the workforce and they did something very interesting. They looked at every a worker's day and they broke it down into the dozens of tasks they did and figured out which ones could be automated. And their conclusion was we'll be able to automate quite a bit, but by no means the entirety of any given worker's job, which means the worker will have more time for those other bits, which were always the social, emotional, empathetic and judgment-driven as aspects of their job, whether you're a delivery person. Yeah,
35:25 Speaker 2 now I read that and I thought, hang on a minute though. What because what they're looking at is they're looking at the jobs of um basically well-paid information workers and saying, well, you can't automate their jobs away, but um, the bits you can automate are the bits that are currently uh many of them are bits that are currently done for them by other people. So the typing pool, you know, we've got rid of the typing pool because we all type for ourselves. Exactly. So, you know, uh this means that the support workers for those people are potentially put out of business by uh by AIs or
35:50 Speaker 1 they're moved up. Yes, or they have to find something else
35:51 Speaker 2 to do. But I think just
35:52 Speaker 1 because the architects are safe doesn't mean that the people who work for the architects are if you walk down a British high street, the main street uh today, and it you'll what one of the things you'll notice is a a plethora of uh massage parlors, um, nail salons and uh barbershops. Service businesses. Because these are the things that uh you you can't do through Amazon. Everything else you can do through Amazon or or expensive. So interior design, yoga,
36:15 Speaker 2 Zumba,
36:15 Speaker 1 whatever.
36:15 Speaker 2 That's that's the future of employment. Coffee shops. Aaron Ross Powell, Jr.
36:18 Speaker 3 Okay, so we've talked a lot about some of the abstract notions of this, and you know, this is not a concrete answer because we're talking about a fiction film. But how possible in reality is the X march in a scenario? And uh a warning to all our all our listeners that spoiler alerts are about to follow. So if you're really bitter about spoiler alerts, you should probably sign off now. The reality that the character, the main embodied AI, Ava, could essentially bite back to her enslavement. To me, the most fascinating part of this story, and we have no time to talk about it right now, but I do want to explore this at some point in the future is sort of the gendering of the AI, which I think is incredibly fascinating. Um real how real is that scenario? Aaron Ross Powell, Jr.
36:57 Speaker 0 Yeah, so the the So the whole film is predicated on the on on the idea. Well, it seems to be predicated on the idea that uh that Ava is very is not only a human level AI, but is a very human-like AI. So the humanoid aspect. Human like, of course, she she looks like a human, but I but but I mean human-like in her mind. And her objectives and her objectives and her motives. So, you know, so she if you if you were a person uh in those circumstances, you would want to get out, right? And in fact, very often uh science fiction films that portray AI, that's a fundamental premise that they use for how they work is that they assume that we're gonna assume that the AI is very much like us and has the same kinds of motives and drives uh for good or for ill. They can be good motives or bad motives, they could be evil or they could be good, you know.
37:43 Speaker 0 Um but it's not necessarily the case that AI will be like that. It all depends how we build it. And if you're just gonna build something that is very, very good at making decisions and solving problems and optimizing. They may just sit
37:56 Speaker 2 down and say, I just want to sit here and do math. We really have no idea what their
37:58 Speaker 1 motivations will be. Yeah, I mean, if the AI had been modeled on a 45-year-old dad, it would have been perfectly happy being locked up and it's shared at the bottom of the garden with uh an Xbox.
38:06 Speaker 3 And some magazines, right?
38:07 Speaker 0 Oh, but but then Busc, just moving on a little bit from that though, uh, it is worth pointing out some of the arguments that people like Nick Bostrom and so on have have advanced and that you shouldn't anthropomorphise these creations. You shouldn't think of them as too human-like.
38:20 Speaker 3 And the film, I I saw it three times, as I mentioned. And the third watching, I noticed that there's a scene where Nathan has a post like a photo of himself on his computer where he programs like he's at his computer all day, like hacking the code, which I think is so fascinating because there's almost this narcissistic notion which kind of ties your notion of the anthropomorphization of the AI.
38:40 Speaker 2 You use the term anthropomorphism because um it is I've noticed you use the word creatures to refer to AI. And I think that's I think that's really telling because they are going to be more like aliens or more like animals than they are like humans. I mean, the chances of them being just like humans
38:54 Speaker 0 are very small. We might try and uh you know architect their minds that they are very very human-like. But can I just come back to the Nick Bostrom kind of argument? Because he points out that although we shouldn't anthropomorphize the uh the AI, nevertheless, if we imagine this very, very powerful uh uh machine capable of solving problems and and and answering questions, that there are what uh uh people who who think about this talk refer to as convergent instrumental goals.
39:20 Speaker 3 You'll have to break that down for us really quickly, yeah. So
39:22 Speaker 0 anything that's that's really, really smart is gonna have a number of goals that it's good that anything is gonna share. And these are are gonna be things like um like self-preservation and gathering resources. If it's sufficiently powerful, then any goal that you can think of, if it's really really good at solving that goal, then it's gonna want to look at preserve itself first of all, because how can it how can it, you know, maximise the number of paper clips in the world is to use Nick Bostrom's argument if it doesn't preserve itself, or if it doesn't gather as many resources as it is as it can. So that's what that's their argument for why we have to be cautious about building something that is a very, very powerful. uh AI a very powerful optimizer. That's the basis of the thing. So I think I think the very important thing here is that is that the media tends to get the wrong end of the stick here and think think of this as some kind of evil terminator like thing.
40:12 Speaker 0 And so we might not we might think that those arguments are flawed, the arguments by Bostrom et al. Maybe we do maybe dun but I think there's a very very serious case to answer there. And in order to answer it, you have to read their arguments. You can't just kind of assume uh what you think their arguments are right the derivative.
40:29 Speaker 3 What's that's the problem with a lot of technology discussion in general is to always revisit these in a very derivative way versus viewing the original. But putting that exhortation aside how do people make sense of this? Like what how do they make sense of what is possible?
40:44 Speaker 0 How do so how do we think about the future really when it comes to artificial intelligence and I think the only way to do it is is uh is actually to kind of uh set out the whole tree of possibilities that we can imagine and and try to you know not sort of fixate on one particular uh way that things might go because we just don't know where we're going to go down that tree at the moment. So there's a whole tree of possibilities. Is A gonna AI going to be human like is it or or not? Is it going to be embodied or not? Is it going to be a whole collection of these kinds of things? Is it going to be a collective is it going to be conscious or or or not? Is it going to be self-improving uh in this exponential way or not? We you know I don't think we really know but we can lay out that huge range of possibilities and we can you know try to analyze each possibility and think, you know, what would steer us down in that direction and what would the implications be?
41:34 Speaker 3 It's a great way to approach it. Well that's another episode of the A6 and Z podcast. Thank you so much for joining everyone.
41:41 Speaker 0 Thank you. Thank you.
Previous Company Building in Crypto: A Conversation between Brian Armstrong and Chris Dixon Next What's Next for Education Startups Page updated August 25, 2026 · transcript and summary are machine-generated