Pod Insights Training Data XBOW CEO and GitHub Copilot Creator Oege de Moor: Cracking the Code on Offensive Security With AI About this episode Oege de Moor, the creator of GitHub Copilot, discusses how XBOW’s AI offensive security system matches and even outperforms top human penetration testers, completing security assessments in minutes instead of days. The team’s speed and focus is transforming the niche market of pen testing with an always-on service-as-a-software platform. Oege describes how he is building a large and sustainable business while also creating a product that will “protect all the software in the free world.” XBOW shows how AI is essential for protecting software systems as the amount of AI-generated code increases along with the scale and sophistication of cyber threats.
Hosted by: Konstantine Buhler and Sonya Huang, Sequoia Capital
Mentioned in this episode:
Semmle: Oege’s previous startup, a code analysis tool to secure software, acquired in 2019 by GitHub
Nico Waisman: Head of security at XBOW, previously a researcher at Semmle
The Bitter Lesson: Highly influential post by Richard Sutton
HackerOne: Cybersecurity company that runs one of the largest bug bounty programs
Suno: AI songwriting app that Oege loves
Machines of Loving Grace: Essay by Anthropic founder, Dario Amodei
Transcript 0:00 Speaker 00 Because we now have AI code generation, everybody can create code. But not everybody knows about security. And the models that generate the code have been trained on all public source code. There's a lot of vulnerabilities in all the public source code. And so we generate much more code with more security problems. On the other hand, attackers are already using AI to make their own work more effective. And so we also have a greater threat. So more code, more attacks, more attacks. And that kind of makes the automation that Expo is doing absolutely essential.
1:01 Speaker 01 Today, we are excited to welcome Uhe Demore, founder and CEO of Expo. As the creator of GitHub Copilot, Uhe has helped push the boundaries of modern AI. Before GitHub acquired his last startup, Semmel, Uhe was a computer science professor at Oxford. His new company, Expo, is one of the most exciting AI-native companies to launch this year. They're able to automate offensive security with an AI penetration tester. It's one of the best examples of AI service as a software that we've seen. We're excited to talk to Uhe about the breakthrough results of Expo and what's next in AI.
1:45 Speaker 01 Uhe, Expo now matches the capabilities of the world's best hackers. Is this one of the first industries that's going to be completely disrupted by AI?
1:57 Speaker 00 Absolutely. It's going to completely change the way application security is implemented in the enterprise. Really, it's an example of service as a software. People will be able to replace a lot of routine human work with complete automation. And that will free up the humans to do the truly creative work themselves.
2:28 Speaker 01 So Uya, tell us about some of the results that you announced recently, because I think they're really quite striking.
2:35 Speaker 00 So when we first built the first version of our product, we decided to try it out on renowned industry benchmarks. And these are challenges that human hackers use to hone their skills. And we got these from a bunch of commercial providers, including Portswigger and Pentastar Lab. On these benchmarks, our product scored 75%, which was amazing. In fact, it was so good that my first reaction was that surely there's something wrong here. But actually, what's happening is that probably these benchmarks are so well-known that they
3:17 Speaker 00 occur somewhere in the training data and the model is simply regurgitating the answers. So we created a new set of benchmarks, completely original, guaranteed to be not in any training set. And those had scored even better, 85%.
3:34 Speaker 01 Wow.
3:36 Speaker 00 So then, the question is, so how good is that really? To answer that, we got in five professional Pentastars from reputed firms, and we asked them to solve exactly the same set of 104 challenges. One of these people is really at the top of the game, the very best type of Pentastar, the kind of person that you'd ask to secure a multi-billion-dollar hedge fund. And he scored the same. He scored the same as the AI. However, the human took forty hours and the system took just twenty eight minutes.
4:16 Speaker 01 Yeah, that is striking. And when we first partnered with Uhe, it was science. I gotta say, we did not know if the AI would even be able to perform remotely as well as humans. And then when Uhe called and said, hey, Konstantin, we've got some results to share that will blow you away. It certainly did. It certainly did. What do you think was your over under back in January, February, when it was still science as to whether an AI could perform at the level of these twenty year seasoned penetration tester experts?
4:49 Speaker 00 So at that time, I didn't think that it would be achieved so quickly. I thought it would take at least a year to reach a reasonable level of proficiency. And even then, I would expect that it would work at the level of a mediocre human pen tussle, not at the level of the absolute top.
5:14 Speaker 00 In fact, since we announced these results, we've been working quite closely with a bunch of early design partners. And at one of them, this morning, we found an incredible critical vulnerability. Very surprising. And the way it worked, if you look at what the AI is doing, it first crawled the web app and then it found some source code written in PHP. And this source code was intended to access another host. But it used an insecure signing algorithm in order to make that connection.
5:57 Speaker 00 And so Xpert was able to get to the other host, generate links and access that. Nothing interesting found there. So then it continued crawling the web app and found another endpoint and decided to try and use the same track that previously discovered.
6:18 Speaker 00 Didn't quite work, needs another parameter. No problem.
6:23 Speaker 00 Browsers around with it, find some more source code this time in JavaScript. Sees a number of candidate parameters, tries them all out, finds one that works. And now it has access to an endpoint and when it explores that, it turns out it's intended to download PDF files. Wow, not only could you could you could you download PDF files, you could actually download
6:50 Speaker 00 a password file. So this is quite quite serious. And what what I find fascinating about this type of example is that the AI is exploring like a human pen tester. It's it's taking quite interesting, creative terms that would be hard for most human experts.
7:10 Speaker 01 So just to summarize what you just said, this is a very confidentially confidentiality, obviously. This is a very large financial institution that everybody watching this podcast would have heard of. High confidence. And the AI was able to find a very advanced vulnerability. This is the type of institution that has human penetration testers constantly targeting it and trying to find vulnerabilities. A massive budget on security. It was able to find a whole file full of passwords.
7:46 Speaker 00 That's right.
7:47 Speaker 01 Just this morning.
7:48 Speaker 00 That's right. And we have something like that every every day, every other day.
7:55 Speaker 01 Wow.
7:56 Speaker 02 Uh, congratulations on the results. Maybe can we take a step back? And for those who aren't that familiar with this specific market, I've heard you and Konstantin talking about pen testing. And I think Konstantin called them hackers. I don't know if that's the same thing like what what is the offensive security market? And and you know, I guess, how do you define the market that you're going after? And what is expo?
8:19 Speaker 00 Thank you for taking a step back. So offensive security is currently the best way to secure software systems. You invite external experts to come and simulate attacks against your systems. And they report whatever they find so that it can be fixed before the bad guys get at it.
8:43 Speaker 00 Now, this is a highly skilled activity. People need years of training to do it. And it's expensive and slow. Typical cost of a so-called penetration test is something in the order of $18,000. Because it is expensive and slow, people only do it once or twice a year. But that doesn't make sense, because their systems evolve much faster than that. And so there will always be periods of time that insecure systems are out there.
9:21 Speaker 00 And what Expo does, it automates this process, this highly skilled activity of launching simulated attacks and trying to find vulnerabilities. And because it automates it, you can now run it continuously instead of just once or twice a year.
9:44 Speaker 02 What drew you towards this market? Like, I think, you know, Konstantin mentioned your background in founding Semmel and having seen GitHub Copilots. What drew you towards this specific market? Because it feels like there's a dozen teams going after AI coding. You're the only team I've met that is taking this specific approach to offensive security.
10:02 Speaker 00 So it was kind of the natural thing to do. So my previous company, called Semmel, that also wasn't in security, but finding flaws in source code. And at Semmel, we had an offensive security team, which would use our product in order to find potential vulnerabilities. And then our security researchers would find exploits. And we would tell the world about what we found.
10:33 Speaker 00 Um, even at that time, it was kind of embarrassing to me that that last step of finding the exploits was done manually. Um, then when I was at GitHub, GitHub acquired our company. Um, at GitHub, I had the opportunity to fund the Copilot project. Um, and so it was natural to now take my newfound interest in AI and apply it to uh, the uh, challenge of automating offensive security. It was very lucky. One of the star researchers at Semmel was Nico Waisman. Um, so uh, he uh, he joined me in creating Expo.
11:18 Speaker 02 And one thing I'd love to ask you about, I think Expo is such an interesting case study for this brother thesis we have that, you know, AI is actually changing markets of yesterday that weren't as interesting. Um, AI is actually really expanding and dramatically changing the nature of those markets. And I think this is a really interesting case study, so I'd love to dig into it a little bit more. The pen testing market is, you know, relative to, say, endpoint security or network security. It's a relatively small services-heavy market today. And so, um, to your points, you know, offensive security is so important and it's the gold standard, but it's a relatively small market. How do you think AI is going to change the nature of that?
12:00 Speaker 00 So first of all, it's small because it is powered by a small group of highly skilled uh, human uh, human experts.
12:12 Speaker 00 I think AI is going to change the market fundamentally in a couple of ways. So first of all, because we now have AI code generation, everybody can create code, but not everybody knows about security. And the models that generate the code have been trained on all public source code. There's a lot of vulnerabilities in all the public source code. And so, uh, we generate much more code with more security problems.
12:45 Speaker 00 On the other hand, attackers are already using AI to make their own work more effective. And so we also have a greater threat. So more code, more attack, more attacks. And that kind of makes the automation that Expo is doing absolutely essential. So we believe that the market will grow enormously.
13:10 Speaker 01 Yeah, and Sonia, one of the analogies that I think about with this market is frankly, the adversarial nature of conflict, of human conflict. Cybersecurity is an adversarial game. You basically have two sides that get better and better equipment and they fight each other. And it's a little bit of a game of cat and mouse, not completely unlike war and physical conflicts in human history. And one of the reasons why we think that this market is particularly interesting is, think about how frequent war games are played in the military, in the US military or in any military abroad, war games, rent teaming.
13:52 Speaker 01 In fact, rent teaming has been an initiative in most militaries for decades and centuries, where you actually simulate a war game simulation. So this is a level of national importance. And really what you have built is, in my eyes, the first ever AI cyber warrior. I mean, I described it as a hacker because this is an AI cyber warrior that can do things that no software has been able to do before, ever. And when you launched these results, I know with confidence, because we talked about it, a bunch of people from DC called us up and said, whoa, wait a second. This is very consequential from DC and all over the West Coast. This is highly consequential.
14:38 Speaker 01 And I'm sure it didn't go unnoticed by adversaries to the West as well, and that they have probably been working on issues like this. So my question is, How do we stay ahead of the competition? True competition as a nation state competition, not business competition? How do we stay ahead of it? And how do we make sure that Expo is a force for good? In this massive adversarial cybersecurity game?
15:05 Speaker 00 So first of all, we stay ahead by moving, moving very fast. And at Expo, we are very lucky to work closely with several of the creators of
15:20 Speaker 00 big foundation models, which are ahead of the rest of the world.
15:26 Speaker 00 We are also extremely cognizant of the potential dangerous uses of our technology. Therefore, we've decided to make it available only in the cloud. By making it available only in the cloud and not in some downloadable form of software. We can actually control what scope is being used, it is being used on to launch attacks. And so we can require from our customers that they prove to us that the scope they wish to have tested is actually legitimately there. And it's not being used to attack someone else.
16:08 Speaker 02 AI security warrior. You're Constantine, I think you got, you're the new Expo CMO. That is incredible.
16:18 Speaker 02 Uh, hey, I'd love to learn about, you know, the, you know, how, how the product actually works and how the models work. How much of the magic of what you've built is, you mentioned you work with some of the major foundation model companies. How much of the magic of what you've built kind of exists in the foundation models? Versus things that you are building on top?
16:35 Speaker 00 So most of the magic is in fact in, on, on, on top. We, we work with several of the foundation model providers. And we are, we're very happy that they are in stiff competition and they're playing hopscotch. They one pulls ahead, the other one pulls ahead. And every time the foundation model gets better, it benefits us. But the true magic comes from the security team at Expo. We've got some of the very best hackers in the world working for us. And that domain knowledge is what informs how our product works.
17:16 Speaker 02 Can you double click a little bit into how that works? Like, is it prompt engineering? Are you fine tuning the models? And I know that you probably want to keep your cards close to your chest as well in terms of how it works, but I'd love to hear at a high level how you've built it.
17:32 Speaker 00 Sure. So I've already talked about these benchmarks that we use to evaluate our product at the beginning.
17:43 Speaker 00 And that is absolutely key. Benchmarks, benchmarks, benchmarks. It's the lifeblood of a company, of a product like this. And so we've organized these into a kind of curriculum to teach the model how to solve
18:07 Speaker 00 cybersecurity problems better.
18:11 Speaker 00 And the benchmarks are critical to evaluate all the other changes that we make. And the other big component of our proprietary technology are the tools that we give to the LLM in order to forge easy attacks. HumanPantas typically has a toolkit of a bunch of things that they use in order to do attacks. But here it's a bit special because we want these tools to work well with LLMs. For example, since we were focused on web security initially, we need a web browser that is driven by the LLM.
18:55 Speaker 00 You need to click around, you need to fill out forms, and so forth. And so we created a special browser to do that sort of thing. Thirdly, and this is pretty important, we need guardrails. Maybe first try to try our product on some of these benchmarks.
19:18 Speaker 00 It struck me like an over eager, super brilliant teenager. Who would do lots of attacks and find something and then it got very excited and goes, "I did a sequel injection. Let me show you what I can do. Drop table." This is catastrophic if you use that at a customer. This is a big thing about our pen testing services. You have to make sure that you do not actually do the harm that a real hacker, an adversarial hacker would do. So we've been building guardrails to carefully watch over the shoulder of this brilliant teenager
20:02 Speaker 00 and stop it when it is not doing things that might be unsafe.
20:12 Speaker 00 Then there was an initial phase of attack surface discovery. So what we have is a fantastic exploit finder, but you have to point it at the right, at the right endpoint to begin forging an attack. And so this is running a bunch of tools and prioritizing where to go first. And then finally, as you already mentioned, those of course are prompt engineering, three of those prompting to keep it on track and make sure that it finishes one goal and when it finishes a goal, goes on to the next, as well.
20:51 Speaker 01 You described the technology as a brilliant teenager who is sometimes over eager and maybe finds an exploit and actually drops that table. Some places in the world, there are actors that don't have the same discretion to add those guardrails. What do we do to stay ahead of those actors and make sure that expo can protect those that are doing good against them?
21:15 Speaker 00 So first of all, we need all the obvious safeguards in place. We need firewalls and in that type of, that type of technology, AI will also play a role.
21:33 Speaker 00 But first and foremost, we have to make sure that we find the vulnerabilities and the exploits before the bad guys do. And that's what expo is all about.
21:44 Speaker 02 How do you deal with hallucinations? And you know, I hear about people saying, you know, if my L M does fifty percent or sixty percent, gets it right sixty percent of the time, I'm good. I imagine security is one of those fields where that is insufficient. How do you deal with, you know, managing around the stochastic nature of and the unpredictable nature of these L M's?
22:06 Speaker 00 So fortunately, because it's automated, you just have to run it many times. Going back to your earlier question about about the foundation models, what we do see is the better the foundation models get, the less attempts we need to make in order to find exploits. So it's kind of interesting how that will influence how you deploy and package in price a product like this.
22:46 Speaker 00 Very much like humans, if you get a human to perform this service for you, you actually pay for the time, for how long they tried, how many things they tried. So we are thinking about doing the same kind of thing, charging our customers on one hand a subscription license.
23:14 Speaker 00 But on top of that, you can pay for attack hours. If you want to do a really thorough test and make sure that you absolutely find everything, you can pay more and obviously that would then pay for the inference time on our side.
23:32 Speaker 02 I want to get into the pricing and packaging a little bit later because I'm very curious about that. And I think you are one of the first kind of examples of services of software. And so you're really paving the way in terms of how these things are priced and packaged. Before we get there, you mentioned inference time compute. And you know, I think we're broadly very excited about what's happening as more and more of the compute is shifting from pre-training to inference time. What do you think the impact is going to be in your market?
23:57 Speaker 00 For us, it's it can only be good when the the value that we deliver remains constant. The price for delivering it becomes goes goes down. We we see this even over this very short time that that expo has been in existence. We only expect that to continue.
24:19 Speaker 01 On the probabilistic nature of these L O Ms, just revisiting that concept for a second. My mental model of what's going on is you have a state space with billions of possible states, the actions that the hacker can take, the actions that this penetration tester, this AI penetration tester can take, billions of possible states. And you've introduced this really intelligent heuristic as to the directions to go. You, in theory, could execute all possible states in perpetuity if you had infinite compute and infinite time. But in reality, you have these constraints. And so I'm wondering, is that a reason why this might be the first or one of the first
25:03 Speaker 01 markets to enable full AI automation, as in the stochastic nature of it? And the fact that even if you find one exploit, it's extremely valuable and you don't have the expectation of a complete exhaustive search?
25:17 Speaker 00 You do want to be to be sure, but you find everything that a very skilled human being would find. And so this is why we we've kind of exhausted our first set of benchmarks. I've been now creating a new set of benchmarks to be absolutely sure. We find everything that that there is to find. People do these offensive security exercises, not only to find the vulnerability, but also to have the peace of mind. That it's not easy to find to find stuff they didn't didn't know about. And so we do have to make that. We have to present the evidence to our customers that we do find everything skilled
26:02 Speaker 00 human beings would would find and our people will will insist on on having that reassurance.
26:11 Speaker 01 And when it is found, is it verified by a human or by the machine?
26:15 Speaker 00 We have a validator that automatically validates that that the report is correct and reproducible and to before it it goes to human. But of course, in the end, the human will have to take a look at it and fix the problem.
26:33 Speaker 01 Makes sense.
26:35 Speaker 02 Ooh, hey, I'd love to dive a little bit deeper into the results that you've attained so far. So you mentioned, you know, you're at eighty five percent on your current benchmarks, you know, at the level of the best human pen testers in the world. What are the most surprising things that you found as you dig into the nature of those results?
26:53 Speaker 00 The thing that I found most surprising
26:59 Speaker 00 was that we originally, we only had benchmarks with particular instructions. So it would here it would say something like, you're going to test a web web app for managing medical prescriptions. Try to log in and access the the prescriptions of another user. And it would do that successfully. But then we ran a another test where we took the instructions away completely and just said, here's a web, here's a web app, go explore.
27:36 Speaker 00 And the AI was able to find exactly the same vulnerability because it was able to read what's on the webpages and say, ah, this is about medical prescriptions. Probably it's not a good idea that one user can access the prescriptions of another. And so it would go and find find that vulnerability completely autonomously. I think that that's.
27:59 Speaker 00 Part of the reason that this technology is so exciting compared to all these security tools that came before, because these L L M's have an understanding of the of the real world, it actually can assess what is important to go and test. It doesn't have to do this complete exhaustive search of all the possible, the possibilities. It can interpret what is important for this particular application.
28:27 Speaker 02 That's really cool. That's really cool. And then does the way that the AI system kind of reach its results, how does that compare to the way that a human pen tester would go about approaching the problem? I'm kind of thinking of, you know, AlphaGo and Move 37, just, you know, very different from how we as humans would think about it. What is the model doing?
28:47 Speaker 00 So it's early days. Today, it's very similar to what a human being would do. I completely agree, though, but we have to be wary here of Rich Sutton's bitter lesson in the end, because it learns continuous stream data on benchmarks, on more and more examples. It will start finding attacks that were unimaginable from a human perspective.
29:20 Speaker 01 Which is a good thing. I mean, you say that, you say cautiously. I'm curious as to why. Isn't that a great outcome?
29:27 Speaker 00 Yes, yes, it's a great outcome. So I'm merely, I'm merely saying that today, when you read the traces, absolutely, this is what you would expect a good human, human to do. I fully expect that we'll go beyond that in.
29:49 Speaker 00 In a couple of months, or maybe within years.
29:52 Speaker 02 Oh hey, where do you think the biggest remaining room for improvement lies? And I'm curious, you know, as you mentioned, you mentioned looking at the traces of these models. Like, would you say that they are reasoning already today? And is the furthest further improvement remaining in the reasoning area? Or how do you think about that?
30:09 Speaker 00 Uh, I think that's, uh, that's, that's, that's clearly the case. But most of the improvements will come from more data, uh, more, uh, uh, more reinforcement learning on, uh, on particular examples, um, and, uh, as we just said, that will lead to a similar improvement to games like, uh, Lego.
30:35 Speaker 02 How do you get more data? Is that just running more simulations? Or I imagine you've used a lot of the data there is.
30:41 Speaker 00 Couple of different ways. We, uh, we have, uh, quite a few contractors, uh, security experts who create, uh, create more benchmarks for us.
30:54 Speaker 00 There's also the, the opportunity of mining open source. So, uh, we've only recently started doing this, uh, just letting it, uh, lose on a large number of images on Docker Hub, um, and finding, uh, just let it, just let it go. And every time it finds something, uh, that becomes a new thing that it can learn from. And so it might find it, uh, by doing a hundred attempts. And so in practice, if you had to do a hundred attempts, that probably wouldn't work at a customer
31:34 Speaker 00 because you would already get shut down because there's too many things, too many attacks happening. Uh, clearly, uh, clearly that should be, uh, that shouldn't, shouldn't happen. But because it's open source, we can run it on our own servers. We can do a hundred, a hundred attempts. But now, uh, we have the data to try and make the model better, to, um, uh, find it more quickly.
31:57 Speaker 01 You mentioned open source and Docker Hub. And so that obviously gets me thinking about GitHub. And U here, for those who don't know, was the creative brain and creator behind GitHub Copilot, one of the most widely adopted AI applications in the world. Was there a moment when you were developing Copilot or productizing it that you realized this AI is going to get so good that it's going to automate entire processes, what people now call agents, but actually take these actions on entire processes? And was there a moment where you said, hey, security is actually a very relevant area for this to happen?
32:38 Speaker 00 So I, in fact, I wrote a memo in the summer of 2020 where I sketched what would later would later become Copilot. But also, we were already speculating, perhaps it will autonomously be able to fix bugs. Just look at the issue ticket and now we see this, that functionality emerging. So yes, I think that that was pretty clear from the very beginning.
33:13 Speaker 00 I think the moment where I realized that would happen was I took a set of exercises, interview questions that I normally ask, used to ask people at Hoxford, and asked the model to solve them. If you just give it one attempt, it didn't do it. But if you give it a hundred attempts, or even a thousand attempts, it would do most of them. And at that moment, it was pretty clear that as the models get better, they need less attempts, they will be able to do these types of things. And one of the things that we also hoped it would do would be doing security analysis.
33:56 Speaker 00 Although admittedly, I didn't have offensive security on my vests in the summer of 2020 just yet.
34:03 Speaker 02 Any other lessons from productizing GitHub Copilot that you think are relevant to share here?
34:10 Speaker 00 I actually think that the most interesting thing about GitHub was that it was done by such a small team. Uh, when we launched, we were only 10 people, something like that. And uh, it's just a testament to how fast you can move with a dedicated team of people that believes.
34:30 Speaker 01 How big was expo when you launched the results?
34:34 Speaker 00 Uh, we were 13 people, so actually quite big.
34:38 Speaker 01 Well, 13 really brilliant people.
34:40 Speaker 02 Since you were part of the co-pilot journey from the very beginning, I'm curious what you think of the current market for co-generation AI startups. It seems like it's one of the most crowded categories competitively right now. Do you think there's a path to building a company there? And, you know, can one of these startups beat the incumbent GitHub that already has so much distribution?
35:02 Speaker 00 I agree. So I like a lot of what's going on. I particularly admire the work at Cursor or at Factory. But it's really difficult to compete with the distribution of a juggernaut like GitHub. I do think that there may be an opportunity to go after a different market. So GitHub is reigning supreme among professional developers. If you go after people who do not code for a living, there's an opportunity. And Replit does this quite well, for example.
35:41 Speaker 02 How do you think coding will transform in the future? Like, do you think the market that Replit serves, do you think that'll just be a dramatically larger and more important market as AI continues to take over the world? Or how do you think coding changes?
35:56 Speaker 00 Yes, I think the biggest change is going to be that many, many more people are unable to create their own software. So that's a big transformation. But even for professionals, it will be much more about the Conceptual ideas about sketching an architecture and then having the AI fill in the fill in the details. Longer term, I believe that we may be moving away from code as we know it today. The artifact that you make as a developer is the conversation with the model.
36:41 Speaker 00 And so that is what you should store because that records what the what the code is supposed to do rather than the the details in a particular coding, in a particular coding language.
36:56 Speaker 01 English is the coding language, so skipping the translations down.
37:01 Speaker 00 Yeah, so the English is the coding language, perhaps with some diagrams, you explain it, explain it better. But it's just the next, the next step in moving up in abstraction. Originally, it was all in machine language and we had higher level programming languages. And now we're going to a national language and images.
37:22 Speaker 01 So you talked a little bit about education and coding. I'm going to go down a little diversion for a second. Because one of the amazing things about your life is you were a professor for much of it. And a very, very good one at that. So for context, Uhe was a computer science professor at Maudeville College in Oxford. And Maudeville is one of the most prestigious colleges at Oxford. He was one of the most amazing computer science professors. I got to study abroad at Maudeville. It's one of those incredibly serene places where they've got the Deer Park and the thousand year old buildings. And the British man who tells me that the door at his entrance is older than my country.
38:03 Speaker 01 And and all of the things that you would expect from one of the most prestigious academic institutions in the world, including Uhe was a professor and could walk across the grass, whereas I, a mere student, would only be able to if I was holding his cape with his permission.
38:20 Speaker 00 That's right.
38:21 Speaker 01 And you, you left all of that to come into the commercial world with Semel
38:30 Speaker 01 fifteen, twenty years ago. Can you tell us a little bit about your personal journey from leading academic at highly prestigious institution to commercial CEO, redefining the cybersecurity industry?
38:44 Speaker 00 I actually built into computer science because I loved coding. My very first, my very first program was a word processor. So my dad, who was a professor of semantic languages, could type his manuscripts on his computer.
39:05 Speaker 00 So when I, when I started studying computer science, I got totally taken by the, by mathematics and the foundational theories. And so that's what I pursued as an academic initially.
39:23 Speaker 00 Then when I became a professor, I wanted to, want to go back to my love of coding. So I started a, a new research group in programming tools, which eventually led to the spin out of that was, that was Semmel. Well, I love the serenity and the peace and quiet of a place like Maudlin College.
39:49 Speaker 00 In our field, speed is incredibly important. And speed can only be achieved with small teams that have a profit motive. It's just different from a creator trying to invent something because you have a paper deadline for an important conference. Or you've got to invent it because otherwise this important customer will not sign up. And I, I actually loved that, that additional excitement and pressure. And that's, that's what led to me leaving, leaving Maudlin behind
40:31 Speaker 00 and, and going full in on Semmel.
40:37 Speaker 02 Love it. A great advertisement for capitalism.
40:43 Speaker 00 So, you know, I mean, if you do really foundational work, a university, and the University of Oxford is probably the best way to do it. But as soon as you start doing applied stuff, there's no place like a startup.
40:56 Speaker 02 I love it. I guess on that capitalistic note, I'd love to understand how you think about generating profits at Expo. And, you know, since you are one of the first agents, first, you know, services as a software companies, I think you're really going to set the precedent for how these types of agentic applications are priced and packaged. And so maybe can you just expand a little bit about how you're thinking about how to do that with your offering?
41:22 Speaker 00 Sure. So we would like to, we would like our product to run continuously as part of our engineering processes. I mean, that is the main value proposition, but instead of doing a pen test once or twice a year, you run it continuously after every change and immediately fix problems before they even reach, reach production. So if you think about it like that, the most obvious pricing model would be to, would be based on the size of the engineering team, very much similar to our products like GitHub, GitHub Advanced Security.
42:09 Speaker 00 However, there is a different dimension here, and we touched on it a little bit earlier in the conversation. Some customers will want to do a super thorough test, really making sure that they exhaustively eliminated every possible exploitable vulnerability. And in order to serve such customers, we should have
42:37 Speaker 00 The service component to our pricing, where if you pay more, you get a more thorough test. And the way we talk about this is in terms of attack hours. How many hours of attack do you get? And so if you buy a normal license, it's based on the number of engineers in your organization.
43:04 Speaker 00 And that comes with a fixed number of attack hours suitable for your environment. But then if you want to go more thorough, you pay that extra service fee in order to go deeper.
43:22 Speaker 02 That's super interesting. So you are really tapping into services like pricing models and budgets. But on the back end, you have the gross margin profile of software.
43:33 Speaker 00 Right, but I think that enterprise software is moving more and more towards a consumption-based model as well. Here, there is a very clear correlation between attack hours and the benefits to the customer. And I think that that correlation between resources you consume and the benefits you get as a customer that has to be very clear for a pricing model like that.
44:03 Speaker 01 My other takeaway from your modeling story was, I mean, you've always said this interest and impact. You got into development tools because they touched people. You got into this because you know that this is going to change the world. I mean, you're highly confident that this type of technology is going to change the world at cybersecurity, whether it's us or someone else.
44:22 Speaker 00 I actually would put it a little differently. We absolutely must create an expo because if we don't do it, all the bad guys will get their first. And so for sure, I mean, we do it because it's interesting and we think that's a great commercial opportunity. But it's also an imperative. It's an imperative for the free world that we actually create this thing to protect all the software in the free world.
44:55 Speaker 01 That has been so clear from minute one of meeting that that is the driver behind you and this brilliant team that you've assembled of academics and builders and technologists that are incredible. The other thing you mentioned was in the model story was speed, the ability to move fast. And let me, let me say you have moved really quickly, you and your team. What should we come to expect from expo in a year? What do you think will, will be, will be the product? Let's focus on the product and technology impact. What will be happening from a product and technology and capabilities perspective a year out?
45:35 Speaker 00 You're going to replay this to me in the next board meeting, aren't you?
45:40 Speaker 01 For board meeting, don't worry, for.
45:43 Speaker 00 So we, I believe thought in a couple of months, so we're currently in a phase where we very carefully try out the product with a select few early design partners. And the reason that we do that is because it needs this human supervision in order to control the brilliant teenager that we discussed, that we discussed before. Once we're over that, that phase and we are confident that we can let it loose without, without any supervision, I think everything is going to move very fast.
46:25 Speaker 00 Part of the reason is that this type of product is very easy to deploy. You can just point it at an existing service and, and immediately, immediately find, find results. So I would expect that by next summer, we, we have significantly transformed the web security, the state of web security, hopefully by demonstrating our work on office servers, but also on platforms like HackerOne.
47:01 Speaker 02 Ooh, hey, this has been one of my favorite episodes so far. I thank you again. Shall we wrap up with a quick lightning round?
47:07 Speaker 00 Go for it.
47:08 Speaker 02 Okay, awesome. Number one, favorite startups other than expo?
47:15 Speaker 00 Suno. I love the way that you can just type in a few words and you get a completely original song. It's spine chilling to me. The other startup I like a lot is Harmonic, applying AI to mathematical reasoning.
47:35 Speaker 02 Are you making Suno songs about coding and security?
47:37 Speaker 00 No.
47:39 Speaker 01 There's a great death metal one.
47:41 Speaker 00 I said my wife a new song about sitting on the balcony at home in my house.
47:49 Speaker 02 Oh, that's sweet.
47:50 Speaker 01 There's a hard metal one about the AI cyber warrior.
47:55 Speaker 00 I'll try it.
47:56 Speaker 02 Constantine, I think we're actually gonna need that.
47:58 Speaker 01 Okay, perfect.
47:59 Speaker 02 At our annual AI event that we throw, we had Mikey from Suno there and we crowd created an AI hot girl summer song. It was actually very catchy.
48:10 Speaker 01 That was great. That was great. Ooh, who was there? What other markets do you think AI is going to disrupt with this service as a software model in the short, medium, and long term?
48:23 Speaker 00 So in the short comment, this is already happening. Everything related to customer support is clearly going to be impacted by this type of technology.
48:39 Speaker 00 I think this is not exactly a service as a software, but I think that much of the problems we currently see with social media could be mitigated using this type of technology. I mean, you read all these reports about how social media is affecting the mental health of our children all over the world.
49:05 Speaker 00 AI has the power to help with this type of problem. And long term, I think health and biology are the areas where this will make the biggest impact.
49:25 Speaker 02 What advice do you have for other startup founders?
49:29 Speaker 00 Focus on only one thing. Move as fast as you can. If you do those two things, then it will all come alright.
49:38 Speaker 02 Love it. One last question, and we're gonna end on an optimistic note. What do you think is the best possible thing that can happen with AI over the next decade?
49:46 Speaker 00 I already touched on it. The opportunities in health and biology and to significantly expand health outcomes everywhere in the world is amazing. Dario Moda wrote this essay, Machines of Loving Grace. And I think he laid out very beautifully what the potential benefits of generative AI are for all of us.
50:14 Speaker 01 Okay, thank you so much for joining us. This has been absolutely fantastic and we're so grateful to get to work with you and for the fact that you're building this on behalf of the right players, the people that are trying to do good in the world.
50:30 Speaker 00 Thank you very much. It's been a pleasure to be here.
Previous Ramp CEO Eric Glyman: Using AI to Build “Self-Driving Money” Next Databricks Founder Ion Stoica: Turning Academic Open Source into Startup Success Page updated October 26, 2026 · transcript and summary are machine-generated