Pod Insights a16z Podcast a16z Podcast | VR, AR, and Beyond: The New Medium of Human Experience About this episode The building blocks for VR and AR are finally here -- but the content is just beginning. So everything you'll actually experience and consume in these new mediums over the next few years is being built right now. Formats aren't yet defined or locked down, and the field is bubbling up with experiments in forms, formats and genres, from narrative to games to live events. As we begin to have real time rendered characters and AI-driven environments that you can interact with, the storytelling structure will also need to completely change.
Are these mediums inherently social -- or just the opposite? What will self expression look like? What experiences are being built? Because fundamentally, that is the "celluloid" we are now working with -- human experience, says Within cofounder and filmmaker Chris Milk.
Bigscreen founder and CEO Darshan Shankar, Lytro CEO Jason Rosenthal, and Milk join a16z's Kyle Russell in conversation about the challenges, potential, and emotional power of these new technologies -- on this episode of the a16z Podcast, recorded at the inaugural a16z summit.
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:00 Speaker 00 The content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. For more details, please see A16Z.com/disclosures.
0:19 Sonal? Hi, everyone. Welcome to the A16Z Podcast. I am Sonal. Today, we're kicking off our new series on new medium storytelling with a conversation about VR, AR, that's virtual reality and augmented reality, and beyond. And what does it mean when the celluloid that we're now working with is actually the human experience? Joining us to have this conversation, we have Within co-founder and filmmaker Chris Milk, BigScreen founder and CEO Darshan Shankar, and we have Lightro CEO Jason Rosenthal in conversation with A16Z's Kyle Russell in a conversation that was recorded at our inaugural A16Z Summit. In this conversation, they cover everything from the challenges and potential of these new technologies to the emotional power a new medium affords.
1:03 Kyle Russell? Why are we excited about putting big goggle-style computers on our faces? To understand A16Z's perspective, I think it's valuable to define terms. I don't want to assume anyone in the room has played with every gadget like I have. So the big buckets to think about in terms of the emerging computing paradigms we're going to be talking about today are augmented reality, which lets you see the world differently, and virtual reality, which lets you see a different world. The former gives you superpowers. You're never going to go to an event like this and meet someone where you're pretty sure you've seen them before, but you don't remember their name. You're never going to have that awkward exchange because their LinkedIn profile is going to be floating above their head, once you've got glasses-sized AR headsets.
1:48 Kyle Russell? Virtual reality, on the other hand, gives you true telepresence. You're going to be able to work and play from what feels like the same environment, but from any distance. It also lets you put yourself into, more directly so than any other previous medium, into the shoes of someone else. You get to truly see the world from someone else's perspective. And so I'm joined today by three amazing entrepreneurs who are helping to make those things reality. To start, though, I kind of want to do a little bit of level-setting. Darshan, you're a technical founder. You spent a lot of last year building BigScreen yourself. Why did now feel like the appropriate time to jump into VR as an ecosystem?
2:27 Speaker 03 Well, in 2015, it actually wasn't certain that now was the right time. That was a big question. The question was how good were consumer headsets going to be? How many people would buy them? Would there be a real market for it? And now is the right time because finally, after years of development, both on the hardware and software side, there's actually high-quality consumer VR headsets that are out there. Headsets that don't actually make people sick. At this point, it's the software that's making people sick, not the actual hardware itself. So a lot of those low-level blocks are just good enough that we can start building great products, great companies, that just weren't really possible before, just a year or two ago.
3:06 Kyle Russell? Right. And so, Chris, Within is already one of the main distribution platforms that our colleagues down in Hollywood are using to reach people with the VR content that they're producing. How is Hollywood thinking about the space? How are they thinking about the investments that they're making, their expectations around what success looks like, given where the ecosystem is today? How are they thinking about it?
3:29 Speaker 04 Well, down in Hollywood, we're starting to see a shift from VR pieces that are exclusively funded by marketing budgets, because that's all there was to fund them with in the last year, to more of the forward-thinking studios actually funding content for the purpose of monetizing it. You see, just, I think, last week, really the first example of this, which is Fox's The Martian VR Experience, just sort of cinematic VR, sort of a game, and it's priced at, I think, $19.95, right?
4:13 Speaker 04 And we'll see how it does. But these are businesses that are used to making content that costs a lot of money and then selling it and making their money back. That's not entirely possible with the current addressable market, I think, in virtual realities, when you're spending millions of dollars on a piece of content. But the studios realize that... They need to start exploring to be in a position that they can make really refined, high-quality content when the addressable market is actually there.
4:47 Kyle Russell? To follow up a little bit, when I think about my media diet, it's heavily bifurcated between the blockbusters I go to every couple of months in theaters and binging via Netflix, HBO Now, Hulu. You talked about a $20, something that's associated with a blockbuster movie, investment in that app in terms of my buying behavior.
5:08 Speaker 04 The thing is, you could have all the money in the world, you could not start Netflix of the art today because there's no back catalog to license. Netflix starts and they license 100-something years of cinema and television and they have a catalog.
5:26 Kyle Russell? Got it.
5:27 Speaker 04 And you pay for it to access that catalog. That catalog doesn't exist in virtual reality. Everything that you're watching or experiencing is being built currently.
5:35 Kyle Russell? Speaking of which, Jason, at Lightro, you're building the next generation of cameras that are going to be used to produce this high-end cinematic storytelling-based content. How's the experience of VR as a cinematic medium going to evolve over the next few years?
5:51 Speaker 05 Well, I think nobody knows. I think that's one of the most exciting things about it. I think that if you look at, one of the things that's super interesting about VR is it's the first medium of media where the formats aren't defined and locked in from the very beginning. There's lots and lots of experimentation because it's the first way that people can interact natively with the world around them that they know, but in a completely virtual environment.
6:18 Kyle Russell? But if you had to commit to a most probable future path, how are you envisioning where things are going to go?
6:24 Speaker 05 Yes, I think it's going to look like a lot of other forms that we've seen. There's going to be narrative forms of VR, there's going to be inherently game-like experiences, there's going to be live events, and where we are now. And I think Chris and Darshan are doing amazing jobs of running different experiments in this. Nobody yet quite knows what's going to work, so everybody's trying to iterate to the first version of product market that you're going to experience.
6:53 Kyle Russell? And so, you know, in order to make VR headsets, you know, provide a killer app, we're excited for gaming, we're excited for Hollywood to produce content where the entertainment value is just so high compared to what you get from classic cinema that, you know, kind of draws people to the platforms.
7:08 Speaker 05 I would question the premise of that. So I think if you look at any new medium, right, be it the internet or mobile games or what have you, it's never been the companies that were dominant in the last form of entertainment or media that have then become the leaders in the next form. And I think for sure that's going to happen here. I think Hollywood has some interesting assets in that they've got, you know, known IP which people love. And so I think there will be great VR experiences created around Star Wars and Harry Potter and other things, but it's not going to be those companies that own that IP necessarily doing that.
7:43 Kyle Russell? Got it. And where I was going with that, though, was, you know, this is all around kind of industries producing content. However, you know, you look at the web, you look at mobile, kind of previous computing platforms, social media became dominant very quickly. You know, power of network effects, people like to communicate with their friends, they like to share things about their lives. Dershawn, how do you think self-expression is going to work in VR? Like, are we going to be just sharing in the same way that we share photos and videos captured from our phones today, you know, flat media? Are we just going to be doing that with 360 videos as our phones get that capability? Or are there kind of other avenues of self-expression that VR opens up?
8:25 Speaker 03 It's hard to say, probably all of the above, in a sense that 360 content, once it becomes extremely easy to film, extremely easy to capture for every consumer, the way we capture with our phones, any sort of photos, that becomes really easy to share. That becomes, but first, you have to make it really easy to capture, make it really easy to distribute. You have to solve some of those low-line problems first. But that whole range, like for big screen, it's a lot more about sharing your 2D content. So you might be working on something and you might be able to share it with a friend. It doesn't require 360 capture technology to catch up and go mainstream. But in terms of like user-generated content, allowing people to create kind of like what Minecraft has accomplished in the past decade, allowing people to create their own virtual worlds and
9:08 Speaker 03 explore it with their friends, we might be a little early for that. In terms of user-generated social media type functionality, we again, might be too early for that. It's hard to predict when it'll be ready for that. But if you look at the history of the internet, the web, and mobile, a lot of bundling was occurring in the early days where it wasn't completely user-generated. It wasn't GeoCities super early on. It was AOL. It was kind of bundled news before everybody could make their own website. It was like major news websites that you would visit first before you started making your own, before WordPress or something. So, hard to say. It might be a little early, but even today, you can create some pretty cool stuff and share it with your friends.
9:51 Kyle Russell? Got it. And so, one of the common concerns that I hear a lot when I'm trying to, you know, talk about how amazing VR is going to be as it takes off is, "Yeah, but isn't it really isolating?" You know, you're wearing something that's literally, in the case of VR, AR, maybe not so much, but with VR in particular, you're wearing something that's literally blocking your view of the rest of the world around you. And so, people have understandable concerns like, "Are we all going to be in our own kind of escapist fantasies, not interacting with each other?" From a storyteller's perspective, you know, cinema, we go to the movie theater together. We sit on the couch and watch Netflix or catch up
10:29 Kyle Russell? on Game of Thrones with our friends. Jason, how do you think about cinematic and social kind of coalescing in VR?
10:40 Speaker 05 I mean, to me, I think that the, I think the bull case is that VR has the potential to be the inherently most social medium kind of of any form of technology that we've seen so far.
10:49 Kyle Russell? Even with the goggles?
10:50 Speaker 05 Because I think, imagine, you know, if you had the power to, instead of just going to the movies with your friends, like what if you could actually be in the movies with your friends and, right, you could each give each other, you know, unique superpowers and interact within a virtual environment. And so, I think that if you look at where we are, and we sort of think that media is social today, right, because we can watch an NBA game or a football game and like have a tweet stream either, you know, on the side of it or on our phones. But imagine if you could actually be immersed in the whole experience and rather than, you know, needing a text-based metadata set of social interactions, if you could be physically immersed. And I think actually, if you look at what Darshan's doing with BigScreen,
11:32 Speaker 05 right, I mean, you can explain your product better than me, but it is like it is the most fun way to play video games with your friends, doing it in a VR environment versus the way we do it today.
11:43 Kyle Russell? Yeah, so really quickly, because I want to keep on the social aspect a little bit, Darshan, could you maybe explain what BigScreen does for its users?
11:48 Speaker 03 Sure. So BigScreen puts you into a virtual world. You can invite your friends or your colleagues into it. You'll see your computer in front of you, all of your existing apps, whether it's video games, or Netflix, or whatever you have on your computer screen. And when you look over, you'll see your friends. They could be thousands of miles away, but you'll see them, you'll see an avatar of them. It's like being in a living room or a conference room together with people that aren't there physically with you. That's the social collaborative power of VR being applied to our day-to-day lives that we already do today.
12:21 Kyle Russell? And so, kind of expanding on that, I have a recommendation for everyone in the audience of a video you should check out. On YouTube later, find time to search for Microsoft Research Hologportation. There's a demo from Microsoft Research where basically they surround a room with cameras and depth sensors and scan people in and are able to then kind of teleport their holograms to different environments. And so you get this kind of social experience, but it looks like they're in the room with you rather than making you, you know, put on a headset that completely blocks out the real world. And so I mentioned that to transition a little bit to AR, because, again, something that we hear a lot, even from people like Tim Cook at Apple, is, you know, they're
13:04 Kyle Russell? more excited about augmented reality than virtual reality because it doesn't have that blocking out the real world. You actually get to make eye contact with people. With that said, AR is a couple of years out even relative to VR in terms of difficulties around optics, in terms of computer vision. So as you're building virtual reality products and experiences today, how are you thinking about AR? What lessons that you're learning from virtual reality are going to apply to augmented reality as that, you know, kind of emerges, maybe becomes more dominant for certain use cases? What transfers over and where do you have to start with fresh eyes?
13:36 Speaker 03 I think fundamentally what you're talking about is a medium that is human experience. Like, that is the celluloid. It is actual human experience that you perceive as human experience. So AR is human experience in the place that you are. VR is human experience in any place that you could be transported. And I think
13:57 Speaker 04 What you're going to see in the long term, especially with virtual reality, is the sort of democratization of human experience in the same way that the internet led to the democratization of information. So you can, I mean, it sounds like hype, but we're in the early stages of it right now, where you can feel like you are anywhere.
14:16 Kyle Russell? And because the tech industry, we love to come up with new acronyms and more jargon to throw at people, there's a couple of other terms that people are kind of talking about a lot in this space. Mixed reality, the idea of merging virtual objects and augmented experiences together, kind of a nebulous blend. Intel talks about merged reality. So virtual reality experiences where it's scanning in the room and so it puts, if I was wearing a headset, it would project virtual avatars onto all of you so that I still feel like I'm kind of present here, even though I'm in a virtual world. Do you spend much time thinking about these kinds of blended experiences?
15:00 Kyle Russell? Or how do you even think about the relative mix that makes sense to go with as you're building things?
15:07 Speaker 05 I think your point is right, which is that the way it'll likely play out is that there's going to be a broad spectrum, right? From a purely synthetic virtual world, like The Matrix, all the way up to the point that you opened with us all having our Facebook and LinkedIn profiles floating above our heads in meetings. And I think that the, I mean, the good thing is that we're sort of in the stage of the medium now where everybody's working on the underlying infrastructure technologies and kind of the first experiences around those. And it doesn't matter if it's AR or VR, the investments that you need to make right now and the opportunities that that creates, I think, are exactly the same.
15:48 Speaker 05 And what will happen, I think, is that as both mediums mature, we'll start to see them diverge and see new applications that may work better in one versus the other, and then others that work well across both.
16:02 Kyle Russell? Right. And incorporating even more mediums, today, virtual reality is kind of growing out of gaming with things like what Within is working on, film is contributing to a lot of the content that's available.
16:17 Kyle Russell? In terms of, I guess the correct term would maybe be asymmetric experiences. So people who are in virtual reality, some of the people involved in the experience with other people are on their phones, also engaging with them. When does it make sense to build something asymmetric? And when does it make sense to build something where it's VR/AR from the outset and you don't compromise on that? I think that's like, kind of a philosophical debate.
16:41 Speaker 04 This is a question I was just asking you in the group.
16:42 Kyle Russell? Yeah, I know. And so I think that was a fun discussion. So I'd love to rehash on the bit, maybe.
16:47 Speaker 04 I mean, personally, I think you should build things that are the most inclusive to the most amount of people, that they can have compelling experiences with it. And when you get into the finer details, it really depends on the experiences that you're building. I think that there's, people are looking at this medium right now as it's either an evolution of cinema, or it's an evolution of video games. At first, it was an evolution of video games, and I think that, the reason that was, is because the people making it were all video gamers. It was like what Mark was talking about at the opening yesterday with Edison thought that the phonograph would be used for sermons, because that was his world, that was his reality. And the people that were building virtual reality were gamers. So everything was, is going to be about games.
17:30 Speaker 04 And then slowly, we started to see cinematic VR bubble up in the last few years, and then it's like, okay, well, maybe it's the evolution of movies. And it's actually, I don't think it's either of those, because I think it's its own new medium in its own right. It will take things from each of them, and it will evolve, like what Jason was saying, that the format is undefined. That's a really important thing to understand about this medium, that makes it actually different than other mediums, is if you look at cinema, for instance, you have, the format is birthed at the birth of the technology. You have a sequence of rectangles, played one after another. That's birthed with the motion picture camera, the motion picture projector. That format does not change over the lifespan of the medium. The language of storytelling, in the form of a feature film, or a television show, or
18:14 Speaker 04 digital video, or UGC, that does evolve, that does change, but the format always stays the same. What's different about virtual reality is the format is fluid. So you and you see it changing on a bimonthly basis where you're now able to tell a new kind of story because the actual format has changed.
18:34 Kyle Russell? To ground that a little bit, what kinds of specific changes unlock new types of experiences? Like what kinds of improvements to the underlying platforms kind of drive those new capabilities?
18:44 Speaker 04 Well, it's really, it's about levels of immersion and it's about your interactivity with the world around you. So right now you have a real-time rendered world that does not look real, that you can interact with, or you have a photographed experience that is photorealistic and that people look human and you can, at the moment, you can't walk around the room. What Jason is building, you will be able to walk around the room, but you can't interact with the person that's been shot. Brad Pitt was there, he was photographed, you can't walk up to Brad Pitt and start talking to him and he'll talk back to you. You imagine on the on the real-time rendered side, that's going to get faster.
19:27 Speaker 04 You're going to have real-time rendered humans and environments that are driven by AI that you can interact with. And at each stage of this, the storytelling structure also has to completely change.
19:38 Kyle Russell? Eventually, it just all culminates to Westworld. And so those of us in the room are privileged in that getting to try virtual and even some AR, virtual reality and even some AR experiences isn't completely out of reach just based on cost alone. Most of us can probably get access to one of these headsets if we really tried. And so I'd love to kind of just go through one by one. What's one experience that you would recommend? The device they would need and the specific experience that you think everyone should try to understand maybe a specific facet of where these spaces are going? I know that's a tricky one, so whoever kind of maybe has an answer top of mind can jump in first.
20:20 Speaker 03 I'd actually probably say, so Oculus made a film called Henry, and that was the first time where they have you like sit down on the floor and they have this experience that's like happening all around you. And it was like an interesting storytelling experience. And that was my personal first time where I had tried a what film in VR could really be like native to VR, taking advantage of VR. High-end VR headsets like the Oculus Rift, that's where, if you want to try Henry, try it on the Oculus Rift. That took advantage of being able to actually stand up, walk around a little bit and to really get a sense of being in a different place with a character that actually, the animated character makes eye contact with you.
21:05 Speaker 03 And I've heard anecdotally, people would take off the headset and they'd have a tear because it was like, oh, that was a cute story. It was a kind of a sad story. That was my first kind of a wow experience.
21:19 Kyle Russell? Got it. If no one has one prepared yet, I'll hop in with my recommendation. So I think an experience definitely worth trying is Destinations, an app from Valve on the HTC Vive. And so what Destinations does is it lets you basically upload entire environments that other people can explore and you can add things like stand at these specific points and an overlay will show up giving context about the scene you're in. There's one experience where they actually took thousands of photos from a Mars rover and used a technique called photogrammetry, which extracts the geometry from those photos to make a rendered scene. And so you actually get to stand on Mars.
22:04 Kyle Russell? And so you are kind of walking around and it gives you little blips of information about, oh, this is the rover, this is how long its mission was, this is how long it took to collect all these photos. And all I can think is, this is the future of museums in the nearer term. In the further term, this is the future of education. Like, why, you know, why read a textbook about, you know, significant moments in history when you could just kind of actually get to live it? I think that that's absolutely worth a try if you can get your hands on it.
22:33 Speaker 05 Chris is too modest, but like if you haven't seen his film Clouds Over Sidra, which is about the Syrian refugee crisis, I think you can watch here, right? In the Innovators Billion? Like, I mean, it's the most human way to connect someone to what's going on in another part of the world. And you watch it and you feel like you're there and you have a level of empathy for what's going on that you just don't get from reading the newspaper or watching YouTube or anything like that.
22:59 Speaker 04 I love Pearl, which is, you should see on a Vibe, which is Google, what are they called? Story Studios?
23:06 Speaker 05 Yeah.
23:07 Speaker 04 That's Oculus.
23:10 Speaker 05 Google Daydream Labs?
23:11 Speaker 04 No. Anyway, no. Story, it's Story something, anyway.
23:17 Speaker 05 Pearl.
23:19 Speaker 04 It's just a beautiful little animated story that you can lean and move around in, actually not move, but you're in the front seat of a car and the whole story takes place within the framing of this car between this father and daughter and it's music driven and it's just, it's really utilizing the limitations of what's possible now to form something really beautiful and human. I'll tell you just a personal story that's interesting that I haven't really shared publicly, which is, it's an experience that you can't actually have, but I would imagine you will be able to have in the coming years as the technology gets better. And it's very hard to capture video in VR right now. Jason's working on, you know, a super robust professional version. In the last three years, we've had to build our own camera systems to film the things
24:04 Speaker 04 that we want to film because there's no camera, the cameras didn't exist. And I filmed something, this was like two and a half years ago, I shot this soccer game in Los Angeles and we're doing a test and I was there with our camera tech and I was there with my girlfriend at the time and...
24:24 Speaker 04 It went onto a drive somewhere. We looked at some other footage and at the time, we were testing how close you could get and where you could put camera positions. And about a year later, someone found some of the footage on the drive that I had not seen and stitched it. And post that moment in time, I'd gone through a really difficult breakup with my girlfriend and it said, "Hey, you should, we found this thing on the drive, you should watch it." I put this headset on and I was standing next to myself, holding the hand of the girl that I was no longer speaking to. And I have never in my life experienced a more powerful piece of media than that moment. No movie, no anything.
25:06 Speaker 04 It was more powerful than any voicemail, than any photograph, than any video. It was like I was, you know, Ebenezer Scrooge in Christmas Past. Like I was standing there with my old self and I felt the presence of my old self and I felt the emotions of myself with my girlfriend at the time. And UGC is very difficult at the moment, but it will transform an industry of consumers capturing memories and replaying them.
25:39 Kyle Russell? Well, thank you for sharing that anecdote. I know that that might have just brought those memories up again. And thank you all for joining. To get a taste of the kinds of experiences we were talking about, there's a Within booth with a couple Gear VR headsets up at the Innovator's Pavilion. Thank you all for your time.
Previous Company Building in Crypto: A Conversation between Brian Armstrong and Chris Dixon Next What's Next for Education Startups Page updated October 9, 2026 · transcript and summary are machine-generated