Pod Insights a16z Podcast a16z Podcast | The Present State and Future Possibility of Virtual Reality About this episode Our virtual reality-enabled future is arriving, but it’s hard to know -- as it is with every new technology platform -- how quickly we’ll all make the transition to VR and what it will ultimately look like. For example, beyond gaming and entertainment what applications does it seem like VR is best suited for, and will we all be wearing full VR-enabled body suits some day? When will the Matrix become real?
a16z’s virtual reality-obsessed Kyle Russell and Sakunthala Pandit are joined by perceptual and computational neuroscience expert Beau Cronin, to offer their informed opinions on the present state and future of VR. They give the rationale for travel, shopping, and education getting the VR treatment, and why software developers would kill for a VR-enabled dev environment. So strap your “face toy” on and dive into virtual reality.
Transcript 10,428 words · 3 speakers
0:00 Speaker 1 Hi, this is the A16Z Podcast. I'm Kyle Russell, a deal partner here at the firm looking at virtual reality and augmented reality among a few other things. I'm joined today by two people who know a lot about VR. I'll let them introduce themselves so you can put a name to a voice. Saku, if you'd like to start off.
0:15 Speaker 2 Hey, I'm um Sakantala Pan Sivatna. I'm a V search I'm in the DLM research team here in Jason Horowitz. I focus on VR and other amongst other emerging technologies. Before this, I was an intern at Oculus.
0:28 Speaker 0 And hi, my name is Bo Cronin. I have a background in computer science and perceptual and computational neuroscience, and I've become particularly interested in VR recently because it allows us to, in many ways, expand our perception of both the real world and worlds that we get to make up from scratch.
0:46 Speaker 1 So thanks you both for uh joining today. Um really excited to talk about you know kind of where VR is at and you know what aspects of it I think are maybe underconsidered by people who just kind of look at it as a raw set of technologies of things that track motion and our screens attached to someone's face. I think that there's a lot of aspects where you know, if you look at mobile or desktop computing and other ways that we use technologies, uh aspects that are commonly discussed aren't even brought up sometimes when you're looking at VA uh when you're looking at VR. So one of the things I'd like to like kind of just check in on first is where VR is at today in terms of the technology and in the commercial space. So recently, uh Samsung and Oculus released the Gear VR, um, Saku. You actually used to work at Oculus, so maybe it'd be fun for you to talk about where uh the consumer platform is at today.
1:36 Speaker 1 Aaron Powell
1:37 Speaker 2 Right. So the big thing with Oculus is that um the VR headsets, which had which had been around for maybe 10, 15 years before that finally became affordable. So and also usable. So we got the right ratio um being VR high quality enough, and also it's reasonable for the average person to buy one. So um the Oculus Kickstarter came out in 2012 was a massive success. Um and then we've had two releases so far, the the DK1, which is the Kickstarter, the DK2, which is like a second developer kit that Oculus release. And we're still waiting for the the fi the consumer, which is like the super high quality version, which is coming out next year. Um
2:16 Speaker 1 then to kind of give an idea of the evolution of this tech. So DK1, basically a screen with motion sensors that you strapped to your face. And that gave you know something that felt like presence.
2:25 Speaker 2 Then
2:26 Speaker 1 with DK2, they uh took that same kind of core concept and added a camera that tracked your position.
2:30 Speaker 2 Actually, that's not quite true. Um so DK1, it didn't like we what we say with presence is um it's like a buzzword in VR to describe the visceral feeling of feeling like you're actually in the place that the the uh the screen says you are. So the way I explain it to people is if you're in a in a VR universe, you're standing on the ledge, you feel kind of scared that you're gonna fall off. Right. And it's not like a psychological thing, it's like you know, physical reaction to to the distance. Aaron Powell
2:55 Speaker 1 Right, your lizard brain reacting to input.
2:58 Speaker 2 Exactly. Um and DK1 was not quite there. Um but we're I
3:02 Speaker 1 don't know. Look at all the videos of people who would like go on a roller coaster and fall out of their seat.
3:06 Speaker 2 Well, okay, even the Oculus V said it wasn't okay. Maybe, you know, it it's it was it was um it definitely did hit your lizard brain, but maybe didn't quite click presence. Um but with DK2 and C V1, we definitely get to that point.
3:17 Speaker 0 Right. And I think one of the interesting things about each system that's come out, DK1, DK2, now the Gear VR, which we'll probably get to in a second, and then these prototype systems that we will also talk about that are gonna be commercially available, is that uh each one has kind of proven something additional about what this technology can do. So I think the DK1 for a lot of people, I've actually never used one, but many people who have, was the first time that they went, oh my gosh, you can do really high quality tracking. You know, didn't have positional tracking, but you can do really high quality, low-latency tracking. The screen is good enough that I can imagine where it's gonna go, and it allowed the right kind of people to sort of have that leap where they said, oh wow, this is this is going to be real. It's worth me devoting my time and energy to it. DK2 was beyond that. Now, Sam, with the with the gear VR, you have something that's available for anyone to buy for a hundred dollars on top of the price of a Samsung phone.
4:08 Speaker 0 That proves that this is something that people can enjoy in their own homes. And so it's just each of these systems sort of taking it one step beyond bringing this to a broader and broader audience and making them realize, oh wow, this really is for real. Aaron Powell
4:21 Speaker 2 I think one thing to add is that the different types of tracking we have in VR. So the DK1 had rotational tracking, so it would track uh, you know, if you turned your head, it would it would be able to tell which direction it's it had turned. Um but the DK2 had positional tracking, which was like an outside camera. So even if you were walking around and um it was no longer anything to do with you know something stuck to your head, it could actually tell it was like a more complex measurement of movement.
4:48 Speaker 1 Right. The extent to which DK1 felt accurate was to the point where Oculus could make an accurate neck model essentially is when you were moving your head around, did it kind of feel like it was following your head in a natural way. Then with DK2, you could actually lean around and look at something from different angles, and it was like you were acting that's kind of what gave the I'm here feeling of I see this like really cool I don't know, we'll say, you know, the demos we see are cars or you're in a video game that's an arcade-style shooter or something, and you're in a cockpit and you can go up and read the dials on the like mech you're in, where it's I feel like I'm there because I can move around.
5:21 Speaker 0 And this is actually a pretty good time for me to give my my pitch for the power of motion parallax, right? Because the ability to do that doesn't, I mean there's there's a reason that that is so visceral, makes things feel so real, is because that turns out to be an incredibly powerful cue for our brains to understand the structure of the environment that we have around us. So when you look at something like the Gear VR, which does not have uh positional tracking and positional tracking is not available in any uh purely mobile uh system that doesn't have an external camera, uh, it's taken a step back there. It doesn't have that positional tracking. And so when you go and you look at it, you'll find that a lot of the content on there, much of which is awesome and great and totally worth the price of admission, uh, is designed around having experiences that are far enough away from you that uh that that lack of positional tracking isn't as obvious, basically because the angles involved are so much smaller that it's not as important to have motion parallax, which is really important for what we call sort of near-field VR, things that are in your personal zone uh and just really to my mind kind of ratchets up the level of experience you can provide, a whole you know, a whole other level.
6:24 Speaker 0 Aaron
6:24 Speaker 1 Ross Powell Right. Because it's interesting because I think that a lot of people who are real VR fanatics think positional tracking is the end all be all. Yes, me. But no, exactly. So but when I played with the Gear VR, I did notice that so many of the other aspects improved. The resolution to the point where I could see pixels, but it wasn't particularly distracting. Frame rates were kept high enough, plus and this is kind of nerdy, but Samsung uses low persistence displays, so pixels are lit up for shorter amounts of time, so you have like less blur as you move your eyes across the screen. All these little things added up to an experience that really does feel like worth the $99, plus maybe the phone if you're like me and refuse to give up your iPhone. Uh but at the same time, I then go and do, and maybe now we can talk about the future a little bit, a uh Oculus C V1, which is the upcoming consumer model, or the HTC Vive dev kit, which is from uh HTC and Valve.
7:13 Speaker 1 You go and try those and you can walk around and you have controllers that track your hands, and that's an even it feels like a whole other generational shift where it's how could we go back to the previous way of doing things.
7:25 Speaker 0 Yeah, that's right. And so I think it's worth saying there is an incredible amount that they got right with the Gear VR. It's awesome. I mean, so the standard line that I've seen online, which I agree with is if you already have one of those phones, just go buy it. Don't think about it, go get it. It's it's a no-brainer, like and I I totally agree. Then thinking about the uh systems like that you just mentioned, the the Vive or sort of the upcoming version uh of the Oculus. Um yeah, there is absolutely a whole other level. So there's two pieces to that that I think are really important. One is, as you just mentioned, the ability to move around in larger volumes, and that's great. The other one, of course, is input and being able to use for both of those systems and in addition to also the uh the Sony system that's coming up as well, your hands in a very natural way without really having to think about it and have one-to-one highly accurate hand tracking using handheld controllers in those environments.
8:18 Speaker 0 And that opens up not just a whole lot of awesome game mechanics and so on and so forth, but it really starts the race, I think, for people who want to develop other kinds of applications in VR and want to think about this as an overall computing platform and think, you know, like what kinds of experiences can be provide there in terms of uh productivity apps, just as as one example.
8:39 Speaker 2 It's worth noting that the original idea of VR was not just a headset, it was a headset and gloves. So if you guys remember like uh Nintendo Virtual Boy, which is very AT is very gawky and retro. It wasn't just uh something you put in your face, but it also came with gloves. So that's always been part of the idea of VR. It's you know, it's it's not just something not just um something that you see, but something that you interact with. Aaron Powell
9:00 Speaker 1 Don't you miss when gadgets came with names like power glove. I really miss those days where it was like, no, now you're playing with power. Okay, cool. Um but no, it it is interesting to think that you know in a few years we're you know, people like us who are really into the cutting edge are going to have what, gloves plus hand controllers, plus maybe cameras tracking our hands. We're gonna have all of these different constraints that are gonna limit what kinds of software is made and also introducing other opportunities. So I think maybe now as we're thinking about the future, it's a good time to think about what does design look like for VR in terms of these differ mobile-based headsets where you're plugging in your phone. You know, they don't have positional tracking, but you know, they can render a 3D world with crazy high frame rates and it looks great. All the way up to these controllers where it feels like may you know, sometimes you forget that you're in a virtual reality because moving your hands feels so natural, interacting with the world feels so natural, especially as we add things like haptic feedback.
9:55 Speaker 1 But thinking about those constraints and the spectrum of what VR looks like, what What is development for that going to look like? I mean, are people going to is it going to be very like differentiated based on different categories? And how does that affect like what gets developed first? Aaron Powell
10:10 Speaker 0 Yeah, absolutely. So I'm not a designer, but I am following closely a number of very, very smart designers who are trying to figure out exactly these questions. What does interaction look like? What are the affordances that we want to offer? How can we make these uh interfaces at the same time, you know, at once powerful and very natural.
10:32 Speaker 2 So I think in VR design, there's a split between um like realistic design and the or skew morphic design and then something which is a little bit more abstract and a little more symbolic. So with the phone, um, you know, our buttons like everything seems to kind of have make an attempt to resemble a real world object, the skew morphic design. Um with VR, maybe a general purpose app, you know, it might be, you know, for your writing app, it might be a virtual desk and you're sitting there writing. Um that might be the way to do it. Or it might be that there's some kind of more abstract way of interacting with words in VR that we just haven't thought of yet. And it's it's worth remembering that these things take a long time to develop, like with film. Um, you know, it it took like 30 years before the jump cut was invented. So um that's like you know, you're cutting from you know it used to be just like filming um people on stage like like it's like a like a recorded play.
11:24 Speaker 2 Um and it took a while for all those tropes um of editing um to to develop. Aaron Powell
11:29 Speaker 0 So film is what is is one great analogy that we can use. Another one that's more recent is the transition to you know, multi-touch displays on uh on m on mobile. So if you go back to you know the initial uh iPhone release in 2007 and sort of you know where we where we started there and where we've ended up now, we as you know number one as a as a design community, but really as a larger society have absorbed a set of conventions that you know back in 2007, no one, not even the designers knew really that pinch and zoom and you know various kinds of swiping were gonna become these both dominant and natural and you know almost totally subconscious at this point uh, you know, uh components of those interfaces. So right now there are a lot of very smart young designers who are trying to explore this space and figure out, you know, what are the equivalents uh in VR.
12:16 Speaker 0 I mean, one of the challenges that they face is that the uh major systems that are coming out aren't exactly equivalent. So they don't really know what they're going to be able to count on between sort of the Oculus Touch controllers, uh the HTC Valve Vive Lighthouse controllers, which is a wonderful mouthful, and then the PlayStation, uh the PlayStation Move controllers, they all have you know some some pretty important differences between them. Aaron Powell
12:41 Speaker 1 I also think it's interesting to look at the way that the development of the software and then how it's received is going to shape the development of the underlying hardware and technology in terms of right now it's slot your phone into something and maybe have like the ability to tap and use gaze-based controls, and then you have the more advanced headsets, which are basically just uh screen and motion sensors and maybe extra controllers that you plug into something with a lot of raw CPU and GPU horsepower, like a gaming console or a gaming PC. But as these things evolve, it's you can see how it could evolve to be uh the phone still slots into a headset, but then when you're at home, you plug then that headset into something with all of that horsepower, and it's kind of a almost like surface-like it the device changes to meet the needs, or you can also see a standalone headset that has its own brains that's separate from your phone because it turns out people maybe also want to use the phone for some kind of input, and then again, that hooks up to raw horsepower, or you know, that model kind of goes away and the mobile-based approach just takes off even further.
13:46 Speaker 1 And you know, once they're as powerful as a gaming PC of today, you don't really mind the jump in graphics compared to like what your desktop could be.
13:55 Speaker 0 And there are certainly people in the field who say, look, the future of VR is is definitely mobile. It's just a matter of, and I mean Palmer Lucky may even be on record saying as I'm not sure that the tethered desktop version is a very important, but it's a it's it's a stepping stone really until we can get to that fully mobile version. I think one other um related point I would I would make is right now uh these systems and these hardware platforms are are all coming out, and it's wonderful that we have gaming and entertainment as sort of this this wedge application that is driving the initial investment that sort of guarantees there there will be some minimum viable market for these things because there's a certain subset of people who on day one will, you know, will shell out and are waiting for this to happen. But then of course there's this question, which is what does the larger market want? Once you get these out here, you know, what are people really going to find interesting?
14:43 Speaker 0 And so once you get these feedback loops between what the market is really responding to, I think that the set of applications that get a lot of attention, I mean, I have my guesses. I have some things that I think are going to be really compelling in VR, but you know, at the end of the day, most people I know in this space have their own set of guesses and we have fun, you know, debating, you know, what direction it's gonna go, but it's it's a really big open question. Aaron Powell
15:06 Speaker 2 I think even before we get to that point, we have to ask um like how is like um like our thesis is that um VR is not just gonna be for gaming, it's gonna be it's uh you know, it's like the next platform. I think this is what we we believe the Oculus as well. Um it's that this kind of like VR is an enabling technology which makes you feel like you can be anywhere. So that doesn't mean that I mean obviously that's great for entertainment. You can think, you know, wow, I'm in this like super cool gaming world, uh, you know, so much fun. But like there's like way broad applications um beyond that. Um it's just a question of like figuring out how and when that will happen. Aaron Powell
15:44 Speaker 0 So what does it mean for me be able for me to be able to move into any space at will? And especially when those spaces can be populated by either people I know and love and want to interact with, even if they're not around, or people that I've just met. So uh it shouldn't be forgotten that uh Oculus just released their their social alpha, right? Which is the ability to go in and watch uh watch a movie or or video in a fake movie theater with other people who can be from anywhere around the world. And it's very early and the level of interaction is very crude. But the fact that they thought this was, you know, a very early thing to push out there, you know, very quickly says something real, which is that I, you know, I think that uh for a long time, you know, one of the knocks against VR from various quarters has been that it's gonna be this very isolating experience.
16:29 Speaker 0 There has long been a thread, I think going if you go back to you know, Jaron Lanier, you know, back in the day, and then certainly continuing to today, of people saying, you know, wait a minute, really what's interesting about VR, or one of the most important things that's interesting about VR is that it allows us to share experiences with other people. The technology is gonna take a little while for it to catch up, but you know, I'm one believer, and I know that there are others who think that you know social is not just something that's a nice to have in VR, but for many application domains, a a core and fundamental capability that is going to be built into everything, and you know, we don't even really even know how powerful it's gonna be, but it's it's absolutely essential. Aaron Powell And I think a lot
17:11 Speaker 1 of people when we talk about capturing experiences and sharing them, think about using 360 degree video or in the future light fields to capture what's happening around you in terms of big events or concerts. And I think that that misses kind of how important in our lives the the boring times are. Um I think that we're gonna see a lot of social usage of people getting in their VR headsets and quickly scanning in the room they're in and then hanging out with their mom for 20 minutes instead of doing the phone call.
17:44 Speaker 0 Like playing poker, I think is a is a great example that people have used. I think there are some people who I you know in interact with who say, look, the day that I can sit around a poker table with a few friends and have the kind of eye contact and social experience that a good poker game has, that's when I know this technology will have arrived. On the one hand, it's very mundane. On the other hand, when you break down what's necessary to have a good poker experience, it's pretty non-trivial. There are a lot of things you have to be able to get right that that just aren't available today. Aaron Powell
18:09 Speaker 2 One thing that was really popular um on like I think there's this VR um company called AltSpace. One thing that they found was really popular was um people hanging out and watching YouTube videos together. And it's actually it's actually really interesting when you realize that the average person, I think over 40 spends like seven hours a day watching TV or something like that. Imagine if they spent that time, you know, hanging out with virtual friends instead. I think that's just kind of a nicer experience for everyone. Aaron Powell
18:34 Speaker 0 One of the most compelling experiences that I've had in VR was when next VR at the very beginning of the basketball season live streamed the first Warriors game. Um this is before, of course, we need the warriors were gonna go on and be the awesome team that they are this season. But so this was this was great, and the the camera view is right at the half-court line. You're on the court and you are kind of like if you were live at a tennis match, you you know, your head is going from side to side as you're watching this game. You're that close. And basketball is a phenomenal game for this because the ball is big enough that it works with the current resolution, and and they just in a lot of ways totally nailed it. And I was hoping it was gonna be awesome, and you know, I have been pretty unimpressed with a lot of things I've seen in VR. This was the future, and I want to be able to pay money to do this on a regular basis.
19:19 Speaker 0 But I was at that game alone, right? And there's no way for me now. I mean, my wife was, you know, sitting next to me on the couch, she had no idea what I was doing, and it was, you know, on some level incredibly compelling, but also very isolating thing. So I think a huge you know, nut to crack is how can I share those entertainment experiences with those with people who might be in the same room with me or might be across the country. And I think once you've once you've got that, there's just a a really compelling combination. Aaron Powell
19:46 Speaker 2 So one additional point about presence. Um even if you're so some people have said that even if you're with um like a very crude model of another person, it's even because if if you have VR presence, it's kind of still feels like you're with people. Um
20:00 Speaker 1 I think anyone who's tried the Oculus Toy Box demo can probably relate to that in that seeing a floating head and hands with no emotions whatsoever, no textures to convey gender or race or anything like that, you still feel like you're in the room with that person, even no matter what the distance is between you and just seeing how people just gesticulate while they're making a point and knowing that like, oh yeah, this isn't just it it feels so completely removed from the experience of like a Skype call where sure you're see you might even see the actual person doing that, but it doesn't feel like they're there. Aaron Powell
20:33 Speaker 0 So there's an interesting concept from from neuroscience called filling in, which has been around in neuroscience for for a long time, and now I see a lot of people in the VR community using it because it's very useful, which is that once and and it's a little it's a little complicated in the sense that it's hard to predict sort of a priori or from first principles exactly when the brain is going to sort of kick in and start filling things in. But you know, one example that people will often cite in VR is that if you get a certain amount of, for example, visual cues right, then even very incomplete or crude haptic cues will feel very complete because your brain is expecting them, and it's going to basically fill in all those details that you know may not actually be there in the real stimulus. So, you know, the most important thing is don't have your cues be disconsonant.
21:20 Speaker 0 Don't have them contradict one another. So make sure you don't do that. Uh the next level is you could have those haptic cues missing completely. But then if you can go one level beyond that and just have that mere suggestion of haptics or, you know, but whether it's the audio or whatever, uh, you can get a lot of bang for your buck uh with even relatively you know low fidelity and low resolution cues. The flip side though is that it's not always obvious, as I was just saying, what counts as low fidelity. And so there's a lot of experimentation that's taking place to figure out sort of what is the minimal sort of additional cues that we can provide. Uh but once you hit that minimum, you can have these incredibly rich multi-sensory experiences that really kick you into a new level of presence. Toy Box is one great example of that, and and there are a number of others that people are discovering.
22:07 Speaker 0 Aaron Ross Powell
22:08 Speaker 1 As someone who thinks about kind of the public perception of VR though, I will say though that it's a little uh disconcerting that people are already you know people see a computer strapped to your face and they go, oh that you know, that's what makes people think it's isolating. Aaron Powell I like the term face toy.
22:23 Speaker 0 Yeah. That's that's my personal.
22:25 Speaker 1 So over time, you know, as you add things like haptic feedback and you know, making it feel like your arm actually faces resistance as you do something in an experience. You know, are we all going to be strapping on like full-on body suits so that we can feel when our arm is nicked in a game? Oh, I bumped into that wall. Aaron Ross Powell,
22:39 Speaker 0 Jr. Are we gonna go full ready player one, right? Where they're sitting in a room by themselves, yeah, in these in these full body suits, probably catheterized and you know, not leaving for for quite some time. Um yeah, I I think there are there are big questions here about what is going to be socially acceptable and what's going to be socially acceptable in the home, what's going to be, you know, when you get into AR, you know, what about when you're out in in the real world? Obviously, there is a whole set of issues around you know, acceptability of Google Glass, which is I think the example that most people have in in their mind um when they think about it. Um, I don't know. I think one of the advantages of VR is that uh because it's fully immersive and you're not expected to interact with other people face to face and you're doing it to a large degree in the privacy of your own home, or you know, or or who knows, um, it kind of gets a pass on that at the beginning.
23:29 Speaker 0 Um, but these issues are going to be very, very important going forward. And it's really hard to predict how social acceptance is going to evolve, right? Aaron Powell
23:37 Speaker 2 Yeah. Some people are saying that the whole full bodysuit thing might be like a you know a uh a throw uh comeback for the arcade. So maybe not everyone will, you know, buy the like camera kit and body suit and like eye tracker and every single input device, but you might have some places which have all of those things, and then your average user might just use mobile VR. But
23:57 Speaker 1 it's an exceptionally standout experience where you have you feel presence unlike anything you could do at your own home
24:03 Speaker 0 for
24:04 Speaker 1 you know tens of thousands each the setup would be tens of thousands of dollars and things like that. In a uh very meaningful way, the virtual reality works better when it's tricking your brain along its various senses. You know, you're seeing things that look realistic, you're hearing things that make you feel like you're there. Maybe even you feel something that makes you feel like you touch something in that virtual space. So I'm gonna using that framework, I'm gonna pivot to an interesting application set that I think is going to be very important for virtual reality, which is education. You know, we talk about kids learn, some are better visual learners, some are better audio learners, some are better when they write something down or experience the actual task of doing that. So with virtual reality kind of making your brain see these things at a very like intimate w working with your wiring level, what do you think that means for the future of education?
24:50 Speaker 1 You know, Google is talking about putting shipping cardboard headsets to schools, but if you know people are actually able to experience something where they're walking around seeing the constitution be signed for the first time or you know, things like that, what what does that mean?
25:04 Speaker 2 Yeah, well, I the the obvious first application for VR in education is things like you know, flight simulators, learning to drive, things which have like a very visceral, like physical um hind-eye coordination component. So think of it like in the matrix when um Canary's like puts on the headsets like, oh, I know kung fu. I mean, maybe that's kind of what it will be like for a VR. Um it comes to more abstract things, it's can also really be um really helpful because you know, if you have the more intense and varied your memory, um like uh helps with remembering facts and uh you know, like seeing something on screen, you know, sixing your mind more than reading on a book, um, seeing in VR even more so. Aaron Powell
25:47 Speaker 0 So I think there are a few things. And as I I have a three-year-old daughter, I've been thinking a lot about you know how she's going to experience school and you know, if and when she'll ever even go to college and all these things. And so I think that um for VR in education, I think that it's fairly obvious that both history and science education should just be in VR. Like, why on earth would I want to, you know, read about a you know these important historical events rather than be present within them. I mean, so that to me is just a matter of producing the experiences or getting.
26:18 Speaker 1 Yeah, yeah. Yeah, we can't do that. Absolutely, absolutely. And and
26:21 Speaker 0 and there are and there are a couple of studios that are that are doing this. Um the the one experience that my three-year-old daughter has watched more than any other is this Apollo 11 experience, which if you haven't seen it, super awesome. They've done great assets for the Saturn V rocket and then inside the capsule, and she's actually been on the moon and the whole thing. It's not even in full release yet, but I have like the pre-release version, it's great. So for that kind of thing, like, duh, go do it. Like, let's get this out there. I mean, that that that's really obvious. Going back to uh what Saku was saying, where I've personally been pretty interested is how do we translate abstract concepts into VR. So for example, you know, CS concepts. So I don't really get distributed systems. It's never anything I really studied uh, you know, intensely. I've never had to implement them, but I know that they're really important.
27:08 Speaker 0 So what does it mean for a lot of different uh you know machines and computers to have to reach consensus. You know, what what how are those messages getting passed around? I haven't seen the VR version of the thing that teaches me that, but I bet when I do see it, I'm gonna understand those concepts much better than if I read about them, and maybe even better than if I implemented them. You know, maybe, maybe not. I'm not sure. But I think that I'd really like to see those kinds of abstract concepts be rendered, you know, in these environments. And I want to sort of see what that means. Aaron Powell
27:39 Speaker 2 I don't know if you guys have heard of uh Brett Fricks' Kill Math project. And so his idea was to take uh up shot concepts like math and translate it into just like symbols which you can kind of which kind of translate the ideas, but in a kind of visual and indirect way. Um like one thing you might do with VR is for to teach chemistry would be like have a virtual chemistry set, you can just move things around and experiment with them, and then you have you know, it you have to get one yourself. You can do dangerous things. Um
28:08 Speaker 1 you can see things like uh in the gaming space, traditional gaming space. You have things like Kerbal Space Program, which is kind of an animated silly take on building a rocket and trying to send it on missions to the moon or wherever else. And a lot of it is just trying to build it. Oh no, I failed. Let me tweak this one thing. Oh, I failed again. Uh, and you do that a hundred times, and then finally you understand some of the basic concepts of physics and like how these things actually work. And you look, and NASA is actually now trying to make a kid's version of this game that's maybe a little bit more simplified, but still gets the same fun aspects, you know, still captures that. And there's a few areas, you know, Bo, you brought up science education, like the looking at cell splitting, which do students engage with more. Uh a piece of paper, you know, printed on an 8x11 sheet where you see cells split in eight different frames or an animation that shows it in distinct colors or in a 3D model that shows what's happening inside the cell at a very detailed level.
29:02 Speaker 0 That's right. And so high fidelity, one of the Philip Rosedale's company is is using uh cell biology in particular as I think their their sort of initial application for for exactly this reason. And I've I've I haven't been I haven't seen it in VR, but I've seen it projected on a screen, but it's awesome. I mean it's it's it's great. And I I think that that kind of uh topic is is just is perfect for it. Aaron Powell
29:23 Speaker 2 Well, another use case I hear talks about a lot is um a VR dev environment. Nearly every developer I talk to about VR um wishes that they could have a headset that they could put on and then have you know 10 or 15 screens to code on. Um I know Bo you've had
29:38 Speaker 0 and one of the complaints right now is that people who are developing for VR particularly are taking on and putting off or put they have to take their HMD on and off hundreds of times throughout the day. So just at a minimum, you want to be able to create VR within VR. And then beyond that space, right, there's all the whole tool set. So there's creative tools, there's the equivalent. Of the Adobe creative suite, there are development environments of other of other types. What will uh an office environment within VR look like? How will that work? That that's clearly a big area. We know we need a lot of innovation there going back to the design and interact interaction uh issues we were talking out about before. So a ton of interesting stuff to be done there. But then beyond that, you know, sort of the tool space, which which has to happen or else you know we're not gonna get anywhere, there are there are a number of others which we can at least touch on touch on briefly.
30:28 Speaker 0 Um so one that I'm personally very excited about is you know, uh roughly speaking, how do you kill video conferencing, right? So, you know, everybody I don't know if you guys have an investment in any video conferencing companies, but I'll just go ahead and say it. Okay, then this is my opinion only. Everyone hates video conferencing. I mean, it's just uh it has a lot of real, real drawbacks. And you know, you can't uh really make eye contact with people, you don't really know who's paying attention. It's really you know, it's really difficult to have a natural and free-flowing conversation, just you know, a lot of real, real issues there. So if you could give um people the ability to, you know, be in a room together and have a productive business conversation together, there's a ton of value there. And I think a pre-existing market. Aaron Powell
31:12 Speaker 1 And I think that at the firm we definitely see instances of that, not quite through implemented through VR, but we use Beam Robots where you know it's the head on a tablet driven around by wheels. Uh and you can actually, yeah, it's in you're engaged with this person, it feels like you're talking directly to them because it's just their head looking back at you. You can tell when someone's distracted, like, no, we're actually engaging eye to eye. And when you see that, it's so much better than having to do an insert video conferencing application here. It it just feels like it's more productive because they're there with you. There's even if there's no delay whatsoever with the video chat, for some reason there's it feels like there's this distance. Right.
31:53 Speaker 0 And it's important to point out that within you know business communication, there are probably at least dozens of individual use cases. Some will be better with telepresence robot, some are probably really well served right now, others, you know, others much less so. I was really excited to see that AltSpace and of course their early adopters, they do their stand-up meetings in VR right now. And I think that's awesome. I mean, it's obviously weird and awkward to a certain degree right now, but that is that that's definitely the future. So, you know, then there's there's business communication, but then you can also imagine you know personal communication. So I think you know, FaceTime has been wonderful for grandparents everywhere. Um so the question is, you know, what is the equivalent that we can provide in VR? And you know, my personal opinion is that that's probably a little bit further away because you know, you're dealing with normals, roughly speaking.
32:38 Speaker 0 And so how how do you get that uh you know, how do you how do you get that at level of adoption? But I think there's also you know a big, big opportunity there. Aaron Powell
32:47 Speaker 1 And there are issues just you know, what kind of avatars will people have? Do you just go with the untextured blank face of Toy Box? Do you try to scan people's faces in? Do you let them create their own avatars like a video game? What works, what's broadly acceptable? Aaron Powell
33:01 Speaker 0 Yeah, absolutely. I I think that the choice of avatar, uh both resolution and you know, realism is incredibly interesting. Um I can make cases for any of the possibilities that you just laid out, and I feel like I have basically no idea which way it's gonna go because this gets into the social acceptance issues and they're really dynamic. Aaron Powell
33:18 Speaker 2 Yeah, I think one interesting other point is that if you if you have really, really good tel telepresence, that affects like how we behave in other ways. Like think about how many miles people travel every year for business. You know, how how far people go visit their friends, you know, um driving, like all those kinds of things. I mean,
33:35 Speaker 0 yeah, and that starts to point out that VR, like pretty much any very general purpose technology and computing platform is going to have some really interesting second and third order effects. Uh and it's we we we can sit here and make guesses, and a lot at the end of the day, a lot of it's gonna come down to some pretty fine-grained issues of uh economics and the quality that's provided and a lot of things. And so yeah, I think absolutely, you know, one of the possibilities is that uh self-driving cars may be less important than we think they're going to be because people find it less important to travel for the same reasons that they do right now. I'm not saying that's going to happen. I'm saying it's it's entirely possible and that VR at least you know plausibly has the power, you know, to have an effect on that level. And you know, could you know just as self-driving cars are probably going to have a pretty big effect on how real estate value is distributed because certain areas will be easier and and harder to get to, I think you know, VR too.
34:32 Speaker 0 I mean, if we have to travel less, if commuting daily for a job becomes less important because you know VR telepresence you know really delivers on its promise, then in 10 or 15 years, you know, what effect is that gonna have on you know where people choose to live relative to where they work and you know and all those issues. I think you know there are a lot of big possibilities there. At this point, we're just guessing about exactly how it's all gonna net out. Aaron Powell
34:58 Speaker 1 Just so that we kind of cover the spectrum of what you know the nerds are will really appreciate in our VR-enabled future and what people in the real world are going to want to experience, people who aren't making it up as they go in their garage. Uh I think an interesting aspect of VR and kind of related category, augmented reality is fashion. You know, the idea that once you have this avatar that represents you in all of these spaces, uh, could you try use that avatar to try on virtual goods that then correlate to the purchases you'll make in the real world. Aaron Powell
35:27 Speaker 2 I think shopping's interesting because e-commerce is has been growing steadily over the last 10 or 15 years, but it's still only like seven to ten percent of all the things that we buy. And given that you know people are willing to like instacut their groceries, they're willing to buy pretty much everything online and closed are like the things that we buy the second most often, and yet it still quite hasn't made the leak to e-commerce yet. And part of the reason with that is you know, it's um you don't want to buy something which doesn't fit, and then you have like you know, some kind of wonky thing that you have to return later. Um so I think VR is interesting in that way. It's it's worth noting that um like online retailers have been trying this problem for a long time. They've been they have these like virtual models, and they're like, oh, maybe people can try things on this way, but none of them have really caught on. So it might be VR that gives it the final push.
36:12 Speaker 2 Aaron Powell
36:13 Speaker 0 Right. So if you look at the argument for the continuing relevance of you know brick and mortar retail, you know, 10, 15, 20 years from now, it's that there are certain physical experiences that you need to have in the context of that store that you're not gonna be able to get at home. And so you know, the idea behind uh behind VR and also and also AR, which we'll get to in a second, is that it really eliminates that that final reason that you actually would ever have to go to a store. Uh so this is you know far more uh the argument is far easier to make in some areas and others. Fashion is is is a great one. Uh another one that I saw pointed out recently uh online is you know furniture and sort of you know large goods. Uh and this sort of gets to the whole area that I think is really interesting of you know architecture and interior design in general, which is that if I can you know envision a space and put new things in it and try them out, holy crap, do I want to do that, right?
37:04 Speaker 0 That is so obviously, at least you know, at a conceptual level, better than the alternatives. As someone who recently, you know, uh bought and furnished a house, oh my gosh, would I have preferred, you know, A, when I was shopping for houses to be able to do virtual walkthroughs, and that's you know, it seems to me like a slam dunk case that um you know will be out there very soon and there there are companies that are going after that opportunity. But then once I had that house, you know, being able to virtually furnish it before uh before actually getting these very I mean I still have a set of closet doors sitting in my garage that are probably never gonna leave my garage because they didn't turn out to be the right ones, and I you know just couldn't know that based on what I saw online. If I could have hung those doors virtually, I wouldn't have those doors sitting in my garage for the next 10 years.
37:51 Speaker 0 And that's you know, that that's pretty compelling. Aaron Powell And
37:55 Speaker 1 with something as intimate as fashion, you know, uh a men's small, medium large, woman's zero, six, and twelve, those all mean very different things at different retailers. And but at the same time, they all have the absolute data of how much fabric is in every different size and every different garment. So why not in the future put your exact measurements into a virtual reality viewing application and see exactly how that piece of clothing drapes over your body size. I mean, what that seems like, you know, we talk about Amazon having two-day shipping or even same-day shipping, and oh, that lets you return things easier and you'll try more things. If you can know exactly how this shirt or dress work on you,
38:33 Speaker 0 I I shop for a lot of clothes online, and I basically accept that only about half of them are gonna fit. And right now for me, it's still so much more convenient as me to do it that way. I just sort of I eat that, but that's clearly you know suboptimal. There's a lot of room for improvement along exactly those lines. Aaron Powell
38:49 Speaker 2 It's worth noting that fashion is a very low-tech industry in general. I don't know if you guys have heard of Zara in the whole Foss Fashion Movement. That was applying a very small amount of technology. Um, you know, just like a little bit of analytics and a little bit of prediction, and then they just became really huge just from you know measuring what people were buying. And so, you know, I think like you know, the fashion industry could really get the huge boost from something like this.
39:12 Speaker 1 If you used uh the DK2 and gone on Oculus Share, or if you now have the new gear VR and go s you know, check out the experiences section of the Oculus store. One of the things that was pretty quick to arrive on those platforms was, oh, here, live on a fantasy beach for 10 minutes. Uh and I think that that kind of that's a very early example of something that also could be big in the future, which is the idea of you know virtual tourism or virtual travel, the idea of if you can simulate presence and being somewhere and capture environments in near photorealistic detail, why not just go there in a headset versus making the real trip and spending maybe thousands of dollars to go see something where you know, frankly, you can't actually go up to that piece of artwork and touch it and feel it. So what's the difference? Aaron Powell
39:57 Speaker 2 Yeah, actually um the company jump VR was they started. It because one of the founders went on a trip to to like Monument Valley in Utah. And he thought, oh my God, this is so amazing. I want to look at it any time I want. And so after that, they made this 360 camera recording rig. So what it does is it it's like a ring of cameras in a row and it records video in 360 degrees. So then you can go to some like really cool location, record what it's like, and then replay it back to yourself on your headset. The other way people are doing capture is through doing light field tech photography. So that means it records like how the light is falling everywhere in so it's another camera rig, but it kind of reconstructs the scene in a slightly different way to 360 video. It's includes lots more information about how the lighting falls on all the objects in the scene.
40:47 Speaker 2 And so that's just another way of playing back the experience of being in some cool place that you can visit.
40:54 Speaker 1 For listeners who maybe haven't tried kind of the scope of these experiences from 360 video to light fields, I think it's important maybe to differentiate in practical terms what those technologies provide in terms of ease of capturing content and then what the experience will actually be like for the viewer. So with 360 degree video, it's actually using traditional camera sensors and optics and then stitching together that footage so that it's 360 degrees around you, and maybe there's even a 3D effect, so there's some depth to what you're looking at. With light fields on the other hand, uh you're actually capturing this as like 3D points. So when you're viewing it, it's not just something that's a video playing at a fixed distance from your head, but there's even the possibility of you know, as we talked about earlier with the DK2 and having camera and being able to look in and what that means with these light fields, you can actually do that and see what you captured from different angles and you know, again, it contributes to the feeling of being there.
41:51 Speaker 1 But at the same time, there are a couple of downsides. Saku, could you kind of detail maybe what the downsides are to light fields given they sound so appealing. Aaron Powell
42:00 Speaker 2 Yeah, light fields the main problem is to just say it's a long time to capture uh, you know, like you know, to take a scene and then take everything in it and everything about the lighting information and then you know, replay it back. Like that's a lot of computational work. But on the plus side, that's a very parallelizable problem. So we might see more of it in the future. But um, so uh
42:21 Speaker 0 and one way of thinking about that is in when uh Litro, Lightro, when it when they recently announced their their light field capture system, yeah, they announced a camera, but alongside it as part of the system is the giant rack of basically disk and compute that you need just to you know capture and then preview the content that you're capturing. It's so it's just you know, at a computational level, it's it's it's a massive problem. The amount of data that you're capturing on a you know bits per second basis is is just truly, truly massive. Aaron Powell
42:51 Speaker 2 It's worth mentioning that most VR isn't done using either 360 or light field. It's done in the same ways games are made using 3D modeling and rendering, like the same techniques people use to do special effects in movies.
43:02 Speaker 1 Yeah, no, and in the medium slash long term, these also it seems like there's potential for them to kind of blend together. Right. You're capturing light fields, but then you do some rough uh computation on it to build you know a underlying 3D model and then you slap the light field on top of it as a texture essentially. Right. And then you have a light field that you can also maybe interact with in a game-like way. And it's fascinating to think about what that would enable then where you capture Rome, but oh, also you can go into the shop or like open the door or you know, in maybe something less uh you know, uh light touch, you know, something like a video game like Grand Theft Auto where you use light fields to capture a real city and then you play around with it. Right.
43:42 Speaker 2 Right. Like uh one cool thing about um Unreal is that they've been making this big push to include like Hollywood quality assets in their game making. So like a lot of VR developers have been using like you know, these these assets which are being captured from like motion motion capture studios or like you know, if um sometimes even light field stuff, um they incorporate it into their games, and so you have this kind of blend between the traditional video game way of doing things and then the new uh light field or um motion capture way of doing things. Aaron Powell
44:12 Speaker 0 One more thing on Lightfield. So uh light fields are really hard to wrap your mind around. Uh and a podcast might be the worst possible medium to explain white light fields. Uh so I just want to uh put in a plug and maybe we'll put it in the show notes for a great set of talks that came out of the USC uh ICT lab that explain it and uh they're awesome. And if you want to understand this topic, it's it's far into explained by the absolute best people in the world to be doing this explanation uh and that is what you should watch in order to sort of get a handle on the topic. So I'm really incredibly excited about the possibilities for uh for travel in VR. I I think it's it's worth pointing out that this is an area where uh the quality of the experience matters a ton. And so I think you know, there's the great uh Paul Sappho line, which is uh never mistake a clear view for a short path or or something along those lines.
45:05 Speaker 0 So so clearly being able to transport yourself to somewhere that you can't actually physically visit or that would be too expensive is just uh uh a huge and uh and I I strongly believe um in this application. But for it to be more than a novelty, and as someone who's tried these out, I think that you know um whether it's uh light fields or even more um exotic uh uh additions like redirected walking in a room scale environment, so really quickly, redirected walking is where you give someone the impression that they are in this sort of infinitely large environment by subtly changing the way in which the virtual scene is rendered so that they are basically prevented from you know walking into a wall, but they walk out to feel like it.
45:48 Speaker 1 You walk a little bit to the left and but what you think you're going in a straight line.
45:50 Speaker 0 Yeah. That's right. And there are all kinds of tricks where you know, as people are either you know going through doors or turning by 60 degrees, you can turn them by maybe 80 degrees and they won't notice, and it's it's it's super cool. So you know, there's gonna be quite a bit that's necessary for me to really feel like I'm at the pyramids of Giza to the level where I don't actually need to go to the pyramids of Giza for this to, you know, um to have uh uh an an experience that I would consider to be to be comparable. But once you've gotten to that point, um I think there's just an incredible amount that that is unlocked. I I think that you know one of the, you know, one of the great potentials for VR is just to uh give a give far more people in the world the ability to have the awesome experiences that currently only a relatively very small number of people can afford or are in the right place at the right time to experience.
46:43 Speaker 0 Um so you know, being courtside at a basketball game is one kind of prosaic example of that. I mean, you're basically removing the scarcity of courtside seats. You are you are transitioning that scarcity from the world of atoms and you know it's now you know zero marginal cost, world of bits, right? It's it's sort of that that standard transition, but applied to uh what up until now were you know were very physical embodied experiences. And I don't want to claim that this is going to be uh a quick transition or that people are gonna consider this to be uh you know a fair trade any time soon. But I do think that is to my mind, sort of one of the most exciting sort of long term uh trends that we have here. Yeah.
47:23 Speaker 1 I'm actually surprised we managed to cover a lot of bases in VR. We talked about the technology and the medium itself, but also kind of design paradigms and things to consider. So I really appreciate you guys you know joining and sitting here for an hour plus to really go and dig into all of these aspects. Um thanks for
47:40 Speaker 2 having us. Yeah, it's
47:40 Speaker 1 been great. Thanks
47:41 Speaker 2 a lot.
Previous Company Building in Crypto: A Conversation between Brian Armstrong and Chris Dixon Next What's Next for Education Startups Page updated August 25, 2026 · transcript and summary are machine-generated