Digging Up Digital Cadavers in XR, with Sector 5 Digital’s Jeff Meisner

Today’s guest — Sector 5 Digital‘s Jeff Meisner — hopes to put grave robbers out of business, among other things. He pops in to talk to Alan about all the experiential learning experiences his company has developed, from digital cadavers to study anatomy, to the VR design process of Bell Helicopters.

[Editor’s Note: due to an uploading error on my part, this episode was previously released last week with the wrong audio. We’re re-releasing today with the correct audio. We appreciate your understanding, and in particular, Jeff Meisner’s understanding in this matter – Chris, Podcast Editor]

Alan: Hi, I’m Alan Smithson. And today, we’re speaking with Jeff Meisner, CEO of Sector 5 Digital, about their pioneering work on the Fantastic Journey Anatomy VR Ride, Fork Lift Training Simulator, and the work they did with Bell Helicopters, shortening design times from years to months. All of this and more on the XR for Business Podcast.

Jeff, welcome to the show, my friend.

Jeff: Thanks, Alan.

Alan: I am super excited. So, Jeff, you are doing some incredible work at Sector 5. Let’s start with the Fantastic Journey Anatomy VR. Right. This just blows my mind.

Jeff: Yeah. Yes. Just as a historical perspective on this, we’ve been working with this particular healthcare client for a couple of years now. And we started out initially doing a 3D digital cadaver, basically, that allowed them to do facial anatomy. And the company is in the business of doing injections into the face and hand. And so they needed a way to have safe areas so the injectors would have training. So we created a basic virtual training tool and that was initially in 3D, not in VR, but it was driven through our tablets and things like that. So it had kind of an AR component to it.

Alan: You will learn in 3D dramatically better than even just on a 2D screen.

Jeff: Yeah, exactly. And we actually did a conference which had over somewhere between 200-300 of their folks training with a massive 3D screen in front of them. So it was used as a training aid, and really now, it’s gone global. So it started initially in the U.S. and got picked up by this company, because they are a global company. And what they wanted to do was take that next step, if you will. And so We’re creating this, what we call a VR Fantastic Anatomy Journey. We’re going to be taking their folks through… well, if you know what Fantastic [Voyage] is —- as most people do — but taking them through the human body. So you’re going to have a really cool edutainment-type experience, whereby you’re going to be on somewhat of a of a VR roller coaster, although it being through the body, we’re going to be adding some elements of teaching at various points. So it’ll stop and you’ll be asked questions. It’s really, the major focus is to be very much a learning experience. But one of the things we’re finding —- and I know you are too, Alan —- is if you make it fun or people, it becomes a much more memorable experience and they want to do it again and again. We’re combining kind of that gaming-type element, if you will, but with actual data and experience, to make it something that their injectors are going to be learning from, and not just the entertainment element.

Alan: When you guys started rolling out the 3D digital cadaver, how are they measuring against baseline? So, what was their baseline learning before? Just a textbook? Or..?

Jeff: No, they were actually using “live” cadavers, and cadavers — and this may sound a little gruesome — but they’re somewhat hard to come by, especially outside of the US. The regulatory issues that you deal with are very, very high barriers there. When we came along with the virtual cadaver initially, as I said, it was really only being used in a very small area. But when they realized that they could take this globally, and they now didn’t have the same barriers that they had in the past, that really opened things up for them and opened up their eyes as to the value that this would bring.

Alan: It’s really incredible. Medical is the by far and away leveraging virtual reality more than any other sector. I mean, design is kind of a close second. You’ve also done some work with design, and we’ll get into that in a bit, but the medical industry is just ripe for disruption. I mean, buying a cadaver, like you said, is onerous, is expensive. It’s heavily regulated. And let’s be honest, people don’t really need that anymore with VR.

Jeff: Correct. And even if these sorts of things are used in front of the intensive training — obviously there’s regulatory bodies and things like that would demand certain certifications and things like that — and in some of that, we can do. But we also can be right in front of that to, again, make these experiences a little more fun, a little more engaging. I will have to tell you that one comment that we got from the training we did was this lady actually said that our training was “more realistic than the cadaver,” which I had to laugh, because it’s like, how could that possibly be? But the fact that she thought that this thing was so immersive that it was actually better than the training on a cadaver, really spoke to me.

Alan: That’s incredible. That’s nuts. But if you think about it, you only get one shot at a cadaver. If you’re pulling it apart, you really can’t, like, split the face apart and look inside the brain if you want. With VR, you can take the whole skeletal layer right out. You can literally see the layers as you need to see them, versus cutting into the skin. There’s definitely, I think, for the physical training of the last part where you have to learn how to inject or that sort of thing, you still need that feel. You need to know how far you go down — there’s a feeling to it that I don’t think will be replaced. But when you’re learning about the anatomy, I don’t think there’s any better way; you can literally just remove all you want. “I don’t want the body to have any skin. Okay. There’s the bones.” This is really powerful.

Jeff: And a lot of people now, instead of what, in the past, you know, they had to fly doctors and nurses, nurse practitioners into training centers. And the cost of doing that was amazing. Now they can go to these areas, to these doctors, set these things up, as you know. I mean, all you really need is a fairly small technology setup.

Alan: Are you guys moving the stuff to Quest now? Or are you still on Vives and Rifts?

Jeff: Yeah, we’re fairly hardware agnostic, so we’ve done projects in just about any type of hardware. I will say we do have the strongest partnership with HTC — with Vive. We’ll get into that a little bit more when we talk about the forklift project. But I mean, it really doesn’t matter to us. As you know, the technology is changing so quickly right now. And so we’ll look at it from a perspective of what is the best technology available at the time for the client.

Alan: Let’s move from Fantastic Journey Anatomy VR and let’s talk about your forklift training, because you mentioned it a couple of times, and training people on moving vehicles, forklifts, excavators, anything where they have to drive around — it becomes really expensive. It’s kind of like bringing a cadaver in; bringing people into a facility and letting them drive around on a forklift, when they have zero experience, is a little bit dangerous, and a little bit expensive. But imagine giving them the ability to put on a headset, practice driving around a warehouse, practice some close calls, maybe some things fall off the shelf. You could practice scenarios that may happen in a warehouse, but are very rare. But you can give people that real sense of practice before they even step on the machine. Talk us through that forklift training, and how that came to be.

Jeff: Sure. Sure. As I mentioned, we have a really strong partnership with HTC Vive and they were coming out with their new Vive Eye Pro headset, which has the eye-tracking in it. And so we’ve been talking to them and we were ourselves trying to think of the best demo that we could go, because we had a conference coming up — the EWTS conference that was just in Dallas fairly recently — and we were talking to them and we said, when we do something that’s really going to be an enterprise-type application, that’s going to take advantage of the new capabilities of the Vive Eye Pro. And so we kind of went through project scope with the folks at Vive and we came up with the forklift training demo. What’s unique and different about this one is utilizing the the eye tracking software of the Vive Pro. We take the user through a simulation where they’re essentially inside a forklift. They have to drive a forklift in a warehouse, and we give them some visual cues. For example, if they look at the row of pallets that are in the warehouse, one of the pallets, we’ll highlight in green. So then they know, that’s where they need to go in and pick up that pallet. And then we also have a loading dock area where there’s different, like, A through F, I think loading dock locations. So once they pick up the pallet, it’s highlighted where they need to drop that pallet off on the loading dock. But in between that process, we are tracking all of their eye movements. So, for example, if they pick up the pallet and they don’t turn their head around to see what’s behind them? We’re tracking all of that. And I think where it really hit home was we put a replay function into the application. So someone will go through this experience — and we timed it specifically to be about a three minute experience because we knew we were taking it to a trade show — but we did a replay in double time. So we had about a one-and-a-half minute replay. And through the replay process, it shows exactly where their eyes were through the entire experience.

Alan: Cool.

Jeff: Yeah.

Alan: What kind of insights are you able to glean from that?

Jeff: Basically, we also — kind of on the side — again, we made a little fun, made it into a game with kind of a leader board. But you got points for doing things right, and then you got points deducted for doing things wrong. So, for example, if you didn’t turn your head when you were backing up, or you didn’t see the cones that were in the warehouse — those sorts of things — you got points deducted. So we can actually take a visual which is showing exactly where the eyes were, and here’s where you got points deducted, or where you got additional points for doing things right. So it’s that training element that reinforces to them, “oh, OK. Yeah, my eyes were down instead of looking up because I had… I should have been looking up higher to the third row of the pallets,” and things like that. So you have a visual interpretation of what you’ve done, but you also have, combined with a scoring system, to reinforce the points or reinforce what what you did wrong.

Alan: That’s really impressive. In one of the things that I think this will do for people is really shorten the training times, because if you’re training on a real machine and you don’t do these things, we have no way as a trainer, or as somebody teaching you, knowing whether you did it right or not. In VR with the eye tracking, now you can say, “hey, you didn’t do this right. Do it again until you get it right.” And people can repeat the training as often or as much as they need to perfect mastery. Are you seeing a decrease in training times with this as well?

Jeff: We haven’t gotten to that point, because it is fairly new to now see, once they actually get on the forklift, is that reducing the training times? But that’s a definite goal, is to have those metrics. And as I said before, this kind of can front end that initial training that people get, that’s fairly boring and people are just, they want to get on the actual forklift. Well, let’s do things ahead of that so that when they get on the forklift, they’re not hurting themselves. They have the concept. “I’ve got to turn my head. I’ve got to look.” You know, those those sorts of things. So I think that’s yet to come. But that is definitely the goal of this moving forward. The other thing that we noticed, at EWTS — because we had over 100 different corporations take this forklift demo in and try it out at the show — what we noticed was a lot of them were saying, “yeah, we have over a thousand forklifts of all different types,” because I know that there has been some really immersive form of training done in the past for specific forklift manufacturers. And that’s fantastic, probably as a next step. The issue, though, is that a lot of these corporations have four or five different forklift manufacturers, so they have something a little more generic. That kind of front ends up process is very valuable for them.

Alan: And I would think that despite the fact that there’s 20 different types of forklift, safety protocols are probably very similar regardless of the machine itself.

Jeff: Exactly. “Hey, you need to follow your eyes. You need to be alert. You need to be looking all around.” Those sorts of things are absolutely the same, regardless of the type of forklift, or regardless of the type of warehouse or materials. So that’s what really makes this exciting.

Alan: One other thing that I thought would be really cool is making an option where it’s like an open play version, where you can have fire come out the back of your forklift and it can go really fast.

Jeff: [laughs]

Alan: Maybe some missiles or something. No?

Jeff: I’m sure any studio would love to add that, Alan.

Alan: [laughs] Yes. You’ve got to shoot the boxes instead of pick them up.

Jeff: Yeah, there you go.

Alan: So you you recently deployed 200 virtual reality headsets for a large airline. You want to talk about that?

Jeff: Yeah. The exciting thing about that, and I think this is part of the value that XR brings, is that we did a project — and this is going back over five years ago — we did a project for the world’s largest airline, whereby they were placing, at the time, the largest commercial aircraft purchase. And we modelled all of their business class and first class cabins in 3D. So we use these 3D models, we did a a website for them. We did a kiosk — in-terminal activations — for them. And so these digital assets that have been been used and been around for over five years, we then took those digital assets into VR and we showed that off. We initially did four headsets at their leadership conference and then their global sales executives just went crazy over it. And they said, “this is exactly what we need in Brazil and in Japan and in Europe and all over the world, because when our sales folks are sitting down with these corporate buyers and these buyers are trying to decide if they want to travel on on this airline or another airline, we can actually take these headsets and we put it into the Oculus Go headset and we could take these headsets, give them to these corporate-travel buyers and say, put this on. This is what your executives would be experiencing if they were traveling on the new 777. Three hundred in first class or business class. And here’s the bar. And they can explore the cabin and they can see how the seats fold down and they can check out the workspace and all of those things.” So that was really revolutionary to them, having that experience and making it much more immersive. And it’s really fundamentally changed the game for that.

Alan: They’re taking the headsets and they’re going to the trade shows or customer meetings. How are they using it?

Jeff: Yeah, they’re actually taking them to customer meetings them. And now, as you know, the barriers to entry with the technology has now come down to make this opportunity available. And also the fidelity of the headsets themselves have gotten so much better that they’re actually going to meetings with these corporations, because — these are like global 2,000 corporations — that their executives are travelling millions of miles on various airlines and international travel. And so they’re actually taking that into those meetings, and not only telling those corporate travel buyers about all the benefits of traveling on this airline, but also they can actually put them in and make it a more immersive experience.

Alan: So we went from training people in forklifts to selling people on airlines.

Jeff: Yeah.

Alan: Oh, and digital cadavers. Holy crap. So you guys are at the forefront of medical, industrial, and sales. And there’s one more big one that… before we put a pin on American — guess I can say what the company has since you said it was the largest — American Airlines.

Jeff: Sure.

Alan: How are they comparing the sales with and without it? Is there a way to do that, or are they just anecdotally saying it’s better?

Jeff: Well, they’ve done their own internal [research]. But unfortunately, for competitive reasons, we’re not allowed to say, you know, how much. But, yeah, they definitely had some kind of before-and-after, A-B-type testing as to how it was before and the response versus how it is now.

Alan: One project that I know you can talk about — because we talked about it previously, and I know there was some massive savings being gained here — but that’s the product you did for Bell Helicopters. Let’s talk about that. These guys designed a helicopter and took it from years to months.

Jeff: Yes. And really, we had been working with Bell for a few years. We had done a VR experience for one of their military aircraft that had not yet… had not actually been built yet. But it was a project that they were selling to the military. And so we did it. We did a VR experience for them, which took them on kind of a mission, if you will. But it was really an eye-opening experience for the CEO of Bell. He’s a game-changer. I mean, he is such a forward-thinking leader. And he really wanted to transform Bell from a really historical, very engineering-centric military helicopter manufacturer to be a leading-edge technology provider of urban mobility solutions. And this was part of the journey, was when he saw the response at this particular show. And this show is called AUSA. It’s one of the world’s largest army show, I believe. And when he saw the response there, he said, “we need to be thinking more like the car companies and we need to be coming up with concept aircraft and things like that.” And he challenged his team — he had created an innovation team within Bell — his team, and this was basically October. On the commercial side, they had the world’s largest commercial helicopter show coming up in March. And he said, “I want a concept aircraft on the show floor in March.” And so basically from October to March, which ended up being less than six months, we went from the sketches of the aircraft; over 100 sketches to 3D models, put the 3D models into VR. Actually had Bell’s test pilots put the headset on and give us different changes back and forth of the aircraft. And we went through that iterative process and came out the other end with only a single 1:1 scale mockup of this aircraft. And we were on the show floor at at Heli-Expo in March. And oh, by the way, we also incorporated two Microsoft Hololens experiences into that one; one for the pilot and one for the passenger. But it was really revolutionary, in that it was really one of the best use cases of enterprise XR — taking a process that historically had taken them years to go through, and multiple models, and to get to that point, and we shrunk that down into less than six months.

Alan: Well, I mean, if that’s an indication, it looks like right across the board, you guys have been in medical, in industrial, in sales and marketing, and design. You guys have touched everything. Sector 5 Digital is kind of like this powerhouse of design in spatial computing. And I’m really excited for what’s next. So, Jeff, what is one problem in the world that you want to see solved using XR Technologies?

Jeff: I think it goes back to the training. If we can take people out of the environments that are hazardous — safety environments — and we can provide training to them such that they can learn to the point that they no longer have to be exposed into these environments, not only in a training perspective, but have the knowledge and capabilities that eliminates any sort of future hazards in those environments. I think that, to me, is an area that’s absolutely ripe for this technology. And I’m so excited about the future and how we can help people in those environments down the road.