Bringing Lego Fish and Global AR Gnomes to Life, with Trigger Global’s Jason Yim

Step 1: Plant augmented reality gnomes across the world. Step 2: …? Step 3: PROFIT!

Just kidding — Trigger Global’s army of AR gnomes has a more solid business plan than the Underpants Gnomes, as well as many other ventures across popular brands, utilizing mixed reality technologies to bring them to life. CEO Jason Yim emerges from his hidden meadow to talk about a few of them.

Alan: Hi, my name is Alan Smithson, the host of the XR for Business Podcast, and today’s guest is Jason Yim. He is the CEO and executive creative director of Trigger Global, the mixed reality agency. He has creatively led over 150,000 hours of development in mixed reality, including as a Snap Lens Studio partner, preferred developer for Facebook, and showcase developer for Euphoria and Google, as well as an early adopter and early developer for Magic Leap. Yim’s recent high-profile work incorporates mixed reality in marketing for Star Wars: The Last Jedi, product development for Hot Wheels, and location-based experiences such as the fish designer for Lego House, and of course, enterprise tools for AR evaluation tool for Honda. Yim is a recognized speaker around the world and he has held the stage at major technology and industry conferences in Singapore, Shanghai, Berlin, Tokyo, Copenhagen, London, New York, Los Angeles, San Francisco, and all over the place. Jason’s returned to his childhood home to speak at TEDx Hong Kong on computer vision bringing toys to life. Yim has also been featured in Apple’s first TV show, “Planet of the Apps,” and won two LA Auto Show award design challenges back to back, with his partners at Honda Advanced Design. Additionally, Jason has been assigned four patents in augmented and mixed reality, with several more pending. To learn more about Trigger Global, you can visit triggerglobal.com.

Jason, welcome to the show.

Jason: Alan, thanks for that kind introduction.

Alan: It’s amazing, just that introduction; you think “Holy crap, you’ve done work with Honda. You’ve done work with Lego. You’ve done work with Snapchat, and Facebook, and Google.” It’s crazy, the things that you’ve done. And you joined us on stage at AWE this year, to talk about supercharging your marketing. Tell me about some of the things you guys are working at right now.

Jason: Yeah, I think for us on the marketing side it’s actually quite an interesting time. We’re seeing basically the market maturing a little bit and then kind of dividing into two big chunks of work. On the introductory to AR side of things, we have the social lenses. So that’s the Snap/Facebook/Instagram approach, where it’s a small experience for a smaller budget and it’s going through someone else’s app, but it’s a much larger user base, which is a good way to start it off. And then the other group of projects that we work on are kind of larger development, where the brand can own their own app or they have an existing app and we’re pushing an AR module into that existing app.

Alan: Let’s break those into pieces, here. The first one you mentioned is smaller ones with social lenses. Can you maybe talk about some of the work you’ve done in that?

Jason: Sure. We were one of Snap’s first agencies — the Lens Studio partners. We actually were kind of a guinea pig as they were developing the Lens Studio itself. I believe we’re probably one of the first people outside of Snap to actually use the tool. On the client side, we’ve worked with everywhere from Adidas, Pepsi, the NFL, all from sports and brands on the lens side. On the Snap side, typically they are coming to us. We either bring opportunities to Snap where we have clients coming in, and that they’re interested in doing a lens, and then we will connect with a Snap team person as well. Or sometimes Snap brings the opportunities to us, where they may have something creatively or technically a little bit unique, and then they’ll bring us in to collaborate.

Alan: So you guys are literally the guinea pigs here. You’re the ones who like, “Hey, that’s a great idea. How do we do that? We have no idea; let’s call them.”

Jason: Yeah, it’s actually a great time, because you get to innovate every single day. Sometimes they come in with very baked ideas and we just have to figure out how to execute. And then sometimes it’s a little bit more open-ended and we get to concept from scratch.

Alan: So how are these brands — especially on the lens side — how are they measuring success? I can’t imagine it’s that cheap. You said it’s on the lower end scale, but what it would be a minimum engagement? $50,000? Or $20,000?

Jason: I would say lenses are, at the very minimum, it’s probably in the $20k-plus range. We tend not to do very many of those, but we know other people are doing those. And then it will range up from there. Our sweet spot’s probably — for lenses — $40 to $80k, or something like that. More than that, people are probably pushing it into the app space a little bit more.

Alan: Ok, so how are they measuring success with these? Because typical marketers — and for the people listening, mixed reality and augmented reality, they’re really pushing the envelope of the technology, but at the same time, you still need to be able to justify this kind of spend — so how are they measuring that?

Jason: Yeah, I think part of the challenge is sometimes the money is coming from the same budget that might be coming out of a digital media budget, in which case the Snap lens or the Facebook effect is basically being compared against more traditional digital media, like some video buy or a social ad of some kind. I would say right off the bat that it is very difficult to beat video placement or something in terms of just impressions. But where I do think AR wins out, is definitely in engagement, either in session time, the amount of average time being spent with the content, or the amount of shares being done with that content. Also interactions. What are they doing in the Lens itself? Are they clicking through? Are they doing other things? Are they going through the transaction and the purchase? Like I think you can pull deeper interaction through the lens. At this early time, I think that they’re still earned media that you can get, instead of your Snap lens may get some press, and then that press then gets its own number of impressions. Or you create a lens, and you create a video out of that, and then you share that, and then the video of that.

Alan: I’ve been seeing a lot of that.

Jason: Yeah.

Alan: It’s like the LeBron James thing, where LeBron comes out of the poster. I think probably only 10 people in the world ever actually did it, but it got a hundred million impressions from that video. Crazy, the things that are possible.

Jason: We’ve done AR things for some of the big brands that have like seven billion media impressions worldwide. So the numbers can get quite high that way.

Alan: Wow. What one did you do that had that many impressions?

Jason: It was a giant movie IP, which is what we can share, but yeah.

Alan: Oh, okay. Awesome. It’s incredible. So, you talked about the smaller social lenses and that stuff, using Snapchat, Facebook, Instagram. What about the larger builds where you’re building it for a company under their own app, that type of thing? What does that typically look like?

Jason: A lot of it comes from innovation groups, so it’s a different budget. On the smaller lens and effects social AR side, we get a lot of requests via the ad agency, spending some of the media budget. On the app side, it’s typically more direct to brand for us. The goals are usually a little bit deeper, so we would recommend an app or module in an existing app. If you’re trying to go for more of a repeat experience or some sort of deeper technology in the AR experience, like are we tying to real time stats? Are we tying to some big database on the back end or besides just the two minutes of fun sort of experience?

Alan: Interesting. There’s some definite ways that this technology can be used for utilitarian purposes, rather than just entertainment. Is that what you’re talking about there?

Jason: Yes, I think for everyone in the industry, it has to move in that direction. The marketing entertainment side of things will always be there, which is great. But the more basic side of that stuff, I feel, will quickly be a little bit of a race to the bottom. The higher level type of entertainment, like let’s say you have you’re starting to build an AI, you’re starting to push volumetric content, real game engines, things like that, that there’ll still be some premium development there. But I think that lower level simple lens stuff will very quickly become a race to the bottom. For AR to be more widely adopted, we’ll be — again, like you said — like the usability stuff. And it doesn’t have to be the main feature in an app, just like GPS and mapping isn’t necessarily the main feature in a lot of apps that we use, but it is a very useful component. Like if I turn on the Starbucks app and I can find a near Starbucks, I’m not spending my whole time in the map, but it’s a critical piece. AR will soon be that piece as well.

Alan: I agree, I find it’s really interesting that you mention that, because this came up — I think you brought it up, but also some of the other people at the panel in AWE brought it up — basically, building AR for the sake of AR is not what we’re going after, building AR is part of something else. It’s just another tool in the toolbox.

Jason: Yeah, and I think we were getting into some projects now that it’s quite interesting what else we have to tie into, what other backend systems or other technologies that are involved. Even at AWE, when you had the gentleman from Macy’s up, there’s a lot of stats out there right now that are talking about mixed reality and the effectiveness of it. So I do feel that it’s a great place to be, and I think the future is bright, for sure.

Alan: You have on your website here, one of the things I think is just awesome: Travelocity’s roaming gnome. While not utilitarian, probably doesn’t move the needle in terms of inspiring me to travel, but just fun. And it really drives the brand message home. Who came up with that idea?

Jason: They actually came up to me at another conference, it was at a Google conference I was presenting, and they came up to talk about what we could do in AR in their app. I think the interesting challenge for Travelocity was — and here’s another use of AR — with Travelocity, you use the app for booking, of course. Right? But they wanted more engagement after the booking had been done. Like, how do you keep the Travelocity app relevant through your entire trip, until you have to book for your next trip? So that was what AR we were hoping to provide there, was have something fun to do while you’re on the trip, and then also something sharable to help you remember that trip. And then later on you could pick up the Travelocity app again when you want to book your next one.

Alan: So how many gnomes traveled?

Jason: I actually can’t share that information, but yeah…

Alan: Ah-ha. OK, so there’s gnomes everywhere in the world. Augmented gnomes.

Jason: Yes. Everywhere.

Alan: I think step 2 for that is to leave them in physical space. So you leave your gnome, and your gnome is tagged with a message.

Jason: That would be great. Like a big map for everything.

Alan: Find The Gnome. Yeah. Why not? You get a lot in hot spots. Canadians all go to either Cuba or Mexico for the winter. So you’d have these hotspots with all these gnomes everywhere. And they should be able to interact with each other. You find somebody else’s gnome and your gnome starts–

Jason: If you leave them too long, they starve.

Alan: They turn to ceramic.

Jason: Yeah. The interesting part about that, though, was that we were embedding this AR module into the existing Travelocity app. And that’s something that we recommend to a lot of clients that have an existing app with a user base, is to instead of building a separate AR app, which is a lot of lift and you’ll spend a lot of money and a lot of energy trying to drive traffic to a brand new app, just place it within an already successful app. But that, of course, brings its own giant series of complications. We were really excited that Travelocity allowed us to do that because sometimes a lot of clients, if the app that they have is a primary revenue generator, they are super protective of that.

Alan: Oh yeah, they don’t want you to touch that thing at all.

Jason: Yeah.

Alan: We’ve run into that as well. It’s one of those things that you’re like, “Yeah, if you break their app, you’re screwed.”

Jason: Right.

Alan: How do you deal with that? Do you develop it as a separate thing and then give them a plug-in for it, or…?

Jason: We want to start discussing integration at the very beginning of the project. So our tech teams are normally aligned very early, and the other developer that actually owns the main app, they probably have a launch cycle that they are on. So we typically have to dovetail our launch with whatever cadence that they’re in and work backwards from there.

Alan: Interesting. I want to switch from LA — because you mostly work in LA — but you also have an office in… is it Billund or Aarhus?

Jason: Aarhus, in Denmark.

Alan: So you have an office in Denmark, in the middle of Denmark, not in Copenhagen, but in the middle of the furthest island from Copenhagen. So what’s that all about?

Jason: We’ve been working with Lego for six and a half years now, I believe. Lego headquarters is in Billund, which is even more remote than Aarhus. Aarhus is actually the second largest city in Denmark. Billund is about an hour away, Aarhus is like the closest major town. Our lead there was ex-Lego, he had left Lego. He actually worked with us on quite a few projects. He left Lego after 10 years, then came and joined Trigger. So for us, it was part of an effort to grow our Lego business out there. I’ve been a Lego fan since I was a little kid, like most kids are. So that was kind of like a dream come true, to be working with them.

Alan: No kidding.

Jason: We went all in.

Alan: [laughs] Of course. You guys made a fish visualizer, right?

Jason: It’s called the fish designer. It’s actually at the Lego Museum. So, super cool. I believe it’s the best performing digital experience there. Basically, what happens is, as a kid you come into the space and there’s six giant digital tanks. So imagine like a big video screen walls that represent massive fish tanks, and all the fish inside and all the creatures and rocks and everything are built out of Lego, digital Lego. And then as a kid, there are these big stations where you can take physical bricks, and build your own fish. You build physically and then you walk up to these scanners on the edge of giant tanks and you scan in your fish. You add eyes and a mouth. There’s like a little magic moment. And then they come to life in the tank. They get sucked in through the pipe from your scanner and then they get kicked back out into the tank as this live fish with their own little bit of AI. Each tank can hold 300 user created fish. And every minute or so, there’s big animations that happen inside the whole tank, and all the fish interact together and stuff. And the kids basically get into this loop where they go back and just build more fish to stick back into the scanner to bring to life.

Alan: Oh, cool.

Jason: We’re super proud of it, because with Lego we often work from concept through to prototyping, and kid testing, and then final product release. And in this case, we did a lot of earlier prototypes that had a lot more digital interaction for the kid, like after you’ve created the fish you could play a game, you could control it, something like that. It was really nice that the testing actually took us back to keeping it much simpler. So it’s a much more elegant solution that the kids just– they build it physically, which is what they want to do. And then the digital magic conversion moment is very, very short. It’s only like 30 seconds and then they can sit back and enjoy their creation.

Alan: That’s so cool. Yeah, I think we as an industry, we tend to overcomplicate things.

Jason: Yeah, for sure. Sometimes we are over our skis a little bit, right? Like we’re trying to do a lot more than the consumer wants or needs.

Alan: It’s so true. We had a meeting about this a couple of weeks ago. We’re talking about how we did a 360 video three years ago and kind of moved away from it, because it just– it wasn’t hard anymore. We were always looking for the challenge and we realized that the majority of people still haven’t even seen that. Like, “Oh, man. We need to go back to basics.”

Jason: Yeah. I mean, we hear a lot of stats of when museums are doing their first VR exhibits. People come in and for 90 percent of the audience, it’s the first time they put on a headset.

Alan: Crazy, right? It’s so second nature for us that we take it for granted.

Jason: Yeah.

Alan: If there’s one takeaway from this entire podcast, it’s “keep it simple”. You can stretch and push the boundaries of this technology, but keep it simple.

Jason: That’s actually sometimes the hardest piece. How do you reduce the friction in these things? Because consumers are used to just opening up the app and seeing the content on a screen. How do you make spatial 3D content just as simple as that?

Alan: That’s a good question. Let me ask you. You did a TED talk on computer vision, bringing toys to life. Is that talking about the Lego one?

Jason: No, that was more a generality. It was similar. We’ve done 30+ digital/physical play prototypes with Lego and other companies, for kid testing and stuff like that. So we understand that there’s a lot of effort in trying to find the fun and find that elegant ease-of-use balance. That TED talk was about those kind of learnings, but in the toy industry.

Alan: Can you send me a link and I’ll put it in the show notes?

Jason: Sure. Yeah, no problem.

Alan: Amazing. Let’s shift gears away from toys and social lenses. Let’s talk about the things that you did for enterprise tools. You designed an AR design evaluation tool for Honda. Can we talk about that?

Jason: Yeah, so we co-created with Honda. We own the tech IP for the tool. We work with one of their design teams. And what we’re trying to solve is– basically, in the design process for cars right now, you start off on paper, they do 100 drawings or whatever. Then they start working on 3D versions of that. Let’s say you get to like 40 designs. But a critical step in car design is, of course, a volumetric review, like actually seeing the car in 3D space and getting a sense of its presence in a way. And traditionally, that’s done in clay. So the problem with clay is that a life sized clay model takes about eight weeks, and these car companies are spending upwards of $50,000 a month just buying clay.

Alan: Wow.

Jason: Yeah. So, we were tasked to try to come up with a system in AR, to not necessary replace the clay model, but to do an interim step before the clay model. Let’s say we can get the cost down and things like that, so of course, we did. Our version, it can be in a single day instead of eight weeks. The costs are much lower. They can then evaluate 10 cars in AR volumetrically first, before they can then commit to one clay model car. Not only are we saving on costs in time, but I feel like you can end up with a better product because a lot of the design ideas that would have been cut early, because of costs gets to live through another milestone, and continue on to the design process. The big challenges, though, were we started with Hololens, and then it was ported to ARKit, it was ported to Windows Mixed Reality, full backpack solution. And then the later version is Magic Leap, but all of those see-through — besides the ARKit, of course — but see-through headsets on AR, as soon as you take it outside, it’s, you know–

Alan: Useless.

Jason: It’s not the best experience.

Alan: Yeah. It’s funny because I’ve seen some people put shields in front of the Hololenses and stuff, light shields to dim it. It’s not the greatest experience when you can’t really see the holograms. That’s going to be a really hard problem to solve for consumer augmented reality.

Jason: Yeah. And I think this for the car design world, it’s actually specifically a very hard problem to solve, because their legacy approval process happens outdoors that way. So we had to put digital cars next to physical cars, which is how they normally evaluate their designs. That’s why a lot of design studios are actually in California because of–

Alan: Nice weather, all the time.

Jason: Yeah, nice weather all the time. So that’s the interesting thing. Like, maybe the technology drives you to an indoor experience. But like–

Alan: I just tried the Varro — the Varjo or Varro? — headset, the XR one, which is using front facing cameras to capture the outside world. So with that, it’s actually blocking out the real world completely, using cameras to recreate it, and then creating digital content on top.

Jason: That’s cool.

Alan: Yeah. It works really well. I think it would actually be a perfect solution for what you’re doing. And I know they’re working with Volvo.

Jason: That’s cool. What is the resolution of that?

Alan: It’s actually a foveated headset. So what they did was, they did a fixed foveation. In the center, it’s human eye resolution. And then it’s about– it’s a little square, almost an inch of your vision. And then as it goes out to the edges, it blends into a more traditional headset. So it’s very, very clear.

Jason: Wow, amazing. We will definitely figure–

Alan: The headset’s big and bulky, but it doesn’t feel like it when it’s on your head, because they’ve weighted it. They actually put weights in the back of it, to offset the feeling of how heavy it is. They made it heavier to make it feel lighter.

Jason: But I feel like it’s being used in the design process now, like everyone who sees it understands the benefit of it, and everyone understands that the hardware and technology is always going to be improving. So we’re on the right track and we just– with every hardware update — we’ll check out this Varjo for sure — the whole experience will improve and hopefully the results will improve as well.

Alan: I think it’s getting there every step. And then I really love the fact that you guys are right on the edge of consumer applications that are fun and exciting, kids’ applications. But then also you’re building these real world enterprise tools, that companies are using to evaluate and design future automobiles.

Jason: Thanks. I think for brands and developers getting in, I think what we’ve learned through this stuff is you just have to be early, try to be first, try to get as much experience across many industries as possible, because everything cross-pollinates everything else. The stuff our team knows from all this trial and error, I think it shows. But also there’s no other way to learn that experience, besides trying it out.

Alan: Yeah, well, it’s not like you can look it up and say, “Hey, how do I do this?” Because what you’re doing is, for the most part, never been done. I mean, we’ve done, I think, three or four world firsts and– well, four. There was no manual. There was no “Hey, let’s look it up on YouTube and how to do that.” It didn’t exist. So I commend you guys, with 150,000+ hours of development.

Jason: Thank you. That’s no creative hours, too. That’s just 3D in-dev.

Alan: Straight-up dev. Holy moly. Well, they say mastery’s at 10,000 hours. So you guys are 15 times that.

Jason: I think the need to experiment and the need to innovate is really great for the team. Every day you’re doing something different and there’s always something proud to write home about. The problem as a business is if you’re always innovating, how do you–

Alan: How do you make any money?

[laughs]

Jason: You’re burning so much

[money]

.

Alan: People don’t understand. Innovation is expensive. And even though companies are paying you to do it, it’s like you build it once, and then it never gets used again. It’s not like building a product where you build it, and then keep iterating and making it better and better and better. Projects are a different animal, and it’s definitely something that you guys have mastered. So, congratulations.

Jason: Thank you. We are owning our own IP now, so our tech IP. So there are some platforms that we can get to build on top of and resell to different clients and improve over time. So that’s helping us a little bit.

Alan: Love it. Well, let me know. I’m happy to consider some of those products and platforms for the XR Ignite program. So with that, is there anything else you want to leave listeners with? There’s so many different things here to unpack. But is there anything else you wanna leave people with?

Jason: Yeah, I think the sports stuff that we’re doing is really interesting right now. We’re doing kind of AR portals into live games for the NBA, and actually for the PGA Tour as well. So you’re at home watching TV. You can plant a door in AR and then step through it and then you’re courtside at the finals. And then the next level up from that is what we’re doing with the NHL, and soon with another sports league as well, where we’re bringing live telemetry from the game, live stats.

Alan: That’s so badass.

Jason: Yeah. When you’re talk about utility, it’s not a wow factor thing. It’s a– with the NHL, you’re watching the game. And then, let’s say you see a great play. You can actually drop the rink onto your coffee table and see that play, recreate it in AR with all the live data from that play, like how fast the puck was shot and how fast the players were going. All that data represented in AR. So I think that’s super interesting to us.

Alan: That’s badass. Come on, let’s be honest.

Jason: It’s cool to be first. So, that was fun.

Alan: It is always cool to be first. Well, thank you so much, Jason. It’s been an incredible interview. And I’m sure we’re gonna have to do this again, because in six months time, you’ll have 100 more things to talk about.