By now, most smartphone users have experienced some form of augmented reality on their phones, whether they play Pokémon Go, use Snapchat, or actually remember the days of Yelp Monocle (from way back in 2009). But for the people who make the apps, AR isn’t quite as simple, particularly when the apps they’re building involve 3D or “volumetric” objects — stuff you can walk around and interact with through the screen of the phone.
Which is why Apple should start making that process a whole lot easier — especially if it is planning to release AR glasses in the future, ones that will need some sort of “killer app” or group of apps to justify their existence. Other companies, like Facebook, have now taken steps to let developers build AR experiences into their in-app cameras. You might not normally think of Facebook and Apple as direct competitors, but by doing this, Facebook has essentially set the stage for what is going to become an AR platform war on the phone. And Google already has its own AR platform, Tango. Amid the iOS, macOS, and potential Siri speaker reveals at WWDC next week, Apple needs to give due attention to AR.
Augmented reality refers to any kind of computer-generated image that appears on top of a view of the real world. For the sake of this argument, however, let’s focus on smartphones. There are 2D AR apps: ones that do a pretty basic job of slapping a flat, non-dynamic digital image over the stuff you’re seeing through your camera lens. Developers say this is the easiest kind of AR mobile app to build.
True 3D AR apps on mobile require sophisticated engineering right now
Then there are 3D AR apps, which usually require more advanced software development and image processing. Think: Snapchat’s filters like the digital flower crown hovering over your head as you move around in real time, or the virtual furniture you can examine and walk around in home-renovation app Houzz. These may seem like lightweight (or lighthearted) applications, but Snapchat’s lenses require a serious amount of engineering, and the Houzz 3D AR feature took months to build. Another home-renovation platform that offers 3D imaging, Modsy, does most of its image processing after you’ve taken smartphone photos, which can take anywhere from five minutes to several hours. Afterward, it sends the rendered 3D images back to you.