You know the trope in Sherlock-esque detective shows where some brilliant sleuth cracks a case by drawing on their nigh-photographic memory? Well, one of the most fascinating and terrifying things about augmented reality glasses is that they turn everybody into that sleuth — and as a new Microsoft patent filing shows, that can be used for things besides crime-solving. The recently published application covers a system that would let its HoloLens glasses track small items like car keys, stopping wearers from misplacing them. More broadly, the patent describes a system that can monitor the status of objects without any instructions from users, keeping tabs on anything that’s important to their lives.
The patent’s basic idea is pretty simple. HoloLens has outward-facing cameras that can make a spatial map of a room, and machine vision technology can identify or track specific objects in an image. So if, for example, you put your keys down on a table, HoloLens could hypothetically spot them through the camera and quietly note their position. When you’re about to leave the house, it could give you the keys’ last known location, even if they’ve since been covered up by a newspaper or slipped under a couch cushion.
By the time this makes sense, we might not even have keys
This seems like a pretty inefficient way to find your keys today, or possibly ever. For one thing, you’d need to be consistently wearing smart glasses, which is currently staggeringly inconvenient. If you drop your keys somewhere without looking at them, the system might not register that they’ve moved. It’s not clear how much it would help if you slipped something in a coat pocket or bag and then carried it around, unless it’s got some very complex multi-location tracking. It’s not hard to slap a Bluetooth tracker on an item if you’re consistently losing it. And by the time we get to a future of 24/7 augmented reality, we may not have keys at all.
But what’s really interesting isn’t the idea of HoloLens tracking an object. It’s HoloLens learning what items matter to you and choosing what to follow, before you ever worry about losing something. To be clear, you could designate objects: one example has a traveler telling HoloLens to track their passport while abroad. In other cases, though, it could check to see how often you interact with an object, or when you move it around, and start tracking anything that hits a certain threshold.