Lexus AreneOS lets you point at objects outside your car and get info

TOKYO — Concept cars range from basically ready-to-produce to total fantasy. Lexus brought two concepts to the Japan Mobility Show that fell somewhere in the middle, with each closer to one end. The LF-ZC was more clearly previewing an upcoming model, whereas the LF-ZL was farther out, somewhere between fantasy and production. But two of its features seem closer than we thought. We already talked about the various simulations of old cars, but the other feature is the "Interactive Reality in Motion" feature of the future AreneOS Lexus is working on. We got to ride in a prototype with the system, and while it's very early in development, it presents some interesting possibilities.

What "Interactive Reality in Motion" basically lets you do is point at things outside the window of the car, and have the car give you information and more. So if you're driving past a restaurant, you can point at it, and ask for information. The car can hypothetically tell you what it is, what the ratings and hours are, and even potentially help you put in a reservation. 

Lexus devs demonstrated these features on a test course set up in a parking lot at a Toyota test center outside Nagoya. And it did generally what Lexus says it will. Whether you're in the driver or passenger seat, you can point out the window, say the wake word or phrase, and it will identify the point of interest.

There are plenty of caveats to this, as Lexus has clearly said, this is a very early prototype. The objects standing in for businesses weren't sandwiched between a bunch of other ones that could be identified instead. We were also going very slowly. And the Lexus folks admitted that a previous group experienced some significant technical difficulties. My group also ran into a couple of more minor hiccups like the sensors getting mixed up with where someone was pointing or getting tripped up by slightly off questions or something being asked too quickly.

With all that said, it still worked pretty well for an early prototype. There has been some good thought put into the interface, too. For instance, the developers said they had very clear reasons for the pointing as the interface. It's obviously intuitive for anyone, and is a good tracking point for the car. But it was also used instead of, say, following your head and eyes for the object you want, since they want you to keep all that pointed at the road ahead while your hand does the moving.

In fact, this system generally is focused on making sure you're less distracted behind the wheel. If you're driving solo, this would be an option for getting information about things and managing something like a reservation instead of fiddling with the infotainment screen or phone.

As impressive as all this is, there are things I have some concerns about. I really think that an issue not just for this, but for all infotainment systems, is making the voice controls much more seamless. Even the best systems can still be a little dimwitted when taking voice controls; taking a long time to respond or not understanding something. The developers noted that they're hoping to take advantage of generative AI to help with understanding questions and phrasing as well as for searching for information.

Other concerns regard accuracy of information and privacy. Lexus wouldn't give information on exactly what sources are being used right now for getting information from the internet to the driver. But regardless of the search engine used, accuracy can be affected simply by a business not really having much of a presence, and there just not being info. And that could be frustrating for owners that don't understand how the system works. There's also the question of user privacy, as this system will have the opportunity to gather huge amounts of data on a driver based on where they are and what they're interested in and engaging with. Plus, this car will clearly be monitoring the outside of the vehicle to identify locations, and where that footage and info goes could be concerning.

All in all, the Interactive Reality in Motion function of AreneOS could be super useful when things are ironed out, and it's clearly on track to be so. But hopefully Lexus is working just as hard on making it secure and respectful of privacy as it is at making it functional..