Living the Networked Life in New York City

Derek Kuhn

Last week, we demonstrated our latest concept car, built on a Porsche 911 Carrera S, at AT&T’s Living the Networked Life Event in New York City. The car has been on the road since we unveiled it at CES — it even went to Spain for MWC. We ran out of time to ship it to New York, so Mark Rigley, manager of the concept development group at QNX, and I drove it from Ottawa to SoHo. It was my first time driving the Porsche anywhere outside of a ball room, and it turned out to be a great drive.

Mark is super protective of the car, as he’s entitled to be, and he and the team have done a great job putting it together. But wouldn’t you know it? He let me drive and, as I’m pulling off Interstate 81 in Cortland NY, we get rear-ended. No — for real. I pull over, Mark jumps out, and I’m checking the side mirror to open the door safely in traffic but I’m mortified. Turns out that the Porsche’s rear valance is strong like a bull, and we emerge with just a series of scuff marks.

We drove through Binghamton, Scranton, and Poconos to get to the city. Traffic got faster and denser as we entered New Jersey, but Mark, to my surprise, let me drive again! We arrived at the hotel and handed the car off to a professional detailer before it was parked on site.

After setting up the car, we were ready. Soon we were joined by press, analysts, and AT&T partners and executives who all came out to see an important part of the car’s story — the use of AT&T’s WATSON speech engine. The Porsche is a great way to illustrate how helpful WATSON can be in an environment like the car.


Derek (seated in the car) with Mazin Gilbert of AT&T and Jordan Crook of TechCrunch.

The event was a big success and featured a presentation from Krish Prabhu, president and CEO of AT&T Labs Research; it also included demonstrations from AT&T Labs and AT&T Foundry. The demos included a smart door that recognizes you before you reach for your keys, and a smartphone application that performs language translations in real time.

The car will be on the road in coming weeks, including a stop at the CTIA Wireless show in New Orleans, from May 8 to 10. If you're there, be sure to check us out in the Solutions Showcase.
 

Techcrunch interview: Talking with (and about) AT&T's WATSON personal assistant

Yesterday, I shared an Engadget video highlighting how the QNX concept car takes advantage of WATSON, AT&T's cloud-based natural-language speech engine. Here's another video, from Jordan Crook of TechCrunch, in which AT&T's Mazin Gilbert describes what WATSON does and QNX's Derek Kuhn demonstrates how WATSON works.

The entire video is interesting, but jump to the 3:30 mark if you want to see the car. The video provides a very nice view of the QNX-equipped dash, btw.



The Engadget video and the TechCrunch video were both shot yesterday at AT&T's "Living the Networked Life" event in New York.

CES press: This just in!

OK well, maybe not 'just in' but these articles and releases were posted yesterday and are quite exciting if you are following QNX technology:

TeleCommunication Systems Supplies Advanced Navigation for QNX CAR 2 Application Platform

TI Takes The Driver’s Seat With An Unrivaled, Full System Solution For Connected Automotive Infotainment

AT&T's Watson coming to cars: won't mind if you ask it if you're there yet

Pimp your ride with augmented reality — Part II

Last week, I introduced you to some cool examples of augmented reality, or AR, and stated that AR can help drivers deal with the burgeoning amount of information in the car.

Now that we’ve covered the basics, let’s look at some use-cases for both drivers and passengers. Remember, though, that these examples are just a taste — the possibilities for integrating AR into the car are virtually endless.


AR for the driver
When it comes to drivers, AR will focus on providing information while reducing distraction. Already, some vehicles use AR to overlay the vehicle trajectory onto a backup camera display, allowing the driver to gauge where the car is headed. Some luxury cars go one step further and overlay lane markings or hazards in the vehicle display.

Expect even more functionality in the future. In the case of a backup camera, the display might take advantage of 3D technology, allowing you to see, for example, that a skateboard is closer than the post you are backing towards. And then there is GM's prototype heads-up system, which, in dark or foggy conditions, can project lane edges onto the windshield or highlight people crossing the road up ahead:



AR can be extremely powerful while keeping distraction to a minimum. Take destination search, for example. You could issue the verbal command, “Take me to a Starbucks on my route. I want to see their cool AR cups”. The nav system could then overlay a subtle route guidance over the road with a small Starbucks logo that gets bigger as you approach your destination. The logo could then hover over the building when you arrive.

You'll no longer have to wonder if your destination is on the right or left, or if your nav system is correct when it says, “You have arrived at your destination.” The answer will be right in front of you.

AR for the passenger
So what about the passenger? Well, you could easily apply AR to side windows and allow passengers to learn more about the world around them, a la Wikitude. Take, for example, this recent video from Toyota, which represents one of the best examples of how AR could make long road trips less tedious and more enjoyable:


 

Building a hands-free future

The end of my street is governed by a three-way stop. The other morning I was backing out of my driveway when someone rolled past the stop sign and came within inches of hitting me. I stopped, glared at him, and resumed driving. Two stop signs later, the same guy squeezed past my car (in the same lane), completely oblivious to what he was doing.

Why was he driving like this? Probably because he was deeply engrossed in a conversation on his cell phone.

Where I live, using a handset while driving has been illegal for over a year. You cannot talk, you cannot text, you cannot “Facebook”, you cannot Tweet — even if you're stopped at a red light. This makes perfect sense to me. As a driver, your primary responsibility is to control the vehicle. And yet I see people texting on the freeway, talking on their cell phones, and doing who knows what else on an alarmingly regular basis.

The QNX-powered BMW
ConnectedDrive system

Society has become obsessed with mobile devices, and it will take more than legislation to change its behavior. The answer, I think, is to embrace the behavior in a way that makes it possible to interact socially while maintaining control of the car. We’ve seen great progress in hands-free/phone integration, and BMW ConnectedDrive offers an example of how drivers can access email and other smartphone services more safely.

This is the tip of the iceberg. Integrating the handset with the infotainment unit is going to change the way you interact with your car. Intelligently designed apps, combined with multi-modal human machine interfaces, will let you Tweet or update Facebook using speech recognition, keeping your eyes on the road.

Without taking your hands of the wheel, you’ll be able to call a friend and decide that you want to go to dinner, do a local search to find out what’s available, check a restaurant review on Yelp, make a reservation, text your friend back with the time and place, and aim your navigation system at the restaurant. And you’ll be able to do it using natural language. None of this “please say a name” stuff.

Seems futuristic? It’s not. People are working on it today. In fact, QNX-based systems, such as Toyota Entune, already offer a taste of this hands-free and highly personalized future.
 

Speech interfaces: UI revolution or intelligent evolution?

Speech interfaces have received a lot of attention recently, especially with the marketing blitz for Siri, the new speech interface for the iPhone.

After watching some of the TV commercials you might conclude that you can simply talk to your phone as if it were your friend, and it will figure out what you want. For example, in one scenario the actor asks the phone, “Do I need a raincoat?”, and the phone responds with weather information.

A colleague commented that if he wanted weather information he would just ask for it. As in “What is the weather going to be like in Seattle?” or “Is it going to rain in Seattle?”.

Without more conversational context, if a friend were to ask me, “Do I need a raincoat?”, I would probably respond, “I don’t know, do you?” — jokingly, of course.

Evo or revo?

Are we ready to converse
with our phones and cars?

Kidding aside, systems like Siri raise an important question: Are we about to see a paradigm shift in user interfaces?

Possibly. But I think it will be more of a UI evolution than a UI revolution. In other words, speech interfaces will play a bigger role in UI designs, but that doesn't mean you're about to start talking to your phone — or any other device — as if it’s your best friend.

Currently, speech interfaces are underutilized. The reasons for this aren't yet clear, though they seem to encompass both technical and user issues. Traditionally, speech recognition accuracy rates have been less than perfect. Poor user interface design (for instance, reprompting strategies) has contributed to the overall problem and to increased user frustration.

Also, people simply aren't used to speech interfaces. For example, many phones support voice-dialing, yet most people don't use this feature. And user interface designers seem reluctant to leverage speech interfaces, possibly because of the additional cost and complexity, lack of awareness, or some other reason.


"Relying heavily on speech can lead
to a suboptimal user experience..."
As a further complication, relying heavily on speech as an interface can lead to a suboptimal user experience. Speech interfaces pose some real challenges, including recognition accuracy rates, natural language understanding, error recovery dialogs, UI design, and testing. They aren't the flawless wonders that some marketers would lead you to believe.

Still, I believe there is a happy medium for leveraging speech interfaces as part of a multi-modal interface — one that uses speech as an interface where it makes sense. Some tasks are better suited for a speech interface, while others are not. For example, speech provides an ideal way to provide input to an application when you can capitalize on information stored in the user’s head. But it’s much less successful when dealing with large lists of unfamiliar items.

Talkin' to your ride
Other factors, besides Apple, are driving the growing role of speech interfaces — particularly in automotive. Speech interfaces can, for example, help address the issue of driver distraction. They allow drivers to keep their “eyes on the road and hands on the wheel,” to quote an oft-used phrase.

So, will we see a paradigm shift towards speech interfaces? It's unlikely. I'm hoping, though, that we'll see a UI evolution that makes better use of them.

Think of it more as a paradigm nudge than a paradigm shift.

---

Recommended reading

Situation Awareness: a Holistic Approach to the Driver Distraction Problem
Wideband Speech Communications for Automotive: the Good, the Bad, and the Ugly