Video: The secret to making hands-free noise-free

 

Explaining a highly technical product to a broad audience is tough. To succeed, you must reach out to people on their own terms, without being condescending. Most people love a good explanation, but everyone hates being talked down to.

Case in point: The QNX Acoustic Processing Suite. This software runs in millions of cars and offers a benefit that everyone can relate to: clear, rich, easy-to-understand hands-free calls. But once you start explaining how the suite does this, it's easy to get mired in technical jargon and to forget the bigger picture — something that even a technical audience wants to see.

So we dropped the jargon and opted for a creative approach. It involves a marching band, a rock guitarist, and, for good measure, an electric fan with a really long extension cord. Seriously.

Intrigued yet? Well, then, grab some popcorn and dim the lights:




Interested in learning more about this technology? Check out the acoustic processing page on the QNX website.

BTW, companies that use the QNX Acoustic Processing Suite in their products include OnStar, whose FMV aftermarket mirror recently won a CES Innovations Design and Engineering Award.

Posted by Paul Leroux
 

Beyond the dashboard: discover how QNX touches your everyday life

QNX technology is in cars — lots of them. But it’s also in everything from planes and trains to smart phones, smart buildings, and smart vacuum cleaners. If you're interested, I happen to have an infographic handy...

I was a lost and lonely soul. Friends would cut phone calls short, strangers would move away from me on the bus, and acquaintances at cocktail parties would excuse themselves, promising to come right back — they never came back. I was in denial for a long time, but slowly and painfully, I came to the realization that I had to take ownership of this problem. Because it was my fault.

To by specific, it was my motor mouth. Whenever someone asked what I did for a living, I’d say I worked for QNX. That, of course, wasn’t a problem. But when they asked what QNX did, I would hold forth on microkernel OS architectures, user-space device drivers, resource manager frameworks, and graphical composition managers, not to mention asynchronous messaging, priority inheritance, and time partitioning. After all, who doesn't want to learn more about time partitioning?

Well, as I subsequently learned, there’s a time and place for everything. And while my passion about QNX technology was well-placed, my timing was lousy. People weren’t asking for a deep dive; they just wanted to understand QNX’s role in the scheme of things.

As it turns out, QNX plays a huge role, and in very many things. I’ve been working at QNX Software Systems for 25 years, and I am still gobsmacked by the sheer variety of uses that QNX technology is put to. I'm especially impressed by the crossover effect. For instance, what we learn in nuclear plants helps us offer a better OS for safety systems in cars. And what we learn in smartphones makes us a better platform supplier for companies building infotainment systems.

All of which to say, the next time someone asks me what QNX does, I will avoid the deep dive and show them this infographic instead. Of course, if they subsequently ask *how* QNX does all this, I will have a well-practiced answer. :-)

Did I mention? You can download a high-res JPEG of this infographic from our Flickr account and a PDF version from the QNX website.



Stay tuned for 2015 CES, where we will introduce even more ways QNX can make a difference, especially in how people design and drive cars.

And lest I forget, special thanks to my colleague Varghese at BlackBerry India for conceiving this infographic, and for the QNX employees who provided their invaluable input.

Building (sound) character into cars

Tina Jeffrey

Modern engines are overachievers when it comes to fuel efficiency — but they often score a C minus in the sound department. Introducing a solution that can make a subtle but effective difference.

Car engines don’t sound like they used to. Correction: They don’t sound as good as they used to. And for that, you can blame modern fuel-saving techniques, such as the practice of deactivating cylinders when engine load is light. Still, if you’re an automaker, delivering an optimal engine sound is critical to ensuring a satisfying user experience. To address this need, we’ve released QNX Acoustics for Engine Sound Enhancement (ESE), a complementary technology to our solution for active noise control.

The why

We first demonstrated our ESE technology at 2014 CES
in the QNX technology concept car for acoustics.

Many people assume, erroneously, that ESE is about giving cars an outsized sonic personality — such as making a Smart ForTwo snarl like an SRT Hellcat. While that is certainly possible, most automakers will use ESE to augment engine sounds in subtle but effective ways that bolster the emotional connection between car and driver — just like engine sounds did in the past. It boils down to creating a compelling acoustic experience for drivers and passengers alike.

ESE isn’t new. Traditionally, automakers have used mechanical solutions that modify the design of the exhaust system or intake pipes to differentiate the sound of their vehicles. Today, automakers are shifting to software-based ESE, which costs less and does a better job at augmenting engine sounds that have been degraded by new, efficient engine designs. With QNX Acoustics for Engine Sound Enhancement, automakers can accurately preserve an existing engine sound for use in a new model, craft a unique sound to market a new brand, or offer distinct sounds associated with different transmission modes, such as sport or economy.

The how
QNX Acoustics for Engine Sound Enhancement is entirely software based. It comprises a runtime library that augments naturally transmitted engine sounds as well as a design tool that provides several advanced features for defining and tuning engine-sound profiles. The library runs on the infotainment system or on the audio system DSP and plays synthesized sound synchronized to the engine’s real-time data: RPM, speed, throttle position, transmission mode, etc.




The ESE designer tool enables sound designers to create, refashion, and audition sounds directly on their desktops by graphically defining the mapping between a synthesized engine-sound profile and real-time engine parameters. The tool supports both granular and additive synthesis, along with a variety of digital signal processing techniques to configure the audio path, including gain, filter, and static equalization control.



The value
QNX Acoustics for Engine Sound Enhancement offers automakers numerous benefits in the design of sound experiences that best reflect their brand:

• Ability to design consistent powertrain sounds across the full engine operating range
   
• Small footprint runtime library that can be ported to virtually any DSP or CPU running Linux or the QNX OS, making it easy to customize all vehicle models and to leverage work done in existing models
   
• Tight integration with other QNX acoustics middleware libraries, including QNX Acoustics for Active Noise Control, enabling automakers to holistically shape their interior vehicle soundscape
   
• Dedicated acoustic engineers that can support development and pre-production activities, including porting to customer-specific hardware, system audio path verification, and platform and vehicle acoustic tuning
   

If you’re with an automaker or Tier One and would like to discuss how QNX Acoustics for ESE can address your project requirements, I invite you to contact us at anc@qnx.com.

In the meantime, learn more about this solution on the QNX website.

New release of QNX acoustic processing suite means less noise, less tuning for hands-free systems

Paul Leroux

This just in: QNX has released version 2.0 of its acoustic processing suite, a modular software library designed to maximize the quality and clarity of automotive hands-free systems.

The suite, used by 18 automakers on over 100 vehicle platforms, provides modules for both the receive side and the send side of hands-free calls. The modules include acoustic echo cancellation, noise reduction, wind blocking, dynamic parametric equalization, bandwidth extension, high frequency encoding, and many others. Together, they enable high-quality voice communication, even in a noisy automotive interior.

Highlights of version 2.0 include:

Enhanced noise reduction — Minimizes audio distortions and significantly improves call clarity. Can also reconstruct speech masked by low-frequency road and engine noise.

Automatic delay calculation and compensation — Eliminates almost all product tuning, enabling automakers to save significant deployment time and expense.

Off-Axis noise rejection — Rejects sound not directly in front of a microphone or speaker, allowing dual-microphone solutions to hone in on the person speaking for greater intelligibility.

To read the press release, click here. To learn more about the acoustic processing suite, visit the QNX website.


The QNX Aviage Acoustic Processing Suite can run on the general purpose processor,
saving the cost of a DSP.

 

Wanted: Haunted Vehicles

Halloween is just around the corner, and that reminds me of the haunted room at Lucent Bell Labs. Mind you, it wasn’t really haunted. But for a moment, I was convinced.

Let me explain. As I entered the room, I could hear two of my colleagues talking to each other, and by the sound of their voices, they were both sitting right in front of me. But when I looked, I could see only one person. Creepy, to say the least.

It took a few seconds, but I finally realized what was happening: The other colleague was in a different room, talking over a perfectly tuned prototype of a conference phone. The sense of presence was so real that I couldn’t help but feel we were all in the same room — even after I became aware of the “trick” being played!

It was then that I realized it: We don’t know what we’re missing until we experience it.

Making it real
Current telephone calls don’t sound like face-to-face conversations because the telephone network and terminals band-limit speech from about 50-10000 Hz down to 300-3400 Hz. To make matters worse, the phone’s single channel of audio eliminates spatial information about the sound source. As a result, we perceive most sounds as coming from the same point in space.

But here's the thing: The historical reasons for transmitting these single-channel narrowband speech signals no longer apply. Current technologies — such as wideband speech coders, spatial audio, and VoIP — are enabling speech communications with wider bandwidth speech and greater spatial information.

Many in the industry refer to these next-generation telecommunications systems as telepresence systems. “Telepresence” refers to the degree of realism created by a telecommunications system. Traditional systems have low telepresence while newer systems that use wider bandwidth speech and spatial audio have high telepresence.

Some people believe that a visual display is a must-have for a telepresence system. In reality, a display can decrease telepresence if its quality is poor. Experience shows that an audio-only system can have such high telepresence that people can't distinguish it from face-to-face communications — witness my haunting experience at Lucent Bell Labs.

Until recently, widespread deployment of telepresence systems has hit a roadblock: lack of standardization. Fortunately, the IETF CLUE Working Group and ITU-T Study Groups 16 and 12 are actively developing standards to remedy this situation.

Pimp my ride with telepresence
Telepresence systems have a lot to offer in an automotive environment. For instance, they could:

• reduce driver distraction
• make it easier to understand speech in the presence of vehicle noise
• reduce the fatigue that comes from trying to understand a degraded voice signal

Moreover, a telepresence system makes the talker on the far end of the phone connection sound more like they are in the vehicle; it also makes the talker easier to identify.

Successful deployment of telepresence in an automotive environment depends on several factors:

• attention to the design of vehicle platforms
• use of high-performance acoustic processing algorithms (AEC, NR, etc.), such as those provided by the QNX acoustic processing suite
• the ability to transport telepresence signals between telephony terminals — this is being enabled by increased VoIP availability (via LTE, for instance)

I don't know about you, but I'm looking forward to the day when my vehicle is haunted like that lab in New Jersey!

For additional reading on this topic, download the whitepaper, "Wideband Speech Communications for Automotive: the Good, the Bad, and the Ugly".

 

Attending SAE Convergence? Here’s why you should visit booth 513

Cars and beer don’t mix. But discussing cars while having a beer? Now you’re talking. If you’re attending SAE Convergence next week, you owe it to yourself to register for our “Spirits And Eats” event at 7:00 pm Tuesday. It’s the perfect occasion to kick back and enjoy the company of people who, like yourself, are passionate about cars and car electronics. And it isn’t a bad networking opportunity either — you’ll meet folks from a variety of automakers, Tier 1s, and technology suppliers in a relaxed, convivial atmosphere.

But you know what? It isn’t just about the beer. Or the company. It’s also about the Benz. Our digitally modded Mercedes-Benz CLA45 AMG, to be exact. It’s the latest QNX technology concept car, and it’s the perfect vehicle (pun fully intended) for demonstrating how QNX technology can enable next-generation infotainment systems. Highlights include:

• A multi-modal user experience that blends touch, voice, and physical controls
• A secure application environment for Android, HTML5, and OpenGL ES
• Smartphone connectivity options for projecting smartphone apps onto the head unit
• A dynamically reconfigurable digital instrument cluster that displays turn-by-turn directions, notifications of incoming phone calls, and video from front and rear cameras
• Multimedia framework for playback of content from USB sticks, DLNA devices, etc.
• Full-band stereo calling — think phone calls with CD quality audio
• Engine sound enhancement that synchronizes synthesized engine sounds with engine RPM

Here, for example, is the digital cluster:



And here is a closeup of the head unit:



And here’s a shot of the cluster and head unit together:



As for the engine sound enhancement and high-quality hands-free audio, I can’t reproduce these here — you’ll have come see the car and experience them first hand. (Yup, that's an invite.)

If you like what you see, and are interested in what you can hear, visit us at booth #513. And if you'd like to schedule a demo or reserve some time with a QNX representative in advance, we can accommodate that, too. Just send us an email.

A sound approach to creating a quieter ride

Tina Jeffrey

Add sound to reduce noise levels inside the car. Yup, you read that right. And while it may seem counterintuitive, it’s precisely what automakers are doing to provide a better in-car experience. Let’s be clear: I’m not talking about playing a video of SpongeBob SquarePants on the rear-seat entertainment system to keep noisy kids quiet — although I can personally attest to the effectiveness of this method. Rather, I’m referring to deliberately synthesized sound played over a vehicle’s car speakers to cancel unwanted low-frequency engine tones in the passenger compartment, yielding a quieter and more pleasant ride.

So why is this even needed? It comes down to fuel economy. Automakers are continually looking at ways to reduce fuel consumption through techniques such as variable cylinder management (reducing the number of cylinders in operation under light engine load) and operating the engine at lower RPM. Some automakers are even cutting back on passive damping materials to decrease vehicle weight. These approaches do indeed reduce consumption, but they also result in more engine noise permeating the vehicle cabin, creating a noisier ride for occupants. To address the problem, noise vibration and harshness engineers (OEM engineers responsible for characterizing and improving sound quality in vehicles) are using innovative sound technologies such as active noise control (ANC).

Automotive ANC technology is analogous to the technology used in noise-cancelling headphones but is more difficult to implement, as developers must optimize the system based on the unique acoustic characteristics of the cabin interior. An ANC system must be able to function alongside a variety of other audio processing tasks such as audio playback, voice recognition, and hands-free communication.


The QNX Acoustics for Active Noise Control solution uses realtime engine data and sampled microphone data from the cabin to construct the “anti-noise” signal played over the car speakers.

So how does ANC work?
According to the principle of superposition, sound waves will travel and reflect off glass, the dash, and other surfaces inside the car; interfere with each other; and yield a resultant wave of greater or lower amplitude to the original wave. The result varies according to where in the passenger compartment the signal is measured. At some locations, the waves will “add” (constructive interference); at other locations, the waves will “subtract” or cancel each other (destructive interference). Systems must be tuned and calibrated to ensure optimal performance at driver and passenger listening positions (aka “sweet spots”).

To reduce offending low-frequency engine tones (typically <150 Hz), an ANC system typically requires real-time engine data (including RPM) in addition to signals from the cabin microphones. The ANC system then synthesizes and emits “anti-noise” signals that are directly proportional but inverted to the original offending engine tones, via the car’s speakers. The net effect is a reduction of the offending tones.


According to the superposition principle of sound waves, a noise signal and an anti-noise signal will cancel each other if the signals are 180 degrees out of phase. Image adapted from Wikipedia.

Achieving optimal performance for these in-vehicle systems is complex, and here’s why. First off, there are multiple sources of sound inside a car — some desirable and some not. These include the infotainment system, conversation between vehicle occupants, the engine, road, wind, and structural vibrations from air intake valves or the exhaust. Also, every car interior has unique acoustic characteristics. The location and position of seats; the position, number, and type of speakers and microphones; and the materials used inside the cabin all play a role in how an ANC system performs.

To be truly effective, an ANC solution must adapt quickly to changes in vehicle cabin acoustics that result from changes in acceleration and deceleration, windows opening and closing, changes in passenger seat positions, and temperature changes. The solution must also be robust; it shouldn’t become unstable or degrade the audio quality inside the cabin should, for example, a microphone stop working.

The solution for every vehicle model must be calibrated and tuned to achieve optimal performance. Besides the vehicle model, engine noise characteristics, and number and arrangement of speakers and microphones, the embedded platform being used also plays a role when tuning the system. System tuning can, with conventional solutions, take months to reach optimal performance levels. Consequently, solutions that ease and accelerate the tuning process, and that integrate seamlessly into a customer’s application, are highly desirable.

Automotive ANC solutions — then and now
Most existing ANC systems for engine noise require a dedicated hardware control module. But automakers are beginning to realize that it’s more cost effective to integrate ANC into existing vehicle hardware systems, such as the infotainment head unit. This level of integration facilitates cooperation between different audio processing tasks, such as managing a hands-free call and reducing noise in the cabin.

Earlier today, QNX announced the availability of a brand new software product that targets ANC for engine tone reduction in passenger vehicles. It’s a flexible, software-based solution that can be ported to floating or fixed-point DSPs or application processors, including ARM, SHARC, and x86, and it supports systems with or without an OS. A host application that executes on the vehicle’s head unit or audio amplifier manages ANC through the library’s API calls. As a result, the host application can fully integrate ANC functionality with its other audio tasks and control the entire acoustic processing chain.

Eliminating BOM costs
The upshot is that the QNX ANC solution can match or supersede the performance of a dedicated hardware module — and we have the benchmarks to show it. Let me leave you with some of the highlights of the QNX Acoustics for Active Noise Control solution:

• Significantly better performance than dedicated hardware solutions — The QNX solution can provide up to 9dB of reduction at the driver’s head position compared to 5dB for a comparative hardware solution in the same vehicle under the same conditions.
   
• Significant BOM cost savings — Eliminates the cost of a dedicated hardware module.
   
• Flexible and configurable — Can be integrated into the application processor or DSP of an existing infotainment system or audio amplifier, and can run on systems with or without an OS, giving automakers implementation choices. Also supports up to 6 microphone and 6 speaker-channel configurations.
   
• Faster time to market — Speeds development by shortening tuning efforts from many months to weeks. Also, a specialized team of QNX acoustic engineers can provide software support, consulting, calibration, and system tuning.

For the full skinny on QNX Acoustics for Active Noise Control, visit the QNX website.

A question of concurrency

The first of a new series on the QNX CAR Platform for Infotainment. In this installment, I tackle the a priori question: why does the auto industry need this platform, anyway?

Define your terms, counseled Voltaire, and in keeping with his advice, allow me to begin with the following:

Con•cur•ren•cy \kən-kûr'-ən-sē\ n (1597) Cooperation, as of agents, circumstances, or events; agreement or union in action.

A good definition, as far as it goes. But it doesn’t go far enough for the purposes of this discussion. Wikipedia comes closer to the mark:

“In computer science, concurrency is a property of systems in which several computations execute simultaneously, and potentially interact with each other.”

That’s better, but it still falls short. However, the Wikipedia entry also states that:

“the base goals of concurrent programming include correctness, performance and robustness. Concurrent systems… are generally designed to operate indefinitely, including automatic recovery from failure, and not terminate unexpectedly.”

Now that’s more like it. Concurrency in computer systems isn’t simply a matter of doing several things all at once; it’s also a matter of delivering a solid user experience. The system must always be available and it must always be responsive: no “surprises” allowed.

This definition seems tailored-made for in-car infotainment systems. Here, for example, are some of the tasks that an infotainment system may perform:

• Run a variety of user applications, from 3D navigation to Internet radio, based on a mix of technologies, including Qt, HTML5, Android, and OpenGL ES
• Manage multiple forms of input: voice, touch, physical buttons, etc. 
• Support multiple smartphone connectivity protocols such as MirrorLink and Apple CarPlay 
• Perform services that smartphones cannot support, including:
  
  • HVAC control
  • discovery and playback of multimedia from USB sticks, DLNA devices, MTP devices, and other sources
  • retrieval and display of fuel levels, tire pressure, and other vehicle information
  • connectivity to Bluetooth devices
• Process voice signals to ensure the best possible quality of phone-based hands-free systems — this in itself can involve many tasks, including echo and noise removal, dynamic noise shaping, speech enhancement, etc. 
• Perform active noise control to eliminate unwanted engine “boom” noise 
• Offer extremely fast bootup times; a backup camera, for example, must come up within a second or two to be useful
   

Jugging multiple concurrent tasks

The primary user of an infotainment system is the driver. So, despite juggling all these activities, an infotainment system must never show the strain. It must always respond quickly to user input and critical events, even when many activities compete for system resources. Otherwise, the driver will become annoyed or, worse, distracted. The passengers won’t be happy, either.

Still, that isn’t enough. Automakers also need to differentiate themselves, and infotainment serves as a key tool for achieving differentiation. So the infotainment system must not simply perform well; it must also allow the vehicle, or line of vehicles, to project the unique values, features, and brand identity of the automaker.

And even that isn’t enough. Most automakers offer multiple vehicle lines, each encompassing a variety of configurations and trim levels. So an infotainment design must also be scalable; that way, the work and investment made at the high end can be leveraged in mid-range and economy models. Because ROI.

Projecting a unique identity

But you know what? That still isn’t enough. An infotainment system design must also be flexible. It must, for example, support new functionality through software updates, whether such updates are installed through a storage device or over the air. And it must have the ability to accommodate quickly evolving connectivity protocols, app environments, and hardware platforms. All with the least possible fuss.

The nitty and the gritty
Concurrency, performance, reliability, differentiation, scalability, flexibility — a tall order. But it’s exactly the order that the QNX CAR Platform for Infotainment was designed to fill.

Take, for example, product differentiation. If you look at the QNX-powered infotainment systems that automakers are shipping today, one thing becomes obvious: they aren’t cookie-cutter systems. Rather, they each project the unique values, features, and brand identity of each automaker — even though they are all built on the same, standards-based platform.

So how does the QNX CAR Platform enable all this? That’s exactly what my colleagues and I will explore over the coming weeks and months. We’ll get into the nitty and sometimes the gritty of how the platform works and why it offers so much value to companies that develop infotainment systems in various shapes, forms, and price points.

Stay tuned.

POSTSCRIPT: Read the next installment of the QNX CAR Platform series, A question of architecture.

QNX Acoustics for Voice — a new name and a new benchmark in acoustic processing

Tina Jeffrey

Earlier this month, QNX Software Systems officially released QNX Acoustics for Voice 3.0 — the company’s latest generation of acoustic processing software for automotive hands-free voice communications. The solution sets a new benchmark in hands-free quality and supports the rigorous requirements of smartphone connectivity specifications.

Designed as a complete software solution, the product includes both the QNX Acoustics for Voice signal-processing library and the QWALive tool for tuning and configuration.

The signal-processing library manages the flow of audio during a hands-free voice call. It defines two paths: the send path, which handles audio flowing from the microphones to the far end of the call, and the receive path, which handles audio flowing from the far end to the loudspeakers in the car:





QWALive, used throughout development and pre-production phases, gives developers realtime control over all library parameters to accelerate tuning and diagnosis of audio issues:



A look under the hood
QNX Acoustics for Voice 3.0 builds on QNX Software Systems’ best-in-class acoustic echo cancellation and noise reduction algorithms, road-proven in tens of millions of cars, and offers breakthrough advancements over existing solutions.

Let me run through some of the innovative features that are already making waves (sorry, couldn’t resist) among automotive developers.

Perhaps the most significant innovation is our high efficiency technology. Why? Well, simply put, it saves up to 30% both in CPU load and in memory requirements for wideband (16 kHz sample rate for HD Voice) and Wideband Plus (24 kHz sample rate). This translates into the ability to do more processing on existing hardware, and with less memory. For instance, automakers can enable new smartphone connectivity capabilities on current hardware, without compromising performance:



Another feature that premieres with this release is intelligent voice optimization technology, designed to accelerate and increase the robustness of send-path tuning. This technology implements an automated frequency response correction model that dynamically adjusts the frequency response of the send path to compensate for variations in the acoustic path and vehicle cabin conditions.

Dynamic noise shaping, which is exclusive to QNX Acoustics for Voice, also debuts in this release. It enhances speech quality in the send path by reducing broadband noise from fans, defrost vents, and HVAC systems — a welcome feature, as broadband noise can be particularly difficult for hands-free systems to contend with.

Flexibility and portability — check and check
Like its predecessor (QNX Aviage Acoustic Processing 2.0), QNX Acoustics for Voice 3.0 continues to offer maximum flexibility to automakers. The modular software library comes with a comprehensive API, easing integration efforts into infotainment, telematics, and audio amplifier modules. Developers can choose from fixed- and floating-point versions that can be ported to a variety of operating systems and deployed on a wide range of processors or DSPs.

We’re excited about this release as it’s the most sophisticated acoustic voice processing solution available to date, and it allows automakers to build and hone systems for a variety of speech requirements, across all their vehicle platforms.

Check out the QNX Acoustics for Voice product page to learn more.

QNX Acoustics for Active Noise Control wins a Silver Stevie

Lynn Gayowski

The winners of the 11th annual International Business Awards have been announced and I'm happy to share that QNX Acoustics for Active Noise Control (ANC) has won a Silver Stevie Award in the software category, for Best New Product or Service of the Year! The awards program honours the achievements of organizations and working professionals worldwide, and received more than 3,500 nominations this year from dozens of countries. It feels great to be chosen as a winner among so many entries.
If you're unfamiliar with QNX Acoustics for ANC, it's a software solution that can dramatically reduce unwanted engine harmonic noise inside the cabin of a vehicle. The software's algorithms for noise cancellation can run on an existing CPU or DSP in the infotainment system, eliminating the need for dedicated hardware controller modules. The end result is significant savings for automakers and a quieter ride for drivers and passengers.
This is the second award for QNX Acoustics for ANC, after a win in February at the Embedded World conference's embedded AWARDs. If you want to learn more about the solution, read our white paper titled A Software-Based Approach to Active Noise Control in Automobiles. Congratulations QNX!

Am I crazy for talking to my car?

Earlier this afternoon, I participated in a connected car panel at SpeechTEK 2012, hosted by our friend Mazin Gilbert from AT&T. The other panelists included Greg Bielby of VoltDelta, Thomas Schalk of Agero, and Hakan Kostepen of Panasonic.

Even though Mazin did a fantastic job, not every panelist had a chance to answer every question. I was itching to answer some, so here are my responses to the questions that I didn't get to answer, or where I feel I could have provided a more complete response.

Have speech technologies matured to the point where they can be used robustly in the car? The general answer to this question from the panel was yes, but I think the real answer is a qualified yes. The technologies exist, but often aren't applied or may need auto-specific adaptations to handle in-cabin noise or other issues. Natural language recognition was an oft-stated driving technology, but a missing piece to the puzzle is hybrid recognition. I don't mean pushing recognition wholesale to the cloud, like Siri does. I mean a true split of the recognition effort, where each part does what it’s best at. Put the front half of acoustic processing in the vehicle to clean up the audio and convert the waveform to frequency-domain data, then send the data to the cloud-based server. The cloud server can then parse and interpret the data, and send back the result.

Hybrid speech rec solves three problems at once: better audio signals (the car can improve audio specific to the in-cabin environment), better cost (frequency data is far more compressed than raw audio, so you pay less for data transfer), and better responsiveness (hybrid rec gives the server time to start working on the response while it's coming in instead of waiting for the whole utterance to finish before starting).

Is driver distraction a major business driver, or is it the "Siri effect"? Currently, the car industry seems to use driver distraction as a reason to push a lot of features into speech. Many of those uses are gimmicky. Personally, I don't care if I can set my climate control system with voice — why would I when I can simply turn a dial? I once had someone ask me about the feasibility of adding voice recognition commands for rolling down the windows. I asked him, "Yes, but wouldn't people just push the window button?"

We shouldn’t implement speech commands just because we can. They may have contributed to excitement in the early adopter crowd, but we're beyond that now. Mind you, there are some seriously useful ways to use voice. For instance, any time you need to pick from a huge number of choices, voice recognition is the natural way to go. Calling contacts ("Call Sarah Potter"), entering destinations ("Go to 3121 South Park Street"), or picking music ("Play Audioslave") are all much easier than using an HMI to enter the same information, and safer to boot. It just has to work consistently and accurately.

Will car makers see more speech moving to the cloud, or will it be a hybrid of cloud and embedded? I disagree with the majority of the panel on this one, and, I think, the majority of people in the industry. Most auto people believe a hybrid between embedded and cloud allows the best of both worlds — good recognition and updatability when connected, and consistent reliability when not. My colleague Andrew Poliak also champions this view with a memorable catch phrase: Zombie Apocalypse. That is, you still want the system to work, albeit partially, when the infrastructure isn't available.

But if you ask me, everyone is missing the point — theirs is a technology-centric point of view. Everyday customer acceptance of a particular technology is notoriously harsh: if it doesn't work well, it gets rejected out of hand. Good cloud solutions beat an embedded solution hands-down; they just need some improvements (see my hybrid bullet above). Once a customer experiences a good solution, they will become frustrated with one that performs poorly. In my opinion, it's better not to offer the service at all, than to try a graceful degradation of capability, because most customers won't understand or care. Spend the effort instead on making sure you always have an acceptable cloud connection — either through multiple redundant mechanisms or a car-based powerful antenna — and you'll be better off. Even when the car knows some data that the cloud doesn't (like a mobile's contact list or music selection), there's no need to handle that on the embedded side. The cloud recognition server is powerful enough to not require the data set a priori. And I think we can predict an eventual migration of phone data to cloud-based data (or cloud-synchronized data) that makes the car's knowledge either easily transferrable or less relevant.

Who makes money, and how, from voice-enabled agents or voice services? This was one of the best questions of the panel, because nobody really knows the exact model, but everybody agreed that customer tolerance is very low. The most likely candidate is ad-based revenue. This doesn't mean reading ads aloud to the driver, but rather, positively influencing search results for either active or temporary situation-based points of interest (POIs). Depending on how valuable the service is to the driver, there will still be an option for service-based payments and high-value apps.

Standards and building mobile apps — will it come? You need standards if you want to build an app platform that will promote application creation and adoption. That's what we're doing with the QNX CAR 2 application platform — creating a way for someone other than the car companies to join the ecosystem and to deploy their apps to the car in a controlled way. But don't forget, you need a standard way to deploy apps for the cloud half of the recognition, too.

To close, let me share two photos. One was taken outside the Marriott Marquis, the hotel hosting the conference just off of Times Square in NYC. The other is from our PR agency, Breakaway Communications. What do they have in common? Wooden water towers. Sorry, I couldn't help myself; I just love those things. They just look so quaint in a city full of glass and brick.

The summer road trip of 2017 – Part II

Lynn Gayowski

Our series looking at how in-car technologies will transform your summer road trip continues with part II. 2017 is around the corner, and between now and then, automakers will introduce a bevy of new features that will make for a safer and more enjoyable summer road trip. In our first part, we looked at your road trip soundtrack, navigation, and mobile device connectivity. This week, we look at safety, acoustics, and autonomous cars as we cruise to the last exit for this blog series.

Staying safe
By 2017, we likely won’t have developed the technology to shrink your mechanic down to a size that allows you to perch one on your dashboard like a bobble-head, but many cars will have a “virtual mechanic.” This application will let you check lights, fluids, tire pressure and other system vitals, all through your center stack, digital instrument cluster, or phone – as seen below. The idea of a safety speedometer is hardly new in concept (see the Plymouth safety speedometer from 1939), but its modern implementation in the cars of 2017 in the form of vision systems performing road sign detection might just mean fewer speeding tickets on your road trip, especially as you cruise through unfamiliar areas. 

Staying in touch
Sometimes you want to take a road trip to get away from the world, but sometimes you still want (or need) to stay connected. Whether it’s phone calls, texts, or emails, all of this information will continue to be seamlessly integrated into your car in 2017. Less fumbling, fewer distractions.

And low-quality, stilted speakerphone calls will be a thing of the past with the emerging crop of acoustic technologies. Driving alone on a stretch of road and miss having your loved ones close by? Advanced duplex technology will make it seem as though the person on the other end of your phone conversation is sitting right beside you in the passenger’s seat.  

Another cool development? You won’t have to struggle to use voice recognition technologies because of your noisy in-car cabin (that’s right, serenely quiet cabins will no longer be exclusive to luxury cars). Vehicles will continue to evolve to meet the strictest CAFÉ and emissions standards, while the negative acoustic side-effects from less damping materials will be countered using software to remove unwanted engine sound. And your engine in 2017 might really sound like purring (or growling, if that’s your preference), as signature sounds are enabled by engine sound enhancement software. So not only will you not feel crazy for talking to your car, you’ll also be less frustrated as you do so cruising down the interstate. 

Beyond 2017: Look ma, no hands!
While it won’t happen quite as soon as 2017, autonomous cars will hit the roads in the relatively near future, forever changing the dynamic of the road trip. Will road trips be more accessible for the elderly and others who can’t physically drive long distances? Will the new meaning of "cruise control" make the road trip more or less enjoyable? All of these considerations are up for discussion. One thing is certain: many of the advanced safety systems of today and 2017 are precursors to cars that could drive themselves. One such example of what the future of autonomous driving will look like is the University of Parma’s DEEVA autonomous car project being developed by the Artificial Vision and Intelligent Systems Laboratory (VisLab).  

How is in-car technology playing a role in your current summer road trip? How do you want it to improve your future road trips? Stay tuned to our QNX_Auto Twitter account and Facebook page for weekly discussions throughout the rest of the summer about 2017 has in store for your road trip.

Acoustics, ADAS, and autonomous cars, oh my!

Lynn Gayowski

Trying to make sense of where automotive technology is headed can be as tricky as finding your way through a poppy field while avoiding flying monkeys. Well strap on your shiny, red, video-watching shoes because Derek Kuhn can help. Derek, VP marketing and sales for QNX, was interviewed at Telematics Detroit and did an excellent job of summing up the latest on automotive acoustics, advanced driver assistance systems (ADAS), and autonomous cars.

QNX announced the new QNX OS for Automotive Safety at Telematics Detroit, so safety was clearly top of mind during the interview. One question posed was whether automakers have the potential to use safety options as revenue generators. There's a quote here I love: "Safety shouldn't be about premium." OEMs need to find cost-effective ways to bring next-generation safety to the mass market, not just luxury vehicles.

The section of the video I find most interesting is when Derek discusses how acoustics in a car play a big role in creating "the emotion and experience of driving." Noise reduction technology and engine sound enhancement both have a significant impact on a driver's affinity for a vehicle, and OEMs are taking note.

Check out the video for yourself here, my pretties:

Just add jelly

I like toast. I also like peanut butter. But you know what I really like? The two of them together. Not only does the combination taste great, but making it is easy: all I need is a knife, a toaster, and a plate. Heck, sometimes, I even skip the plate!

Why am I mentioning this? Well, when designing an embedded system, you often need to source third-party hardware or software components. And when doing so, you should always make sure they follow the toast-and-peanut-butter rule: work together out of the box, with little or no fuss. That way, you can focus on adding your own special jelly and transform your peanut-butter toast into a one-of-a-kind PB&J that customers can't get enough of. (Man, I should never write when I'm hungry!)

Evan-Amos, Wikimedia Commons

Seriously, there is no better way to jump-start an embedded project than to choose components that already work in concert. Which is the idea behind today's announcement between Cadence and QNX Software Systems. In a nutshell, they have announced that QNX Acoustics for Active Noise Control (ANC) has been ported to the Cadence Tensilica HiFi Audio/Voice DSP core.

If you're unfamiliar with QNX Acoustics for ANC, it's a software solution for reducing unwanted engine "boom" inside passenger vehicles. Compact and efficient, it can run on a processor or DSP core in the vehicle's infotainment system or audio amplifier, eliminating the dedicated hardware of conventional ANC solutions.

According to Peter McCarthy of QNX, “modern fuel-saving techniques, such as deactivating cylinders when engine load is light, can cause irritating boom noise that distracts the driver. QNX Acoustics for ANC generates targeted anti-noise over the car’s audio system to cancel out this boom for a more enjoyable ride. By combining QNX Acoustics for ANC with the widely deployed Cadence Tensilica HiFi Audio/Voice DSP core, system designers can reduce engine noise while also eliminating the costs associated with designing and prototyping a custom hardware controller module.”

For more information, read the press release and check out previous posts on QNX Acoustics for ANC.

In-car displays you hear, rather than see

We still have a lot in common with our caveman ancestors. (Yes, I know, they didn't all live in caves. Some lived in forests, others in savannahs, and still others in jungles. But I'm trying to make a point, so bear with me!)

Take, for example, our sense of hearing. At one time, we used auditory cues to locate prey or, conversely, avoid becoming prey. If a cave bear growled, getting a fix on the location of the growl could mean the difference between life and death. At the very least, it helped you avoid running directly into the bear's mouth.

Kidding aside, the human auditory system has a serious ability to fix the location, direction, and trajectory of objects, be they cave bears or Buicks. And it's an ability that's been honed from time immemorial. So why not take advantage of it when creating user interfaces for cars?

Which brings us to spatial auditory displays. In a nutshell, these displays allow you to perceive sound as coming from various locations in a three-dimensional space. Deployed in a car, they can help you intuitively identify voices and sources of instructions, and help pinpoint the location and relative trajectory of danger. They can also improve reaction times to application prompts and potentially hazardous events.

Interested in this topic? Learn more in Scott Pennock's ECD article, "Spatial auditory displays: Reducing cognitive load and improving driver reaction times."

I know, that's a lot to take in. So let's look at an example.

Locating the emergency vehicle, without really trying
Have you ever been cruising along when, suddenly, you hear an ambulance siren? I don't know about you, but I often spend time figuring out where, exactly, the ambulance is coming from. And I don't always get it right. That's called a location error.

Such errors can occur for a variety of reasons. For example, if the ambulance is approaching from the right, but your left window is open and a building on the left is reflecting sound from the siren, you might make the mistake of thinking that the ambulance is approaching from the left. Your mind realizes, quite correctly, that the sound is coming from the left, but the environment is conspiring to mask where the sound is actually coming from.

A spatial auditory display can help address this problem by controlling the acoustic cues you hear. The degree to which the display can do this depends, in part, on the hardware employed. For example, a display based on a large array of loudspeakers can provide more location information than one based on two loudspeakers.

In any case (and this is important), the display can help you determine the location more quickly and with less cognitive load — which means you may have more brain cycles to respond to the situation appropriately.


Helping the driver locate and track an emergency vehicle

A slight right, not a sharp right
I'm only scratching the surface here. Spatial auditory displays can, in fact, help improve all kinds of driving activities, from engaging in a handsfree call to using your navigation system.

For example, rather than simply say "turn right", the display could emit the instruction from the right side of the vehicle. It could even use apparent motion of the auditory prompt to convey a slight right as opposed to a sharp right.

But enough from me. To learn more about spatial auditory displays, check out a new article from my colleague Scott Pennock, whose knowledge of spatial auditory displays far surpasses mine. The article is called Spatial auditory displays: Reducing cognitive load and improving driver reaction times, and it has just been published by Embedded Computing Design magazine.
 

WIRED Autopia slips into driver's seat of QNX reference vehicle

Chances are, you've seen pictures of the new QNX reference vehicle. You may have even seen the "making of" video that QNX released a few days ago. But have you seen any video of the vehicle in action?

If not, check out this vid by Doug Newcomb of WIRED Autopia. Last week, at Telematics Detroit, Doug met up with Andrew Poliak of QNX for a tour of the vehicle and its various features, including a re-skinnable UI and voice-controlled Facebook integration. The camera was rolling, and here's what it caught:

A cool (and connected) cluster

"The more we get connected,
Connected, connected,
The more we get connected,
The happier we’ll be."

Okay, I'll admit it, that's a little over the top. But even if connectivity can't make you happy, it can still breathe a great deal of enjoyment and productivity into your life. And when you build connectivity into devices that have previously stood alone, you open the door to all kinds of exciting possibilities. Case in point: the new digital instrument cluster for the QNX technology concept car.

Here's the cluster at a glance — click to see a bigger version:



Admit it: you'd love to wrap your hands around that steering wheel. I know I would. The Mercedes CLA45 is a sweet ride, and it inspired the QNX concept team to pull out all the stops when designing the new cluster with our partner Rightware, a specialist in UI tools for cars.

Four-point check
Okay, let's hop in and take a closer look. But before we put the car in gear, did your driving instructor ever tell you to do a four-point check? You know, where you make sure your lights, brakes, and other systems are working properly? You do remember to do that, don't you? The cluster makes the task a little easier by checking lights, tire pressures, fluids, and the HVAC system automatically:



Ease off the pedal, buddy
Time to put the car in gear. But before we do, let me tell you about the Plymouth safety speedometer. Designed to curb speeding, it alerted the driver whenever he or she leaned too hard on the gas. In theory, it was a great idea. In practice, it wasn't. You see, the year was 1939. And given the limitations of 1939 technology, the Plymouth safety speedometer couldn't take driving conditions or the local speed limit into account. So the speedometer always displayed warnings at the same speeds, no matter what the speed limit.

Connectivity to the rescue! Some modern navigation systems include information on local speed limits. By connecting the digital instrumental cluster to the navigation system in the car's head unit, the concept team was able to pull this information and display it in real time on the cluster, creating a modern (and much more useful) equivalent of Plymouth's 1939 invention.

Look at the image below. You'll see the local speed limit surrounded by a thick red circle, alerting the driver that they are breaking the limit — the fulfillment of an idea that has been 75 years in the making. Mind you, this isn't the only information that the cluster pulls from the head unit. It can also display turn-by-turn directions, trip information, album art, and other content normally relegated to the center display:



Should you answer?
Oh, hold on, the cluster is alerting us to an incoming call. You can ignore it, or you can answer by pushing a button on the steering wheel. And because this is the QNX technology concept car, it's no ordinary phone call. The car is equipped with QNX Acoustics for Voice, which supports Wideband Plus speech to deliver almost four times the bandwidth of a standard narrowband call. Translation: The person on the far end of call sounds like they're sitting right next to you.



Looking back
Okay, it's been a great drive, but time to head home. And in this case, home is the QNX garage. The garage doors are pretty narrow, and you need to back in carefully, so it's great to know that the cluster also provides a convenient window for the car's rear-view camera:



Meanwhile, in the real world...
Don't make the mistake of thinking this is a Buck Rogers scenario. Because the same combination of QNX and Rightware technology is already powering innovative systems like the Audi virtual cockpit. If you haven't yet seen the Audi system in action, check it out:

Crisper, clearer in-car communication — Roger that

Tina Jeffrey

Over the years, Telematics Detroit has become a premier venue for showing off advancements in automotive infotainment, telematics, apps, cloud connectivity, silicon, and more. If the breadth of QNX technology being demonstrated at the show this week is any indication, the event won’t disappoint. Among the highlights is our next-generation acoustics processing middleware — QNX Acoustics for Voice 3.0 — which has been architected to deliver the highest-quality audio for hands-free and speech recognition systems, enabling the ultimate acoustics experience in the car.

What is QNX Acoustics for Voice?
QNX Acoustics for Voice 3.0 is the successor to the QNX Aviage Acoustics Processing Suite 2.0. The new product includes a set of libraries — standard and premium — that offer automakers ultimate flexibility for voice processing in the harsh audio environment of the car.

The standard library provides a full-featured solution for implementing narrowband and wideband hands-free communications, operating at 8 kHz and 16kHz sample rates, respectively. It also includes innovative new features for performing echo cancellation, noise reduction, adaptive equalization, and automatic gain control. Perhaps the most valuable feature, especially for systems constrained by limited CPU cycles, is the high efficiency mode, which can process wideband and higher-bandwidth speech with substantially less CPU load. The net result: more processing headroom for other tasks.

The premium library includes all the standard library functionality, plus support for Wideband Plus, which expands the frequency range of transmitted speech to 50 Hz - 11 kHz, at a 24kHz sample rate. The introduction of Wideband Plus fulfills the higher voice quality and low noise requirements demanded by the latest smartphone connectivity protocols for telephony, VoIP services, and speech recognition. Let me recap with a table:

Supported capabilities	Standard library	Premium library
Narrowband audio: 300 – 3400Hz (8kHz sample rate)	✔	✔
Wideband audio: 50-7000Hz (16kHz sample rate)	✔	✔
Wideband Plus audio: 50Hz – 11kHz (24kHz sample rate)		✔
High efficiency mode	✔  (Wideband only)	✔
VOIP requirements for new smartphone connectivity protocols		✔
Cloud-based speech recognition requirements for new smartphone connectivity protocols		✔

---

Why is high-quality speech important in the car?
Simply put, it improves the user experience and can benefit passenger safety. Also, new smartphone connectivity protocols require it. Let’s examine two use cases: hands-free voice calling, and speech recognition.

In a voice call, processing a larger bandwidth of speech and eliminating echo and noise from various sources, including wind, road, vents, fans, and tires, dramatically increases speech intelligibility — and the more intelligible the speech, the more natural the flow of conversation. Also, clearer speech has less impact on the driver’s cognitive load, enabling the driver to pay more attention to the task at hand: driving.

Speech recognition systems are becoming a primary way to manage apps and services in the car. Voice commands can initiate phone calls, select media for playback, search for points of interest (POI), and choose a destination.

Technological advancements in pre-processing voice input to remove noise and disturbances helps speech recognizers detect commands more reliably, thereby achieving higher recognition accuracy. Early speech recognition systems, by comparison, were unintuitive and performed poorly. Drivers became so frustrated that they stopped using these systems and resorted to picking up their smartphones, completely eliminating the safety benefits of speech recognition.

QNX Acoustics for Voice 3.0 is a comprehensive automotive voice solution that includes industry-leading echo cancellation, noise reduction, adaptive equalization and automatic gain control.

If you happen to be at Telematics Update in Novi Michigan this week, be sure to drop by our booth to sit in our latest concept car — a specially modified Mercedes-Benz CLA45 AMG — and experience our acoustics technologies first hand.