One day I’ll be Luke Skywalker

Cyril Clocher

What happens when you blend ADAS with infotainment? Guest post by Cyril Clocher, business manager for automotive processors at Texas Instruments

As we all begin preparing for our trek to Vegas for CES 2015, I would like my young friends (born in the 70s, of course) to reflect on their impressions of the first episode of Lucas’s trilogy back in 1977. On my side, I perfectly remember thinking one day I would be Luke Skywalker.

The eyes of young boys and girls were literally amazed by this epic space opera and particularly by technologies used by our heroes to fight the Galactic Empire. You have to remember it was an era where we still used rotary phones and GPS was in its infancy. So you can imagine how impactful it was for us to see our favorite characters using wireless electronic gadgets with revolutionary HMIs such as natural voice recognition, gesture controls or touch screens; droids speaking and enhancing human intelligence; and autonomous vehicles traveling the galaxy safely while playing chess with a Wookiee. Now you’re with me…

But instead of becoming Luke Skywalker a lot of us realized that we would have a bigger impact by inventing or engineering these technologies and by transforming early concepts into real products we all use today. As a result, smartphones and wireless connectivity are now in our everyday lives; the Internet of Things (IoT) is getting more popular in applications such as activity trackers that monitor personal metrics; and our kids are more used to touch screens than mice or keyboards, and cannot think of on-line gaming without gesture control. In fact, I just used voice recognition to upgrade the Wi-Fi plan from my Telco provider.

But the journey is not over yet. Our generation has still to deliver an autonomous vehicle that is green, safe, and fun to control – I think the word “drive” will be obsolete for such a vehicle.

The automotive industry has taken several steps to achieve this exciting goal, including integration of advanced and connected in-car infotainment systems in more models as well as a number of technologies categorized under Advanced Driver Assistance Systems (ADAS) that can create a safer and unique driving experience. From more than a decade, Texas Instruments has invested in infotainment and ADAS: “Jacinto” and TDAx automotive processors as well as the many analog companion chips supporting these trends.

"Jacinto 6 EP" and "Jacinto 6 Ex"
infotainment processors

A unique approach of TI is our capability to leverage best of both worlds for non-safety critical features, and to provide a seamless integration of informational ADAS functions into existing infotainment systems so the vehicle better informs and warns the driver. We announced that capability at SAE Convergence in Detroit in October 2014 with the “Jacinto 6 Ex” processor (DRA756), which contains powerful CPU, graphics multimedia, and radio cores with differentiated vision co-processors, called embedded vision engines (EVE), and additional DSPs that perform the complex ADAS processing.

For the TI’s automotive team, the CES 2015 show is even more exciting than in previous years, as we’ve taken our concept of informational ADAS to the next step. With joint efforts and hard work from both TI and QNX teams, we’ve together implemented a real informational ADAS system running the QNX CAR™ Platform for Infotainment on a “Jacinto 6 Ex” processor.

I could try describing this system in detail, but just like the Star Wars movies, it’s best to experience our “Jacinto 6 Ex” and QNX CAR Platform-based system in person. Contact your TI or QNX representative today and schedule a meeting to visit our private suite at CES at the TI Village (N115-N119) or to immerse yourself in a combined IVI, cluster, megapixel surround view, and DLP® based HUD display with augmented reality running on a single “Jacinto 6 Ex” SoC demonstration. And don't forget to visit the QNX booth (2231), where you can see the QNX reference vehicle running a variety of ADAS and infotainment applications on “Jacinto 6” processors.

Integrated cockpit featuring DLP powered HUD and QNX CAR Platform running on a single “Jacinto 6 Ex” SoC.

One day I’ll experience Skywalker’s life as I will no doubt have the opportunity to control an intelligent and autonomous vehicle with my biometrics, voice, and gestures while riding with my family to the movie theater playing chess with my grandkids, not yet a Wookiee.

Klocwork joins QNX automotive safety ecosystem

Paul Leroux

This just in: Klocwork, a leader in development tools for creating secure software, has become an ecosystem partner for the QNX Automotive Safety Program for ISO 26262.

Klocwork joins a roster of companies, including Elektrobit, Freescale, NVIDIA, and TI, who already support the program, which is designed to help automotive companies build digital instrument clusters, ADAS systems, and other products with functional safety requirements.

Klocwork offers Insight, a source code analysis tool recently certified to the ISO 26262 and IEC 61508 functional safety standards. Insight plugs directly into the QNX Momentics Tool Suite, allowing developers to detect security and safety vulnerabilities on the fly, and to ensure their code meets functional safety standards.

"Klocwork Insight provides real-time feedback during code development, immediately alerting developers to code that may conflict with the MISRA C/C++ coding standards required by ISO 26262," said Grant Courville, director of product management at QNX. "Better yet, Insight plugs into our IDE to provide a seamless and productive development experience."

The QNX Automotive Safety Program for ISO 26262 was created to help automotive companies building functional safety products to leverage QNX Software Systems’ proven competency in certifications, safety-critical systems, and automotive software design. Key elements of the program include an example safety case based on the QNX Neutrino RTOS Safe Kernel, guidelines on safety-critical design for real-time OS-based systems, a suite of professional services, and an ecosystem of supporting vendors who offer complementary hardware, tool chains, graphics technologies, and consulting services for safety critical systems.

Read the press release.

The power of together

Bringing more technologies into the car is all well and good. The real goal, however, is to integrate them in a way that genuinely improves the driving experience.

Can we all agree that ‘synergy’ has become one of the most misused and overused words in the English language? In the pantheon of verbal chestnuts, synergy holds a place of honor, surpassed only by ‘best practices’ and ‘paradigm shift’.

Mind you, you can’t blame people for invoking the word so often. Because, as we all know, the real value in things often comes from their interaction — the moment they stop acting alone and start working in concert. The classic example is water, yeast, and flour, a combination that yields something far more flavorful than its constituent parts. I am speaking, of course, of bread.

Automakers get this principle. Case in point: adaptive cruise control, which takes a decades-old concept — conventional cruise control — and marries it with advances in radar sensors and digital signal processing. The result is something that doesn’t simply maintain a constant speed, but can help reduce accidents and, according to some research, traffic jams.

At QNX Software Systems, we also take this principle to heart. For example, read my recent post on the architecture of the QNX CAR Platform and you’ll see that we consciously designed the platform to help things work together. In fact, the platform's ability to integrate numerous technologies, in a seamless and concurrent fashion, is arguably its most salient quality.

This ability to blend disparate technologies into a collaborative whole isn't just a gee-whiz feature. Rather, it is critical to enabling the continued evolution and success of the connected car. Because it’s not enough to have smartphone connectivity. Or cloud connectivity. Or digital instrument clusters. Or any number of ADAS features, from collision warnings to autonomous braking. The real magic, and real value to the consumer, occurs when some or all of these come together to create something greater than the sum of the parts.

Simply put, it's all about the — dare I say it? — synergy that thoughtful integration can offer.

At CES this year, we will explore the potential of integration and demonstrate the unexpected value it can bring. The story begins on the QNX website.

A need for speed... and safety

Matt Shumsky

For me, cars and safety go hand in hand. Don’t get me wrong, I have a need for speed. I do, after all, drive a 2006 compact with 140 HP (pause for laughter). But no one, and I mean no one, wants to be barreling down a highway in icy conditions at 120 km/hr without working brakes, am I right?

So this begs the question, what’s the best way to design a software system that ensures the adaptive cruise control system keeps a safe distance from the car ahead? Or that tells the digital instrument cluster the correct information to display? And how can you make sure the display information isn’t corrupted?

Enter QNX and the ISO 26262 functional safety standard.

QNX Software Systems is partnering with LDRA to present a webinar on “Ensuring Automotive Functional Safety”. During this webinar, you’ll learn about:

• Development and verification tools proven to help provide safer automotive software systems
• How suppliers can develop software systems faster with an OS tuned for automotive safety

Ensuring Automotive Functional Safety with QNX and LDRA
Thursday, November 20, 2014
9:00 am PST / 12:00 pm EST / 5:00 pm UTC

I hope you can join us!

Japan update: ADAS, wearables, integrated cockpits, and autonomous cars

Yoshiki Chubachi

Will the joy of driving be a design criterion for tomorrow’s vehicles? It had better be.

A couple of weeks ago, QNX Software Systems sponsored Telematics Japan in Tokyo. This event offers a great opportunity to catch up with colleagues from automotive companies, discuss technology and business trends, and showcase the latest technology demos. Speaking of which, here’s a photo of me with a Japan-localized demo of the QNX CAR Platform. You can also see a QNX-based digital instrument cluster in the lower-left corner — this was developed by Three D, one of our local technology partners:



While at the event, I spoke on the panel, “Evolving ecosystems for future HMI, OS, and telematics platform development.” During the discussion, we conducted a real-time poll and asked the audience three questions:

1) Do you think having Apple CarPlay and Android Auto will augment a vehicle brand?
2) Do you expect wearable technologies to be integrated into cars?
3) If your rental car were hacked, who would you complain to?

For question 1, 32% of the audience said CarPlay and Android Auto will improve a brand; 68% didn't think so. In my opinion, this result indicates that smartphone connectivity in cars is now an expected feature. For question 2, 76% answered that they expect to see wearables integrated into cars. This response gives us a new perspective — people are looking at wearables as a possible addition to go with ADAS systems. For example, a wearable device could help prevent accidents by monitoring the driver for drowsiness and other dangerous signs. For question 3, 68% said they would complain to the rental company. Mind you, this raises the question: if your own car were hacked, who would you complain to?

Integrated cockpits
There is growing concern around safety and security as companies attempt to grow more business by leveraging connectivity in cars. The trend is apparent if you look at the number of safety- and security-related demos at various automotive shows.

Case in point: I recently attended a private automotive event hosted by Renesas, where many ADAS and integrated cockpit demos were on display. And last month, CEATEC Japan (aka the CES of Japan) featured integrated cockpit demos from companies like Fujitsu, Pioneer, Mitsubishi, Kyocera, and NTT Docomo.

For the joy of it
Things are so different from when I first started developing in-car navigation systems 20 years ago. Infotainment systems are now turning into integrated cockpits. In Japan, the automotive industry is looking at early 2020s as the time when commercially available autonomous cars will be on the road. In the coming years, the in-car environment, including infotainment, cameras and other systems, will change immensely — I’m not exactly sure what cars in the year 2020 will look like, but I know it will be something I could never have imagined 20 years ago.

A panel participant at Telematics Japan said to me, “If autonomous cars become reality and my car is not going to let me drive anymore, I am not sure what the point of having a car is.” This is true. As we continue to develop for future cars, we may want to remind ourselves of the “joy of driving” factor.

My top moments of 2013 — so far

Paul Leroux

Yes, I know, 2013 isn’t over yet. But it’s been such a milestone year for our automotive business that I can’t wait another two months to talk about it. And besides, you’ll be busy as an elf at the end of December, visiting family and friends, skiing the Rockies, or buying exercise equipment to compensate for all those holiday carbs. Which means if I wait, you’ll never get to read this. So let’s get started.


We unveil a totally new (and totally cool) technology concept car

Times Square. We were there.

It all began at 2013 CES, when we took the wraps off the latest QNX technology concept car — a one-of-a-kind Bentley Continental GT. The QNX concept team outfitted the Bentley with an array of technologies, including a high-definition DLP display, a 3D rear-view camera, cloud-based voice recognition, smartphone connectivity, and… oh heck, just read the blog post to get the full skinny.

Even if you weren’t at CES, you could still see the car in action. Brian Cooley of CNET, Michael Guillory of Texas Instruments, the folks at Elektrobit, and Discovery Canada’s Daily Planet were just some of the individuals and organizations who posted videos. You could also connect to the car through a nifty web app. Heck, you could even see the Bentley’s dash on the big screen in Times Square, thanks to the promotional efforts of Elektrobit, who also created the 3D navigation software for the concept car.

We ship the platform
We wanted to drive into CES with all cylinders firing, so we also released version 2.0 of the QNX CAR Platform for Infotainment. In fact, several customers in the U.S., Germany, Japan, and China had already started to use the platform, through participation in an early access program. Which brings me to the next milestone...

Delphi boards the platform

The first of many.

Also at CES, Delphi, a global automotive supplier and long-time QNX customer, announced that version 2.0 of the QNX CAR Platform will form the basis of its next-generation infotainment systems. As it turned out, this was just one of several QNX CAR customer announcements in 2013 — but I’m getting ahead of myself.

We have the good fortune to be featured in Fortune
Fast forward to April, when Fortune magazine took a look at how QNX Software Systems evolved from its roots in the early 1980s to become a major automotive player. Bad news: you need a subscription to read the article on the Fortune website. Good news: you can read the same article for free on CNN Money. ;-)

A music platform sets the tone for our platform
In April, 7digital, a digital music provider, announced that it will integrate its 23+ million track catalogue with the QNX CAR Platform. It didn't take long for several other partners to announce their platform support. These include Renesas (R-Car system-on-chip for high-performance infotainment), AutoNavi (mobile navigation technology for the Chinese market), Kotei (navigation engine for the Japanese market), and Digia (Qt application framework).

We stay focused on distraction
Back in early 2011, Scott Pennock of QNX was selected to chair an ITU-T focus group on driver distraction. The group’s objective was serious and its work was complex, but its ultimate goal was simple: to help reduce collisions. This year, the group wrapped up its work and published several reports — but really, this is only the beginning of QNX and ITU-T efforts in this area.

We help develop a new standard

Goodbye fragmentation; hello
standard APIs.

Industry fragmentation sucks. It means everyone is busy reinventing the wheel when they could be inventing something new instead. So I was delighted to see my colleague Andy Gryc become co-chair of the W3C Automotive and Web Platform Business Group, which has the mandate to accelerate the adoption of web technologies in the car. Currently, the group is working to draft a standard set of JavaScript APIs for accessing vehicle data information. Fragmentation, thy days are numbered.

We launch an auto safety program

A two-handed approach to
helping ADAS developers.

On the one hand, we have a 30-year history in safety-critical systems and proven competency in safety certifications. On the other hand, we have deep experience in automotive software design. So why not join both hands together and allow auto companies to leverage our full expertise when they are building digital instrument clusters, advanced driver assistance systems (ADAS), and other in-car systems with safety requirements?

That’s the question we asked ourselves, and the answer was the new QNX Automotive Safety Program for ISO 26262. The program quickly drew support from several industry players, including Elektrobit, Freescale, NVIDIA, and Texas Instruments.

We jive up the Jeep

A tasty mix of HTML5 & Android
apps, served on a Qt interface,
with OpenGL ES on the side.

If you don’t already know, we use a Jeep Wrangler as our reference vehicle — basically, a demo vehicle outfitted with a stock version of the QNX CAR Platform. This summer, we got to trick out the Jeep with a new, upcoming version of the platform, which adds support for Android apps and for user interfaces based on the Qt 5 framework.

Did I mention? The platform runs Android apps in a separate application container, much like it handles HTML5 apps. This sandboxed approach keeps the app environment cleanly partitioned from the UI, protecting both the UI and the overall system from unpredictable web content. Good, that.

The commonwealth’s leader honors our leader

I only ate one piece. Honest.

Okay, this one has nothing to do with automotive, but I couldn’t resist. Dan Dodge, our CEO and co-founder, received a Queen Elizabeth II Diamond Jubilee Medal in recognition of his many achievements and contributions to Canadian society. To celebrate, we gave Dan a surprise party, complete with the obligatory cake. (In case you’re wondering, the cake was yummy. But any rumors suggesting that I went back for a second, third, and fourth piece are total fabrications. Honestly, the stories people cook up.)

Mind you, Dan wasn’t the only one to garner praise. Sheridan Ethier, the manager of the QNX CAR development team, was also honored — not by the queen, but by the Ottawa Business Journal for his technical achievements, business leadership, and community involvement.

Chevy MyLink drives home with first prize — twice
There's nothing better than going home with first prize. Except, perhaps, doing it twice. In January, the QNX-based Chevy MyLink system earned a Best of CES 2013 Award, in the car tech category. And in May, it pulled another coup: first place in the "Automotive, LBS, Navigation & Safe Driving" category of the 2013 CTIA Emerging Technology (E-Tech) Awards.

Panasonic, Garmin, and Foryou get with the platform

Garmin K2 platform: because
one great platform deserves
another.

August was crazy busy — and crazy good. Within the space of two weeks, three big names in the global auto industry revealed that they’re using the QNX CAR Platform for their next-gen systems. Up first was Panasonic, who will use the platform to build systems for automakers in North America, Europe, and Japan. Next was Foryou, who will create infotainment systems for automakers in China. And last was Garmin, who are using the platform in the new Garmin K2, the company’s infotainment solution for automotive OEMs.

And if all that wasn’t cool enough…

Mercedes-Benz showcases the platform

Did I mention I want one?

When Mercedes-Benz decides to wow the crowds at the Frankfurt Motor Show, it doesn’t settle for second best. Which is why, in my not so humble opinion, they chose the QNX CAR Platform for the oh-so-desirable Mercedes-Benz Concept S-Class Coupé.

Mind you, this isn’t the first time QNX and Mercedes-Benz have joined forces. In fact, the QNX auto team and Mercedes-Benz Research & Development North America have collaborated since the early 2000s. Moreover, QNX has supplied the OS for a variety of Mercedes infotainment systems. The infotainment system and digital cluster in the Concept S-Class Coupé are the latest — and arguably coolest — products of this long collaboration.

We create noise to eliminate noise

Taking a sound approach to
creating a quieter ride.

Confused yet? Don’t be. You see, it’s quite simple. Automakers today are using techniques like variable cylinder management, which cut fuel consumption (good), but also increase engine noise (bad). Until now, car companies have been using active noise control systems, which play “anti-noise” to cancel out the unwanted engine sounds. All fine and good, but these systems require dedicated hardware — and that makes them expensive. So we devised a software product, QNX Acoustics for Active Noise Control, that not only out-performs conventional solutions, but can run on the car’s existing audio or infotainment hardware. Goodbye dedicated hardware, hello cost savings.

And we flub our lines on occasion
Our HTML5 video series has given companies like Audi, OnStar, Gartner, TCS, and Pandora a public forum to discuss why HTML5 and other open standards are key to the future of the connected car. The videos are filled with erudite conversation, but every now and then, it becomes obvious that sounding smart in front of a camera is a little harder than it looks. So what did we do with the embarrassing bits? Create a blooper reel, of course.

Are these bloopers our greatest moments? Nope. Are they among the funniest? Oh yeah. :-)

Autonomous cars? Suddenly, I’m not so skeptical

Guest post from Emil Dautovic, European automotive business development manager for QNX Software Systems

As a driving enthusiast, I have always felt a bit skeptical about the notion of autonomous cars. The reason is simple: I actually enjoy driving and don’t want someone else to do it for me, in this case the car itself.

Recently, however, my skepticism has begun to soften. I am fascinated, for example, by the SARTRE road train project, where a lead vehicle takes responsibility for a platoon of semi-autonomous cars. Also, recent research from the U.S. Highway Loss Data Institute suggests that, when it comes to some driving tasks, ADAS systems can already put many human drivers to shame.

Autonomous drive will become especially important when today’s “always on” generation starts to buy cars in earnest. They will, no doubt, want to consume multimedia and interact through social media even while on the road, and automakers will need to accommodate them.

HMIs with more (and less) distraction
What would this mean for car makers? Among other things, the infotainment system in a self-driving car could offer an HMI mode that gives the driver more freedom to pay attention to non-driving activities. When the car subsequently needs a human driver (for instance, it becomes disconnected from a road train), the infotainment system could disable these features and immediately go back to a less distracting user interface.

Also, driver assist systems — such as those for detecting animals and pedestrians — would need to be integrated with the road train system to decide how to react when, say, a rabbit runs in front of the car. For instance, should the car brake and warn other cars of the fact, or would it be safer to simply keep driving? It will be interesting to follow this initiative and see how the technical and business aspects evolve, and how, for example, the owner of the lead vehicle will be paid.

For another interesting example of research into autonomous drive, check out the BRAiVE project led by the VisLab team at the University of Parma. The BRAiVE project uses a variety of sensors, with a focus on low-cost alternatives that could realistically integrated into in production cars.

Bells and whistles
So what kind of impact could all this have on a company providing automotive software platforms?

There will, I believe, be an increased demand for a platform that could run all of these applications, enabling the advanced use cases while ensuring that critical functions always have enough processor power. And, of course, the platform will have to be reliable. If this same platform could offer all the bells and whistles available in consumer electronics and demanded by younger drivers, the self-driving future might prove to be a bit closer than we think.

By the way, if you’re unfamiliar with the SARTRE road train project, check out this video:

---

More about Emil
Emil Dautovic is an automotive business development manager at QNX Software Systems, where he is responsible for the European automotive market. Prior to joining QNX, he worked as a business area manager for The Astonishing Tribe (TAT), where he built TAT's automotive business from scratch and helped transform the company into an important player in the automotive HMI field with leading automotive OEMs and tier ones. He has also worked at AU-System (later Teleca and Obigo), where he served as a consultant on GSM base station development and as a sales representative serving mobile phone OEMs and ODMs worldwide. Emil holds an M.Sc. in Electronic Engineering from Lunds Tekniska Högskola.

Top 10 challenges facing the ADAS industry

Tina Jeffrey

It didn’t take long. Just months after the release of the ISO 26262 automotive functional safety standard in 2011, the auto industry began to grasp its importance and adopt it in a big way. Safety certification is gaining traction in the industry as automakers introduce advanced driver assistance systems (ADAS), digital instrument clusters, heads-up displays, and other new technologies in their vehicles.

Governments around the world, in particular those of the United States and the European Union, are calling for the standardization of ADAS features. Meanwhile, consumers are demonstrating a readiness to adopt these systems to make their driving experience safer. In fact, vehicle safety rating systems are becoming a vital ‘go to’ information resource for new car buyers. Take, for example, the European New Car Assessment Programme Advanced (Euro NCAP Advanced). This organization publishes safety ratings on cars that employ technologies with scientifically proven safety benefits for drivers. The emergence of these ratings encourages automakers to exceed minimum statutory requirements for new cars.

Sizing the ADAS market
ABI Research claims that the global ADAS market, estimated at US$16.6 billion at the end of 2012, will grow to more than US$260 billion by the end of 2020, representing a CAGR of 41%. Which means that cars will ship with more of the following types of safety-certified systems:



The 10 challenges
So what are the challenges that ADAS suppliers face when bringing systems to market? Here, in my opinion, are the top 10:

1. Safety must be embedded in the culture of every organization in the supply chain. ADAS suppliers can't treat safety as an afterthought that is tacked on at the end of development; rather, they must embed it into their development practices, processes, and corporate culture. To comply with ISO 26262, an ADAS supplier must establish procedures associated with safety standards, such as design guidelines, coding standards and reviews, and impact analysis procedures. It must also implement processes to assure accountability and traceability for decisions. These processes provide appropriate checks and balances and allow for safety and quality issues to be addressed as early as possible in the development cycle.
    
2. ADAS systems are a collaborative effort. Most ADAS systems must integrate intellectual properties from a number of technology partners; they are too complex to be developed in isolation by a single supplier. Also, in a safety-certified ADAS system, every component must be certified — from the underlying hardware (be it a multi-core processor, GPU, FPGA, or DSP) to the OS, middleware, algorithms, and application code. As for the application code, it must be certified to the appropriate automotive safety integrity level; the level for the ADAS applications listed above is typically ASIL D, the highest level of ISO 26262 certification.
    
3. Systems may need to comply with multiple industry guidelines or specifications. Besides ISO 26262, ADAS systems may need to comply with additional criteria, as dictated by the tier one supplier or automaker. On the software side, these criteria may include AUTOSAR or MISRA. On the hardware side, they will include AEC-Q100 qualification, which involves reliability testing of auto-grade ICs at various temperature grades. ICs must function reliably over temperature ranges that span -40 degrees C to 150 degrees C, depending on the system.
    
4. ADAS development costs are high. These systems are expensive to build. To achieve economies of scale, they must be targeted at mid- and low-end vehicle segments. Prices will then decline as volume grows and development costs are amortized, enabling more widespread adoption.
    
5. The industry lacks interoperability specifications for radar, laser, and video data in the car network. For audio-video data alone, automakers use multiple data communication standards, including MOST (media-oriented system transport), Ethernet AVB, and LVDS. As such, systems must support a multitude of interfaces to ensure adoption across a broad spectrum of possible interfaces. Also, systems may need additional interfaces to support radar or lidar data.
    
6. The industry lacks standards for embedded vision-processing algorithms. Ask 5 different developers to develop a lane departure warning system and you’ll get 5 different solutions. Each solution will likely start with a Matlab implementation that is ported to run on the selected hardware. If the developer is fortunate, the silicon will support image processing primitives (a library of functions designed for use with the hardware) to accelerate development. TI, for instance, has a set of image and video processing libraries (IMGLIB and VLIB) optimized for their silicon. These libraries serve as building blocks for embedded vision processing applications. For instance, IMGLIB has edge detection functions that could be used in a lane departure warning application.
    
7. Data acquisition and data processing for vision-based systems is high-bandwidth and computationally intensive. Vision-based ADAS systems present their own set of technical challenges. Different systems require different image sensors operating at different resolutions, frame rates, and lighting conditions. A system that performs high-speed forward-facing driver assistance functions such as road sign detection, lane departure warning, and autonomous emergency breaking must support a higher frame rate and resolution than a rear-view camera that performs obstacle detection. (A rear-view camera typically operates at low speeds, and obstacles in the field of view are in close proximity to the vehicle.) Compared to the rear-view camera, an LDW, AEB, or RSD system must acquire and process more incoming data at a faster incoming frame rate, before signaling the driver of an unintentional lane drift or warning the driver that the vehicle is exceeding the posted speed limit.
    
8. ADAS cannot add to driver distraction. There is an increase in the complexity of in-vehicle tasks and displays that can result in driver information overload. Systems are becoming more integrated and are presenting more data to the driver. Information overload could result in high cognitive workload, reducing situational awareness and countering the efficacy of ADAS. Systems must therefore be easy to use and should make use of the most appropriate modalities (visual, manual, tactile, sound, haptic, etc.) and be designed to encourage driver adoption. Development teams must establish a clear specification of the driver-vehicle interface early on in development to ensure user and system requirements are aligned.
    
9. Environmental factors affect ADAS. ADAS systems must function under a variety of weather and lighting conditions. Ideally, vision-based systems should be smart enough to understand when they are operating in poor visibility scenarios such as heavy fog or snow, or when direct sunlight shines into the lens. If the system detects that the lens is occluded or that the lighting conditions are unfavorable, it can disable itself and warn the driver that it is non-operational. Another example is an ultrasonic parking sensor that becomes prone to false positives when encrusted with mud. Combining the results of different sensors or different sensor technologies (sensor fusion) can often provide a more effective solution than using a single technology in isolation.
    
10. Testing and validating is an enormous undertaking. Arguably, testing and validation is the most challenging aspect of ADAS development, especially when it comes to vision systems. Prior to deploying a commercial vision system, an ADAS development team must amass hundreds if not thousands of hours of video clips in a regression test database, in an effort to test all scenarios. The ultimate goal is to achieve 100% accuracy and zero false positives under all possible conditions: traffic, weather, number of obstacles or pedestrians in the scene, etc. But how can the team be sure that the test database comprises all test cases? The reality is that they cannot — which is why suppliers spend years testing and validating systems, and performing extensive real-world field-trials in various geographies, prior to commercial deployment.
     

There are many hurdles to bringing ADAS to mainstream vehicles, but clearly, they are surmountable. ADAS systems are commercially available today, consumer demand is high, and the path towards widespread adoption is paved. If consumer acceptance of ADAS provides any indication of societal acceptance of autonomous drive, we’re well on our way.

Are you ready to stop micromanaging your car?

I will get to the above question. Honest. But before I do, allow me to pose another one: When autonomous cars go mainstream, will anyone even notice?

The answer to this question depends on how you define the term. If you mean completely and absolutely autonomous, with no need for a steering wheel, gas pedal, or brake pedal, then yes, most people will notice. But long before these devices stop being built into cars, another phenomenon will occur: people will stop using them.

Allow me to rewind. Last week, Tesla announced that its Model S will soon be able to “steer to stay within a lane, change lanes with the simple tap of a turn signal, and manage speed by reading road signs and using traffic-aware cruise control.” I say soon because these functions won't be activated until owners download a software update in the coming weeks. But man, what an update.

Tesla may now be at the front of the ADAS wave, but the wave was already forming — and growing. Increasingly, cars are taking over mundane or hard-to-perform tasks, and they will only become better at them as time goes on. Whether it’s autonomous braking, automatic parking, hill-descent control, adaptive cruise control, or, in the case of the Tesla S, intelligent speed adaptation, cars will do more of the driving and, in so doing, socialize us into trusting them with even more driving tasks.

Tesla Model S: soon with autopilot

In other words, the next car you buy will prepare you for not having to drive the car after that.

You know what’s funny? At some point, the computers in cars will probably become safer drivers than humans. The humans will know it, but they will still clamor for steering wheels, brake pedals, and all the other traditional accoutrements of driving. Because people like control. Or, at the very least, the feeling that control is there if you want it.

It’s like cameras. I would never think of buying a camera that didn’t have full manual mode. Because control! But guess what: I almost never turn the mode selector to M. More often than not, it’s set to Program or Aperture Priority, because both of these semi-automated modes are good enough, and both allow me to focus on taking the picture, not on micromanaging my camera.

What about you? Are you ready for a car that needs a little less micromanagement?

Squeezing into a tight spot

Paul Leroux

No doubt about it, autonomous and semi-autonomous cars will present a variety of legal and ethical challenges. But they'll also offer many benefits — some of which will be pleasantly surprising.

Take parking, for example. Cars are getting wider, but parking spaces generally aren't. So how do you squeeze into a tight spot and then step out of your car without slamming your door into the car next to you? Well, what if you didn't have to be in the car? This new video from Ford tells all...



This technology is cool, especially for aging drivers who can't crane their necks as well as they used to. Still, some gotchas come to mind. For instance, other drivers might get peeved if you momentarily leave your car on the road so you can park it remotely. Also, what if you squeeze your car into a tight parking spot just inches away from driver's door of the adjacent car — but that car doesn't support remote-controlled parking? How will the driver get back into his or her vehicle?

That said, these problems can be avoided with a little common sense on the part of the user. And I'll bet you dimes to donuts that this new technology from Ford can negotiate parking spaces more adroitly than most motorists. Which means that, eventually, we'll all have vehicles with fewer bumps, scuffs, and scratches. I could live with that.

The ISO 26262 functional safety standard: No way but up?

I was scanning some Google alerts the other day when my eyes stopped at an announcement from Freescale. The headline didn’t mince words: the Freescale Qorivva MPC5643L microcontroller, a 32-bit part based on the Power architecture, has become the first automotive MCU to receive ISO 26262 functional safety certification.

Did you notice? Freescale didn’t say only; they said first. Which suggests they see ISO 26262 as a growing trend in automotive. If so, I think they see right.

If you’re unfamiliar with ISO 26262, let me provide the Reader’s Digest version. First and foremost, it applies to automotive electronic or electrical systems that could pose a hazard (i.e. hurt people) if they malfunction. Examples include anti-lock brakes, traction control systems, adaptive cruise control systems, engine control units, and digital instrument clusters.

Will more automotive
components soon come
with stickers like this?

The standard isn’t concerned with how well such systems perform. Rather, it’s about reducing the risk, and mitigating the effects, of any malfunction that may cause injury or death. So even if something bad unexpectedly happens in a 26262-certified system — and the assumption is that bad things will happen, no matter how well the system is designed and tested — the system will minimize potential harm. For instance, consider the scenario where a high-priority software process enters an infinite loop and starts to gobble up CPU cycles. Obviously, it’s important to prevent this error from happening in the first place. But even if it does happen, the system should prevent the rogue process from starving other critical processes of CPU time. It should also achieve a graceful recovery from the failure state.

ISO 26262 applies to production passenger vehicles with a gross mass up to 3500 kilograms (7716 pounds). Anything else is out of scope. But while the scope is limited, the standard itself is comprehensive. It covers functional safety aspects of the entire development process, from requirements specification to product decommissioning. And in case you were wondering, it’s closely related to IEC 61508, the international safety standard with a very long history and which many other safety standards reference.

So why do I think that 26262 is on the ascent? For starters, the first edition of the standard was published less than a year ago, yet a silicon vendor has already spent the considerable effort to get an MCU certified. Achieving certification to a standard like ISO 26262 doesn’t come easy, so I assume Freescale did it only because they anticipate market demand. (Disclaimer: This statement isn’t based on any special knowledge of Freescale’s business, but is simply my opinion. Interpret it as such.)

TÜV Rheinland:
Also in the game

It doesn’t stop at Freescale. TÜV Rheinland, a global provider of technical services for safety-critical systems, now offers 26262 services (training, consulting, testing, certification, you name it) for a wide variety of automotive components in multiple geographies. And if TUV has gotten in the game, it’s a good signal that the 26262 standard has legs.

Meanwhile, the LinkedIn group dedicated to 26262 has more than 3600 members and grew by more than 50 members last week alone. If you visit the group, you’ll find engineers from automotive OEMs and tier ones looking for guidance on satisfying 26262 requirements — a sure sign that support for the standard is gearing up.

From what I can tell, things haven’t gotten to the point where a company has been mandated to have its automotive systems certified to ISO 26262. But it will happen. And chances are, it will snowball: the more companies that adopt the standard, the more others will feel the pressure and follow suit. Which means it’s only a matter of time before more ISO 26262 product announcements show up in my Google alerts.

A glaring look at rear-view mirrors

Some reflections on the challenge of looking backwards, followed by the vexing question: where, exactly, should video from a backup camera be displayed?

Mirror, mirror, above the dash, stop the glare and make it last! Okay, maybe I've been watching too many Netflix reruns of Bewitched. But mirror glare, typically caused by bright headlights, is a problem — and a dangerous one. It can create temporary blind spots on your retina, leaving you unable to see cars or pedestrians on the road around you.

Automotive manufacturers have offered solutions to this problem for decades. For instance, many car mirrors now employ electrochromism, which allows the mirror to dim automatically in response to headlights and other light sources. But when, exactly, did the first anti-glare mirrors come to market?

According to Wikipedia, the first manual-tilt day/night mirrors appeared in the 1930s. These mirrors typically use a prismatic, wedge-shaped design in which the rear surface (which is silvered) and the front surface (which is plain glass) are at angles to each other. In day view, you see light reflected off the silvered rear surface. But when you tilt the mirror to night view, you see light reflected off the unsilvered front surface, which, of course, has less glare.

Manual-tilt day/night mirrors may have debuted in the 30s, but they were still a novelty in the 50s. Witness this article from the September 1950 issue of Popular Science:



True to their name, manual-tilt mirrors require manual intervention: You have to take your hand off the wheel to adjust them, after you’ve been blinded by glare. Which is why, as early as 1958, Chrysler was demonstrating mirrors that could tilt automatically, as shown in this article from the October 1958 issue of Mechanix Illustrated:


Images: Modern Mechanix blog

Fast-forward to backup cameras
Electrochromic mirrors, which darken electronically, have done away with the need to tilt, either manually or automatically. But despite their sophistication, they still can't overcome the inherent drawbacks of rear-view mirrors, which provide only a partial view of the area behind the vehicle — a limitation that contributes to backover accidents, many of them involving small children. Which is why NHTSA has mandated the use of backup cameras by 2018 and why the last two QNX technology concept cars have shown how video from backup cameras can be integrated with other content in a digital instrument cluster.

Actually, this raises the question: just where should backup video be displayed? In the cluster, as demonstrated in our concept cars? Or in the head unit, the rear-view mirror, or a dedicated screen? The NHTSA ruling doesn’t mandate a specific device or location, which isn't surprising, as each has its own advantages and disadvantages.

Consider, for example, ease of use: Will drivers find one location more intuitive and less distracting than the alternatives? In all likelihood, the answer will vary from driver to driver and will depend on individual cognitive styles, driving habits, and vehicle design.

Another issue is speed of response. According to NHTSA’s ruling, any device displaying backup video must do so within 2.5 seconds of the car shifting into the reverse. Problem is, the ease of complying with this requirement depends on the device in question. For instance, NHTSA acknowledges that “in-mirror displays (which are only activated when the reverse gear is selected) may require additional warm-up time when compared to in-dash displays (which may be already in use for other purposes such as route navigation).”

At first blush, in-dash displays such as head units and digital clusters have the advantage here. But let’s remember that booting quickly can be a challenge for these systems because of their greater complexity — many offer a considerable amount of functionality. So imagine what happens when the driver turns the ignition key and almost immediately shifts into reverse. In that case, the cluster or head unit must boot up and display backup video within a handful of seconds. It's important, then, that system designers choose an OS that not only supports rich functionality, but also allows the system to start up and initialize applications in the least time possible.

Ontario tech companies team up to target the connected car

To predict who will play a role tomorrow's connected vehicles, you need to look beyond the usual suspects.

When someone says “automobile,” what’s the first word that comes to mind? Chances are, it isn’t Ontario. And yet Ontario — the Canadian province that is home to QNX headquarters — is a world-class hub of automotive R&D and manufacturing. Chrysler, Ford, General Motors, Honda, and Toyota all have plants here. As do 350 parts suppliers. In fact, Ontario produced 2.5 million vehicles in 2012 alone.

No question, Ontario has the smarts to build cars. But to fully appreciate what Ontario has to offer, you need to look beyond the usual suspects in the auto supply chain. Take QNX Software Systems, for example. Our roots are in industrial computing, but in the early 2000s we started to offer software technology and expertise to the world’s automakers and tier one suppliers. And now, a decade later, QNX offers the premier platform for in-car infotainment, with deployments in tens of millions of vehicles.

QNX Software Systems is not alone. Ontario is home to many other “non-automotive” technology companies that are playing, or are poised to play, a significant role in creating new automotive experiences. But just who are these companies? The Automotive Parts Manufacturers Association (APMA) of Canada would like you to know. Which is why they've joined forces with QNX and other partners to build the APMA Connected Vehicle.

A showcase for Ontario technology.

The purpose of the vehicle is simple: to showcase how Ontario companies can help create the next generation of connected cars. The vehicle is based on a Lexus RX350 — built in Ontario, of course — equipped with a custom-built infotainment system and digital instrument cluster built on QNX technology. Together, the QNX systems integrate more than a dozen technologies and services created in Ontario, including gesture recognition, biometric security, emergency vehicle notification, LED lighting, weather telematics, user interface design, smartphone charging, and cloud connectivity.

Okay, enough from me. Time to nuke some popcorn, dim the lights, and hit the Play button:

Seminar: managing the growing amount of software in cars

It’s no secret that the amount of software in automobiles is growing rapidly — as is the challenge of maintaining it reliably and efficiently. At QNX Software Systems we focus on areas like infotainment, telematics, clusters, and ADAS, but our long-term FOTA partner, Red Bend Software, takes a more holistic view, working with companies like Vector Informatik to extend FOTA all the way down to ECUs.

To help automakers and tier one suppliers manage their software deployments more efficiently, Red Bend is hosting a seminar Friday September 27 at the Westin Southfield Detroit. Speakers will include representatives from Strategy Analytics, Texas Instruments, and Vector, not to mention our own Andy Gryc. You can register on the Red Bend website.

Will adaptive cruise control spell the end of traffic jams?

Did you know that rear-end collisions account for about 30% of car crashes? For that reason alone, widespread adoption of adaptive cruise control (ACC) can’t come too soon. ACC helps prevent such collisions in two ways: 1) by maintaining a safe, preset distance from the car ahead; and 2) by applying the brakes quickly if that car comes to a sudden stop — more quickly, in fact, than most humans.

Good news is, ACC may soon become pervasive. The folks at Global Industry Analysts crunched some numbers and determined that annual installations of ACC systems will reach 6.9 million units by 2017.

Mind you, ACC isn’t just about safety; it’s also about traffic flow. For instance, a study by Suzuki and Nakatsuji (2003) suggests that travel times shrink significantly when at least 20% of vehicles on the road use ACC. And a study by Kesting et al. (2008) suggests that, in some scenarios, traffic congestion simply disappears when 25% of vehicles use ACC.

Example of adaptive cruise control
Source: Volvo

The picture isn’t all rosy, however. ACC may improve traffic flow, but not in every situation, such as merging from an on-ramp onto a freeway. That said, a study by L. C. Davis (2010) suggests that a technique called cooperative merging can significantly the improve the performance of ACC in this scenario. Meanwhile, a study by Jerath and Brennan (2010) suggests that the benefits associated with ACC may come at a possible cost — “self-organized” traffic jams. This effect, caused mostly by human behavior, may occur in a traffic system where most, but not all, cars use ACC.

Caveats aside, ACC systems continue to evolve. Some drivers tend to slam their brakes and use heavy throttle in traffic, creating congestive shockwaves that ripple down the highway. According to J.C. Power, newer versions of ACC help alleviate this problem by smoothly modulating brakes and throttle in stop-and-go traffic.

And now, a look at ACC from 1939…
If you think the concept of ACC is relatively new, think again. Over 70 years ago, GM created a “Futurama” exhibit for the 1939 World's Fair that showcased a scale-model highway in which cars automatically maintain a safe, efficient distance from one another.

GM predicted this technology would be in place by 1960. They got the timing wrong, but the idea right. Click the video to see a surprisingly prescient look at the car of the future — I’ve already bookmarked the spot for you:



What about you? Have you had much experience with ACC? And if so, has it helped or hindered your driving experience?

Some forward-thinking on looking backwards

The first rear-view camera appeared on a concept car in 1956. It's time to go mainstream.

Until today, I knew nothing about electrochromism — I didn’t even know the word existed! Mind you, I still don’t know that much. But I do know a little, so if you’re in the dark about this phenomenon, let me enlighten you: It’s what allows smart windows to dim automatically in response to bright light.

A full-on technical explanation of electrochromism could fill pages. But in a nutshell, electrochromic glass contains a substance, such as tungsten trioxide, that changes color when you apply a small jolt of electricity to it. Apply a jolt, and the glass goes dark; apply another jolt, and the glass becomes transparent again. Pretty cool, right?

Automakers must think so, because they use this technology to create rear-view and side-view mirrors that dim automatically to reduce glare — just the thing when the &*^%$! driver behind you flips on his high-beams. Using photo sensors, these mirrors measure incoming light; when it becomes too bright, the mirror applies the requisite electrical charge and, voilà, no more fried retinas. (I jest, but in reality, mirror glare can cause retinal blind spots that affect driver reaction time.)

So why am I blabbing about this? Because electrochromic technology highlights a century-old challenge: How do you see what — or who — is behind your car? And how do you do it even in harsh lighting conditions? It’s a hard problem to solve, and it’s been with us ever since Dorothy Levitt, a pioneer of motor racing, counseled women to “hold aloft” a handheld mirror “to see behind while driving.” That was in 1906.

Kludges
For sure, we’ve made progress over the years. But we still fall back on kludges to compensate for the inherent shortcomings of placing a mirror meters away from the back of the vehicle. Consider, for example, the aftermarket wide-angle lenses that you can attach to your rear window — a viable solution for some vehicles, but not terribly useful if you are driving a pickup or fastback.

Small wonder that NHTSA has ruled that, as of May 2018, all vehicles under 10,000 pounds must ship with “rear visibility technology” that expands the driver’s field of view to include a 10x20-foot zone directly behind the vehicle. Every year, backover crashes in the US cause 210 fatalities and 15,000 injuries — many involving children. NHTSA believes that universal deployment of rear-view cameras, which “see” where rear-view mirrors cannot, will help reduce backover fatalities by about a third.

Buick is among the automotive brands that are “pre-complying” with the standard: every 2015 Buick model will ship with a rearview camera. Which, perhaps, is no surprise: the first Buick to sport a rearview camera was the Centurion concept car, which debuted in 1956:


1956 Buick Centurion: You can see the backup camera just above the center tail light.

The Centurion’s backup camera is one of many forward-looking concepts that automakers have demonstrated over the years. As I have discussed in previous posts, many of these ideas took decades to come to market, for the simple reason they were ahead of their time — the technology needed to make them successful was too immature or simply didn’t exist yet.

Giving cameras the (fast) boot
Fortunately, the various technologies that enable rear-view cameras for cars have reached a sufficient level of maturity, miniaturization, and cost effectiveness. Nonetheless, challenges remain. For example, NHTSA specifies that rear-view cameras meet a number of requirements, including image size, response time, linger time (how long the camera remains activated after shifting from reverse), and durability. Many of these requirements are made to order for a platform like the QNX OS, which combines high reliability with very fast bootup and response times. After all, what’s the use of backup camera if it finishes booting *after* you back out of your driveway?


Instrument cluster in QNX technology concept car displaying video from a backup camera.

Drivers want ADAS, but not so sure about autonomous cars: study

In May, market researcher Penn Schoen Berland canvassed 2,506 American drivers about their driving habits. The findings, presented last week at a Ford press conference, are sobering:

• 76% of respondents admitted to eating or to drinking non-alcoholic drinks while behind the wheel
• 53% admitted to talking on a handheld phone
• 33% admitted to fiddling with their mobile gadgets
• 55% admitted to driving beyond the speed limit
• 37% admitted to driving when too tired

And here’s the kicker: 99% of respondents claimed they were safe drivers.

I know, it's a major disconnect. But here's what I find interesting: most respondents also expressed interest in driver assistance systems. In other words, even self-proclaimed safe drivers tacitly admitted they could use help now and then. For instance:

• 8 out of 10 respondents expressed interest in technologies that would help them stay in their lane
• 9 out of 10 expressed interest in technologies that could detect an impending collision and slow the car down

Respondents also expressed interest in systems that could detect a car in their blind-spot, provide voice-activated phone dialing, or park the car automatically. That said, only 39% said they’d feel comfortable riding an autonomous car.

My take? That number will grow significantly once more people drive cars equipped with adaptive cruise control, automatic parallel park, and other driver-assist systems. The more people become accustomed to such systems, the more they'll accept a car that does most of the driving for them.

For media coverage of this study, visit Forbes, Scientific American, and the Wall Street Journal.