Deep-Dive: Behind the scenes at Toyota's R&D center, Part Two

It's impossible to overlook the negative impact the recalls have had on Toyota's reputation for safety and reliability. Seemingly overnight, the automaker that could do no wrong has been brought back down to earth.

Sales of most of Toyota's lineup have been hit hard, with only fleet sales and big incentives keeping the retail side from looking worse. As we first told you yesterday in Part One of our series, in a move unprecedented in its history, Toyota has opened its doors and invited a small group of journalists into its product development facilities in Toyota City near Nagoya, Japan.

The goal is to show the lengths Toyota goes to test and evaluate its products both before and after they go on sale to the public, as well as to detail how those testing regimes have been altered in wake of their recent troubles. Given the sudden intended acceleration scandal, Toyota has chosen to focus on the work it does to verify the performance and reliability of its electronic throttle control systems. Make the jump for a primer on Toyota's engineering and manufacturing processes and how it's attempting to ensure that sudden acceleration will be snuffed out for good.

What is an electronic throttle control system?

BMW was the first automaker to utilize an electronic throttle system (ETCS) when it introduced the V12-powered 750i in the late 1980s. Over the past decade, ETCS has become almost universal in the industry as an important element of meeting increasingly difficult emissions and fuel consumption standards.

Prior to the advent of ETCS, a steel cable provided a physical connection between the accelerator pedal and the butterfly valve in the throttle body (and before that, the carburetor). When you look at a modern engine, you'll find a servo motor mounted on the side of the throttle body that physically opens and closes the plate. The accelerator pedal now contains a pair of position sensors that detect how far the driver has presses down and an electrical connection to the engine management system. The EMS contains software that reads the pedal position and sends a signal to the electronic throttle motor telling it how far to open.



Therein lies the (potential) problem. A software error or electronic fault has the potential to open the throttle independently of the driver's request. This is exactly what people claim has happened, what caught NHTSA's eye and spurred Toyota's unintended acceleration recall. Unfortunately for Toyota, there is no physical evidence to back up these claims, and Toyota maintains that this phenomenon remains impossible to recreate, going so far as attempting to prove it to us during our visit.

Regardless of whether you're talking about the most basic transportation in the world (think: Tata Nano) or an advanced hybrid or electric vehicle, it would be impossible to meet the often contradictory requirements of customers and regulators without electronics and software. As the capabilities of electronic systems have increased, so, too, have the complexity of the interactions in these systems. Developing robust electronic control systems requires endless testing at every level, from the earliest software-in-the-loop simulation to full vehicle-in-the-loop evaluation.




The engine management system alone consists of some 800,000 lines of "C" code split into 1,600 functional modules. Like most manufacturers today, Toyota is using software development tools like Matlab and Simulink to model functions and test them before ever generating a single line of code. Just as simulation is used for developing crash structures, mathematical models of the vehicle and powertrain components are used to check out the software before prototype electronics are produced.

The most basic level of testing is software-in-the-loop (SIL), where logic models are created with the various inputs and outputs to test the system. The engineers can then exercise these models on the desktop through countless iterations that can be easily reproduced while tweaking the models. Once some prototype hardware is ready for testing, engineers can use tools like Simulink to automatically generate computer code that's then loaded into the ECU.



The ECU will be plugged into various levels of simulation hardware. This typically begins with plugging into a computer running the math models of the rest of the hardware and then progressing through various levels with increased hardware integration. This is the hardware-in-the-loop (HIL) testing. Like SIL, the testing can be automated to run tens or hundreds of thousands of iterations and – ideally – go down every possible execution path in the code. HIL testing also includes bench testing on the dynamometers with complete powertrains.

The video meant to be presented here is no longer available. Sorry for the inconvenience.

The next stage is the vehicle-in-the-loop (VIL) evaluations where ECUs, sensors and the rest of the system are installed in the car. The headquarters' technical center in Toyota City has 12 different buildings were engines are tested on 110 engine dynamometers and 60 chassis dynamometers. We began our tour in Building 12, which is filled with chassis dynamometers for testing full vehicles both at ambient temperatures and in thermal chambers at temperatures ranging from -58 degrees to 113 degrees Fahrenheit. They also have low-pressure chambers that can simulate altitudes up to 14,000 feet. The vast majority of the testing that takes place in these labs is related to emissions and fuel economy evaluation as well as durability testing.



For our tour, Toyota set up a Lexus ES350 on one of its open chassis dynos and rigged it with cameras so we could see the engine bay, instrument cluster, down the throttle body, across the pedals at the driver's feet as well as at engine data. A breakout box was plugged in to simulate various failure modes such as failed position sensors on both the throttle body and accelerator pedal.

The technicians ran through a series of tests with the vehicle running at 40 MPH to see what would happen if the sensors failed. As you might expect, in every case the system detected a fault and either limited the throttle to very small openings (limp-home mode) or shut the engine down completely. One case where the engine was immediately shut down: a stuck throttle. This was tested by sticking a long steel spring down the throttle body after the driver floored the accelerator. With the spring in place, the driver released the throttle. When the accelerator pedal detected no application yet the throttle position showed it open, a fault was detected and the engine was shut down.



Another demonstration included a failed processor in the engine management system. Most modern electronic control units feature a primary CPU and a secondary monitor CPU. The primary processor reads all of the sensor signals and contains the control algorithms that process the signals and determine what should be done with the throttle, fuel injectors and other related systems. The monitor CPU also reads the sensor signals but processes them using separate code with different algorithms. The two CPUs swap data every control loop (typically 100-200 times per second) and if the numbers don't agree, an ECU fault is set and the engine is shut down.

Until now, there were two main types of fault codes: pending and full fault codes. A pending code would be set for intermittent faults. Depending on the type of fault, the failure would have to occur a certain number of times within a specified period of time. If the fault did not repeat, the pending code would automatically be cleared. Other faults were set immediately and then stored in memory until cleared by a technician. For 2010, a new class of permanent codes has been added to help detect some of the scenarios that could potentially lead to unintended acceleration. These fault codes cannot be cleared, and if unintended acceleration occurs, these faults will be recorded in the ECU.



Adjacent to the dyno was a bench testing unit consisting of fuel injectors, throttle body and accelerator pedal. This unit is typically used for dealer training purposes to teach technicians how to diagnose fuel system issues. Among the features are terminals that allow for the grounding or shorting of the various connections in the wiring harness. This was used to demonstrate the infamous "no-fault" stuck-throttle condition demonstrated earlier this year by professor David Gilbert on ABC News. You can read all about how his assertions were debunked here.


Testing for Electromagnetic interference

From the dyno facility, we headed over to an electromagnetic compatibility (EMC) test chamber, where engineers work to ensure that automobiles are not interfered with by outside electromagnetic activity (think: cell phone towers, radio transmitters, garage door openers, etc.). Toyota has eight of these chambers in Toyota City and is currently constructing a ninth example at its York Township tech center outside of Ann Arbor, MI. The largest chambers measure over 34 meters by 23 meters by 11.4 meters high, and their walls are lined with 3,000 styrofoam absorption panels to soak up stray electromagnetic waves. The chamber includes a four-wheel chassis dynamometer embedded in a turntable that allows two or four-wheel-drive cars to be tested at speed. The car is surrounded by antennas that can bombard it with radiation at frequencies from 20 megahertz to 20 gigahertz.



Toyota runs its EMC testing at field strengths of 60 volts/meter, significantly higher than those required by current regulations. At these elevated levels, humans can't be in the chamber during the testing, lest they be turned into microwave popcorn. A plastic (to avoid interference problems) robot is installed in the car to apply the gas and brakes and remotely managed from the control room. In addition to these large-scale tests to simulate driving by the likes of radio and cell towers, other tests include scanning the car with a variety of lower power (2-15 watt) antennas and looking for a variety of anomalies ranging from engine shutdown to warning lamps and, of course, unintended acceleration. Naturally, all of the various individual electronic components are tested in smaller chambers by both Toyota and its suppliers to make sure they don't respond negatively to radiation or give off electromagnetic waves that could harm other systems.

The video meant to be presented here is no longer available. Sorry for the inconvenience.

As we all know, it's impossible to conclusively prove a negative. And just because these tests run by Toyota have failed to turn up a fault related to the unintended acceleration claims, the phenomenon can't be categorically disproved. On the other hand, life – and in particular, business – is all about risk management. The degree of testing conducted by Toyota and every other automaker is intended to cover as broad a swath of scenarios as possible. Given the range of tests that are conducted at every level, it's possible to have a sufficiently high degree of confidence (99 percent or more) that a design will work as intended.



The real difficulty is creating a suite of tests to cover the widest possible range of conditions within reason. Engineers have to anticipate anything that can reasonably happen and test for it. That's where Toyota's new "neutral stance" approach discussed in our last installment may help. By listening more to the voices of its customer when problems happen and incorporating some of the anomalous real-world behavior (like stacked floor mats) into its product design plans, Toyota hopes to create even more robust designs in the future.



Every automaker and supplier needs to learn from what has happened to Toyota. All of the big companies have similar testing capabilities, and although their scales may differ, it ultimately comes down to what you do with the tools. Now more than ever, Toyota has a comprehensive range of tools at their disposal, but their smartest move yet may be paying closer attention to what their customers are saying.

The video meant to be presented here is no longer available. Sorry for the inconvenience.