July 21, 2011
Alice and Randy Whitfield of Quality Control Systems Corp. have released a new analysis for Safety Research & Strategies that examines the statistical underpinnings of the NHTSA and NASA reports on Toyota Unintended Acceleration which shows that the agencies based their conclusions about the possibility of an electronic cause on a series of unsupportable suppositions, miscoded data and secret warranty data reported by Toyota’s litigation defense experts, Exponent.
In April 2010, the National Highway Traffic Safety Administration hired the National Aeronautics and Space Administration’s NASA Engineering Safety Center (NESC) to conduct a hunt for possible electronic causes of unintended acceleration in Toyota vehicles. The two agencies’ final reports, released in February, declared that it was unlikely that electronics were at the root the many unintended acceleration complaints. In May, Safety Research & Strategies published its response to the federal government effort in a special edition of The Safety Record: An Examination of the National Highway Traffic Safety Administration and the National Aeronautics and Space Administration Engineering Safety Center Assessment and Technical Evaluation of Toyota Electronic Throttle Control (ETC) Systems and Unintended Acceleration.
SRS’s report noted that the most remarkable discovery of the NHTSA-NASA Toyota Unintended Acceleration investigation was the presence of tin whiskers on every potentiometer-type accelerator pedal they examined. Tin whiskers are crystalline structures, many times thinner than a human hair, which form on the tin solder used on printed circuit boards. They are mighty inconvenient for electronics manufacturers, having been known to produce all kinds of varied and unpredictable electronic malfunction such as shutting down nuclear reactors, satellites and medical devices – to name a few examples.
They proved to be mighty inconvenient to NHTSA-NESC team, too. Not only did they discover the presence of tin whiskers on every potentiometer pedal examined, the researchers actually found them on a pedal tied to a vehicle that had experienced multiple instances of UA. With one promising root cause right there on the lab bench, most scientists who practice their vocation with rigor might have expanded their sample beyond several pedals to determine incidence in the field. They might have looked for tin whiskers in the throttle bodies or ECMs – other components of the engine system that could lead to a throttle malfunction. Instead, NHTSA turned to Exponent, the science-for-hire firm Toyota’s legal team retained to defend the company in class action lawsuits.
Neither agency disclosed this conflict-of-interest. NHTSA claimed that it performed the warranty analysis.
The Whitfields, unpacked the data supporting the alleged Trouble-Not-Found conclusions and they were particularly struck by this:
“It is also extraordinary that the NASA-NESC team, as safety experts, would look upon secret, warranty data reported by a manufacturer’s litigation experts as evidence of the lack of corroboration of an electronics cause of unintended acceleration. This is especially true because the compromised, safety-critical, electronic circuitry was discovered in an accelerator pedal sensor assembly examined in a NASA laboratory… Only NASA’s and NHTSA’s analysis of the complaint data in relation to the secret, warranty data is cited to support the lack of relevance of these findings to public health and safety… It is difficult to imagine NASA itself accepting assurances from a manufacturer of its own spacecraft that similar problems in important safety systems should be regarded as inconsequential. Yet it is the NASA-NESC report on which NHTSA relied to close its investigation,” the Whitfields wrote in their report.
The Quality Control Systems Corp. report also lays bare the extraordinary approach NHTSA and NESC employed when looking at the numbers. Simply put, government researchers postulated that an electronic malfunction that could cause a UA not detected by the engine’s diagnostic system would most likely be the result of two simultaneous faults. A single fault in the electrical system would most certainly be noted by the engine control module and set a Diagnostic Trouble Code (DTC). They further postulated that single faults would be more common that double faults. There are many driving conditions under which drivers have reported UAs, but this study focused on large throttle openings, accompanied by degraded or ambiguous braking.
Then, they looked at two pools of data – consumer UA complaints lodged with NHTSA via its Vehicle Owner Questionnaire (VOQ). This data represented the double-fault, no DTC scenario. The second pool was composed of warranty claims. This represented the single-fault, DTC-set scenario.
The NHTSA-NESC team established a rule for their analysis: if the number of warranty claims was greater than the number of VOQs, this would indicate that electronics was a root cause. But if more consumers complained to NHTSA about Toyota UA than received a warranty repair for an accelerator pedal-related problem, then that would be proof that electronics was not a cause.
Neither agency looked at reporting rates, which would have given context to summary counts. Since this rule was a straight-up computation exercise using raw numbers, the amount that accumulated in each data pool was critical to the result.
The Whitfields attempted to examine the specific VOQ dataset NHTSA used in this analysis. But, only five months after publishing what was billed as the most exacting defect study the agency has ever performed in its history, NHTSA responded to the Whitfields that they didn’t retain the set of Office of Defects Investigation (ODI) numbers assigned to each VOQ. The agency did agree to try to replicate it for the Whitfields, and an examination of the VOQs provided yielded an interesting observation: NHTSA inflated the pool of VOQ complaints by including VOQs in which braking was reportedly effective.
The Whitfields also questioned agencies’ use of consumer narratives to determine precise throttle openings and they raised doubts that throttles could have been wide open, when drivers report no problems bringing their vehicles to a stop with the brakes.
So, we have inflated complaint counts. What about the warranty claim counts? Here is where everything comes full circle. In April 2010, when Toyota was crouched in a defensive posture while U.S. Rep. Henry Waxman and the press ensured that Toyota had a Terrible, Horrible, No-Good, Very Bad Day, every day, The Safety Record Blog published a week’s worth of posts (see Manufacturing Doubt in Toyota Sudden Unintended Acceleration) based on OSHA director and Assistant Secretary of Labor David Michaels’ book: Doubt is Their Product: How Industry’s Assault on Science Threatens Your Health. We wrote admiringly of Michaels documentation of industry artfully draping its interests with science’s mantle. How they use firms like Exponent to come up with “scientific” results to help corporations and entire industries beat plaintiffs in lawsuits or cowl the government into dialing back regulation. Michaels wrote:
“Their business model is straightforward. They profit by helping corporations minimize public health and environmental protection and fight claims of injury and illness. In field after field, year after year, this same handful of individuals and companies comes up again and again. The range of their work is impressive. They have on their payrolls or can bring in on a moment’s notice toxicologists, epidemiologists, biostatisticians, risk assessors the work has one overriding motivation: advocacy for the sponsor’s position in civil court, the court of public opinion and the regulatory arena.”
“Exponent’s scientists are prolific writers of scientific reports and papers. While some might exist, I have yet to see an Exponent study that does not support the conclusion needed by the corporation or trade association that is paying the bill.”
Toyota has followed the playbook well. What remains stunning is NHTSA and NASA’s participation in the process and their willingness to co-opt their own reputations.
Click on the thumbnail below to see the full report.