Earlier this week we told you about a pickup truck comparison in the most recent issue of Consumer Reports that seemed a bit fishy. The comparison pitted the new 2007 Toyota Tundra against the 2007 Chevy Silverado, Ford F-150 and Dodge Ram. It was clearly a fight between the Tundra and Silverado from the get-go, since both were the only two completely new trucks in the test. The Tundra, however, seemed to have an advantage in that it was ordered up with the larger of its two engines, the 5.7L V8 producing 381 hp. The Silverado, while available with a more evenly matched 6.0L Vortec MAX V8, was ordered with a less powerful 5.3L V8 producing 315 hp. Not only that, but the Tundra benefited from a 4.30 rear axle ratio that provides better towing and acceleration than the 3.73 ratio in the Silverado, which sacrifices those qualities for better fuel economy. The Silverado, however, could've been ordered with a more comparable 4.10 rear axle ratio at no extra charge.
Well, the gentlefolks at CR recently posted an answer to everyone's questions about this particular half-ton pickup comparo on their blog, which you should go read by clicking here before going on. They explain that choosing equipment for vehicles involved in a comparison is a tricky thing that involves balancing the objective of several goals.
"In general, we want to test a representative vehicle that is comparable to other vehicles in the test group (and previously tested peer vehicles). We also typically test the version--powertrain and trim level--that most regular consumers will buy. "
Read on after the jump to hear our take on CR's explanation.
At face value, it seemed to us that the cards were stacked in the Tundra's favor.
It appears to us that CR failed in terms of acquiring vehicles that were comparable to each other. We recognize the difficulty that's presented with the new Tundra, since Toyota offers significantly fewer configurations than do Chevy, Ford and Dodge. Nevertheless, there are other configurations of the Silverado that would have better matched the Tundra and likely led to the Silverado scoring higher. We admit, the Silverado may not have won the comparo even if it had been configured to better match the Tundra, as Toyota's powertrain is particularly strong and fuel efficient. In the end, however, we believe the consumer would've been better served by reading about an evenly-matched contest.
On CR's second point, that it typically tests versions of vehicles that most regular consumers will buy, we concede that is a good strategy if the plan is to offer a review that will benefit the largest number of consumers. That's fine if a single vehicle is reviewed, but totally inappropriate for a comparison test. As a consumer, why would I want to read a comparison test of trucks that aren't similar? It would like reading about the Honda Civic versus the Saturn Aura. Comparison tests, at least to us, are not about comparing what people buy, they're about advising what people should buy based on an equal comparison.
CR also gave the Tundra a predicted reliability rating of Very Good based on the reliability of past Tundras and Toyotas in general. The Silverado was labeled as too new to predict its reliability. In our eyes, the Tundra should have also been labeled as too new to predict its reliability, considering it is an all-new model built at an all-new assembly plant in San Antonio, TX. Mechanically speaking, the Tundra of today is completely different than the previous Tundra on which CR's reliability scores were based. The Tundra has also suffered 20 cases of reported camshaft failures in models equipped with the same 5.7L engine CR tested.
We still have a lot of respect for the hardworking people at Consumer Reports and value their opinion, but in the case of this half-ton pickup comparo, we believe its value is limited.