Should You Trust Customer Satisfaction Awards?

You can hardly watch a TV commercial about cars without hearing about how some car or manufacturer won a J.D. Power award. Car manufacturers have embraced customer satisfaction index (CSI) surveys and have crowed about their awards for years. Boat and engine manufacturers have begun to more aggressively promote similar surveys as well, so let’s take a dive into what they are, how those ubiquitous CSI awards are made, and if they are of value to you, the boater.

If you haven't seen one yet, CSI surveys are feedback from buyers (of almost any product or service) that tell companies how buyers perceive their product’s or service’s quality. The goal is happy (and repeat) buyers, as well as the feedback necessary to make a better product. Since 2003, many boatbuilders have been surveying their buyers through a company called AVALA Marketing, a third-party business that works closely with builders and dealers that are a part of the largest industry trade group, the National Marine Manufacturer's Association (NMMA). In fact, all manufacturers that certify their boats through NMMA (which is about 70% of all new boats sold) are required to survey their buyers (an exception is for those who build fewer than 25 boats per year).

While some do it themselves, many use AVALA's services. The goal, according to Steve Pizzolato, director of strategic marketing and former owner of AVALA, is to measure and improve the experience of a new buyer's boat and dealer. Robert Newsome, NMMA's senior vice president of strategy, engineering standards, and membership, said that it was the manufacturers who asked for the program. "Remember when people would say that the two best days of boat ownership was the day you bought, and the day you sold your boat? They wanted to change that by improving their products."

Not All CSI Surveys Are Alike

The J.D. Power awards you frequently hear about tend to dig pretty deep into specific areas, but their sample size can be small with as few as 100 respondents, potentially a tiny fraction of overall buyers. (See "Why No J.D. Power Awards For Boats?" below.) The firm typically ranks manufacturers based on problems per 100 units sold, which often makes only a small number of models "winners"implying that there are "losers."

NMMA surveys, according to Pizzolato, are not quite as deep as J.D. Power's, but cast a wider net. NMMA manufacturers use a census-type survey, which means it is required to send surveys to every buyer. While it often takes substantial follow up, the rate of returned surveys, he says, is as high as 45% to 50%; anyone who's been involved in surveying will tell you that is remarkable.

The surveys ask such questions as the satisfaction level of the dealer experience, initial boat and engine quality, and overall satisfaction. Months later, a follow-up survey asks similar questions as well as more about such things as boat handling, noise and vibration, and safety. Buyers typically score their answers from 1 (extremely dissatisfied) to 5 (extremely satisfied). Builders who get a 90% or higher score earn the Marine Industry CSI award, something you'll likely see at a boat show, in a dealer's showroom, and in their marketing. Pizzolato says that while manufacturers pay a substantial amount for the surveying service, the only additional costs are if the manufacturer wants to buy a fancy award to display, unlike the J.D. Power program, where there is a substantial fee to mention the award in advertisements.

Newsome says that the surveys give manufacturers robust information direct from customers, which helps improve boats. "The surveys drive change, even across models. If customers do or don't like something in one model, that can be addressed in that model and in others as well." Both manufacturers and their dealers get the results of the surveys, which can be used to modify designs and work on problem areas.

Pizzolato says that recently the surveys have begun to focus more on the dealer's service quality, asking such things as how well problems were dealt with, frequency of having to return for the same problem, and whether schedules for service were adequate. That's particularly important to manufacturers, he says, because a poor dealer experience overrides brand loyalty. "Buyers will simply go to another brand rather than having to travel farther to go to a different servicing dealer if their local one can't perform," he says.

In fact, Pizzolato says that the dealer experience is what has improved most since NMMA began including those questions in the surveys. "More manufacturers than ever are collecting data on a buyer's post-sale experience, including satisfaction at the dealership," he says. Builders are sensitive to how consumers react to their after-the-sale experience because it's mostly out of their control and is one of the biggest factors in repeat buying, he says.

Newsome adds that some dealers incentivize their employees in hopes they can earn the coveted award, further improving a customer's experience. The CSI award is a big deal for dealers, he says. "Brand loyalty is critical," says Newsome, "but dealers also want the recognition that they're doing a great job."

Can CSI Awards Help New Boat Buyers?

We think so. First, the more buyers who respond (honestly) to the surveys, the better handle manufacturers and dealers have on where to focus on problem areas, which hopefully translates into better boats and service. Second, boat and engine manufacturers frequently contact unhappy buyers as a result of the survey to try to resolve problems with new boats before they escalate. And third, NMMA awards allow consumers to compare brands and models. If a brand or model has an NMMA award, that means that it scored 90% or higher on CSI surveys and should be a better bet for consumers, especially if the builder earns an award year after year.

That said, just because a model doesn't have an award doesn't mean it's not a good boat. And small or new builders will have a smaller sample size, which can skew results. Non-NMMA builders can’t participate.