Bernheim- Part 3

I've posted a bit about the visitors’ survey used at Bernheim Arboretum and Research Forest.

Now let’s take a look at some of the nuts-and-bolts of the survey design.

[Again, I’m a big fan of Bernheim –literally, in terms of Facebook– and as always, our only goal is to foster conversations about customer research so that we can all learn and improve.]

Question Validity

Several of the questions use the following scale:

• Beyond Expectations
• Good
• Satisfactory
• Unsatisfactory

The biggest concern is the term “expectations.” Again, based on years of validating surveys by having customers take them and then talking to them about the survey-taking experience itself, we know that there are severe issues with any response category using the term “expectations.”

First, expectations are different for every person. Maybe someone had never been there and had no expectations at all. So they automatically could never check the highest category in this scale, since Bernheim can’t exceed expectations that didn’t exist.

Alternatively, maybe someone had very, very high expectations based on word-of-mouth, in which case there’s nowhere to go but down.

So really, the only people who would ever check “beyond expectations” are those who had low expectations to begin with.

Finally, there’s a consistency issue with the entire scale. Many respondents are not clear about the conceptual difference between “good” and “satisfactory.” And “beyond expectations” adds another level of confusion.

So here’s suggestion for a cleaner, simpler, more valid scale:

• Superb
• Acceptable
• Needs improvement

Until next time!

Update: read Part 1 & Part 2.