Scales in market research share a similar purpose with those devices found in bathrooms, gyms and doctors offices. They are both designed to measure. In one case it may be our physical weight, while in the other it may be the importance of an attribute or attitude. In the context of B2B and consumer market research the literature is deep on best practices for scale construction. Some of the obvious areas to consider include:
- Number of items – multiple item scales have greater reliability than single-item measures and should be considered where feasible.
- Number of categories – 5 and 7-point scales are typical for market research, although 9, 10 and 11-point scales are increasing in popularity. We have to balance the increased granularity of higher point scales with increased cognitive load placed on survey participants.
- Direction of the scale points – Should positive values be on the left or right?
In addition to the words we choose and the number of scale points it is wise to consider the visual layout of the scale. Online survey platforms, such as Cvent, allow the survey developer options when it comes to displaying scales. The example below comes from a hotel guest survey. The actual page included two additional questions regarding willingness to recommend.
The items used to create this satisfaction measure incorporate a 10-point scale with positive values on the left, as well as descriptive headers above the scale points. The use of descriptive headers, which happens to work well with a 10-point scale, is a best practice as it minimizes potential for confusion. However, starting the scale with the most positive values could lead toward upwardly biased values. This might be good for an executive whose bonus relies upon satisfaction numbers, but it may also yield an inaccurate picture.
This issue can be addressed using a bit of simple experimental design. Mirrored versions of the survey can be run where potential respondents are randomly assigned to either a high number first design (10 to 1) or the reverse. After data collection is complete the research analyst can test for significant differences. If no differences are noted then use of the 10 to 1 design is accepted.
Another concern is the mixing of different concepts on a page. This is more an issue of survey flow, but mixing satisfaction, with value, with recommendation, may confuse the respondent. Consider allocating these concepts to their own pages.