Create a Ranking Survey + Free Template

Definition: A ranking survey measures preferences by having respondents rank a list of attributes. A ranking question requires participants to rank all items or just a set number—like their top three. The results typically include a ranking score, first-place counts, and a distribution showing how each item performed.

A ranking question is also referred to as an ordinal-polytomous survey question. Ordinal meaning relative to an order and polytomous meaning more than two possible variables.

Crafting the list of attributes is the most important step in designing a ranking survey. Each item should be unique, clearly worded, and not too similar to the others. Keep the phrasing simple. If an option includes too many words or technical terms, it becomes harder for respondents to evaluate and rank the choices.

Avoid including too many attributes because it can overwhelm respondents and reduce the quality of the results. A list of more than 10 items demands excessive cognitive effort, making it hard for people to compare and order each. Generally, ranking questions that require more than ten (10) attributes should be using a MaxDiff question.

Consider your survey’s goals and how the ranking data will drive you toward them—supplemental questions sharpen this focus. They add additional data points that you can use when analyzing results. For example, if doing a market research study, you may want to include a question about household income, then you can segment preference by income.

Finally, assess the stakes of your survey—whether it’s for critical business decisions or casual insights. Based on your goals, a ranking question might not be the best option, and alternative methods could yield sharper, more relevant data. An alternative could be used, which is explained further in the article.

Once your ranking question is drafted, add it to the survey where you see fit. Depending on how many supplemental questions you have, we recommend placing the ranking question toward the beginning of your survey, ideally on page one, and then adding supplemental questions on page two. This way, if respondents get to page two and exit the survey, you still capture the critical ranking data.

If you're using a survey panel, you may have no choice but to put supplemental questions on page one and the ranking question after. In this case, people are being compensated for their responses, so drop-off rates are less of an issue.

When you add a ranking question to your survey there are some options you can toggle:

• Require an answer (can not proceed until answered).
• Define how many attributes you'd like to rank, such as the top three (3) or five (5).
• Clear the selected choices so respondents can start over.

There are two survey ranking question types you can choose from: click ranking or drag and drop. Below are the detials of each along with interactive examples. Depending on your project one type might be more beneficial than others. Both types of ranking questions display the results the same.

Click-to-rank questions are perfect for quickly determining what matters, like in a customer feedback survey. This ranking question lets respondents order preferences with a tap, delivering fast, actionable insights without dragging options.

This type is ideal when you know your audience will mainly be on mobile devices. Scrolling a large list and dragging on a mobile device may not be an ideal option, leading to user frustration and possibly drop-off rates.

This type is also ideal when lists are smaller, like five to ten items, and you're asking to rank a top three.

A drag-and-drop ranking question is ideal when you're doing more advanced research, and respondents may need to take more time evaluating options. Drag-and-drop makes it easy to reorder and visualize preferences.

This type is ideal when you know your audience will mainly be on desktop devices, as it is much easier to drag and drop, especially large lists, with a larger screen and mouse.

This type is also ideal when lists are more extensive, roughly ten items or more.

You can create skip and display logic rules based on ranking questions. Ranking logic is essential for market research surveys to ask follow-up questions. For example, you could use display logic to ask a Gabor Granger for each item a respondent ranked, giving you further insights into the monetary value of a respondent's preference.

While a ranking question can be highly beneficial, there are limitations. The most significant limitation is quantifying the ranking differences. The food manufacturing example might result in "Banana" and "Chocolate" as the top two flavors. However, the distance between the two flavors is an unknown variable. People might like "Banana" 100x more than "Chocolate." This data point is crucial because, if true, the food manufacturer should focus their efforts solely on "Banana" to maximize revenue.

Another downside to a ranking question is survey fatigue. Evaluating a long list of attributes requires a lot of effort and can be prone to errors. Even with click ranking, respondents would need to evaluate all attributes at once before selecting their top three (3).

A solution to both problems is MaxDiff. MaxDiff can help identify what is most and least important (or most/least desired) from a list of attributes. The basic concept is that respondents are shown a small subset of the total attributes (like a random set of five out of ten attributes) and pick what is most and least important. Respondents are shown multiple sets, meaning attributes are compared against one another. In the manufacturing example, "Banana" would have a much higher score than "Chocolate" since respondents compared both against each other.

The ranking score is a weighted calculation. Items ranked first are given a higher value or "weight." The score computed for each answer option/row header is the sum of all the weighted values. For example, if there are five options, the weighted sum for an option a respondent placed in the first position (1) would be worth 5. The points are summarized, and the item with the highest points is ranked first.

The results include how often an item was ranked first and display the ranking distribution with a small bar chart. The color-coding of the ranking distribution makes it easy to see net top/bottom rank attributes.

The Excel export will display each attribute as a column with the respondent's ranking. The column would be blank if a respondent did not rank an attribute.

The sample data below are the results of the click ranking survey used by the food manufacturer. Each attribute has a row with the ranked distribution, first-place counts, and total score.

A feature unique to SurveyKing, is the ability to create a segment report for a ranking question. This report type is helpful to drill down into the data and spot hidden relationships. For example, you might include a question in your survey that asks for the respondents' gender. You could then create a segment report (or a cross-tabulation report) by gender. The results would include the table shown above for both "Male" and "Female". You may notice "Males" prefer a particular attribute that females do not prefer or vice versa.