Conjoint analysis is a survey-based statistical technique that helps determine how people value the individual features of a product or service. A conjoint question shows respondents a set of concepts, asking them to choose or rank the most appealing ones. When the results are displayed, each feature is scored, giving you actionable data. This data can help determine optimal product features, price sensitivity, and even market share.
Basic Concept: Conjoint analysis goes beyond a standard rating question. It forces respondents to pick what product concepts they like best, helping identify what your audience truly values.
If we offered a new menu item for ice cream, which of the following options would be most appealing to you? Please make one choice per set. If no options look appealing, choose "None".
This example is a choice based conjoint model
Conjoint analysis is used when you want identify what people value the most. Conjoint analysis is commonly used to test new product concepts. The above sample question could be used by a restaurant chain to determine additions to a dessert menu. The chain should focus on concepts that provide the most value for their customers. When respondents evaluate this question, concept features are compared against one another, and a preference for each feature can be identified.
With standard rating questions, it would impossible to ask about prices; a separate rating question for each flavor and size combination is needed. This would lead to fatigue and poor-quality data. Conjoint analysis solves these problems with one simple survey question.
Conjoint analysis is an advanced research technique. It involves unique terminology in both the survey setup and in the reporting metrics. To help you get a full understanding, here is a list of commonly used conjoint terms:
These are the high-level product features that respondents will evaluate. Attributes are the very first column in the above sample question. Our example has the following features: flavor, size, and price. If you did a study about a new car offering you might have features such as color, make, model, MPG, and tire type. There is a limit of 20 attributes on the SurveyKing platform.
These are the items that make up a single attribute. In our example the "Flavor" attribute has levels of "Chocolate", "Vanilla", "Cookie Dough", and "Strawberry". When you create the conjoint survey, you define an attribute and levels that go with each attribute. There is a limit of 15 levels on the SurveyKing platform.
This a combination of all your attributes and levels, creating a hypothetical product. In the above example, concepts are the columns that respondents choose. Concepts are sometimes referred to as "cards" in statistical software. There is a limit of 7 concepts on the SurveyKing platform.
Also referred to as a task, a set contains multiple concepts or product offerings. Respondents will choose one concept per set, and then be shown a new set of concepts. There is a limit of 20 sets on the SurveyKing platform.
This is the most important term in conjoint analysis. It defines how a respondent values each attribute level. When all the utilities for all respondents are analyzed, an overall product value can be determined. Utilities are the output of a regression equation.
Utilities have no scale compared to other conjoint projects you run. They only matter in the context of the current question you are looking at.
Sometimes utilities are called "part-worths", or "part-woth utilities". We simply use the term "utility".
This is the most common form of conjoint. The example question above is a choice-based conjoint question. Respondents pick the most appealing concept for each set. Each set contains a random set of concepts, that are evenly distributed. This type of conjoint best simulates buyer behavior, since each set contains hypothetical products (concepts). When respondents choose a full profile, preferences can be calculated from the tradeoffs made. (e.g even though "Strawberry" isn't a preferred flavor, if the price was low enough, it would still provide consumer utility")
As with most conjoint studies, preliminary research is important to reduce the number of attributes and levels to choose from. The less attributes, and levels, means less possible full concepts, reduced survey fatigue, and better quality data. For example a MaxDiff or ranking survey could be completed to find the top four ice cream flavors.
Currently this is the only type of conjoint offered by SurveyKing.
Sometimes referred to as MaxDiff conjoint. Similar to choice-based conjoint, this method shows respondents a set of concepts. In each set, respondents are asked to pick the most/least (or best/worst) concepts. This approach is used when a product or service has features that cause both positive and negative reactions. An example could be studying how parents select daycare. The number of full-time faculty would draw a positive reaction. The percentage of fellow students that are economically disadvantaged could produce a negative reaction.
This is a future addition to the SurveyKing platform.
This method is also similar to choice-based conjoint. Respondents pick the most appealing concept for each set, except with this method, the next set of concepts are not random, but are tailored based on the previous answers. This method is more engaging to respondents, and can help fine-tune the data.
This is a future addition to the SurveyKing platform.
This method displays all possible concepts and asks respondents to rank or rate each one. This method is outdated and was primarily used prior the introduction of survey tools that offer choice-based conjoint. A set of physical cards were presented to respondents. Asking to rank all concepts is error prone, easily causes fatigue, and yields poor quality data.
Ranking and rating conjoint was the method used for full-profile conjoint analysis. As software has progressed, it is now possible to conduct rating or ranking conjoint similar to a choice-based conjoint. Respondents are shown a set of concepts and asked to rank or rate each concept. They could rank by entering a value for each concept, which sums to 100 for each set, or they could simply enter a number-based a on scale. This method is also sometimes referred to as "Continuous Sum Conjoint".
This is a future addition to the SurveyKing platform.
Menu-based conjoint is a new conjoint method. This method gives respondents the ability to pick multiple levels from a menu. For example, a car manufacturer could ask respondents to choose a base model and price, just like choice-based conjoint. But then they could also ask to check a box for each additional feature desired such as "Alloy Wheels for $1,500", "Sunroof for $1,000", or "Parking Assist for $1,500".
This method is much more advanced in terms of front-end programming and back end statistics than choice-based conjoint. Often custom solutions need to be built for a company wishing to create this type of project.
Any survey that contains a conjoint question can be referred to as a conjoint survey. SurveyKing currently only offers choice-based conjoint. Here are the steps needed to add a conjoint question:
An ideal conjoint question will have roughly 5 attributes (rows), 4 concepts per set (columns), and roughly 5 - 10 sets. This will help ensure respondents are not fatigued. A detailed breakdown is below:
We recommend collecting at least 100 responses for each segment being researched. For example, if you wanted to research both males and females, you would want to collect 100 responses for both.
Conjoint analysis uses regression to calculate how different attributes and levels are valued.
Because conjoint uses categorial data (a name like ice cream flavor) instead of continuous data (a number like temperature), a special type of regression is used, called logistic regression. Just like any regression equation, the result of this regression calculates coefficients. These coefficients are referred to as "utilities".
Utility is not a standard unit of measure. It can be thought of as "happiness". Let's say a bunch of respondents choose concepts that contain "Cookie Dough" and only a few choose concepts with "Vanilla". Even without doing the math, you can imagine that the coefficient for "Cookie Dough" would be higher than the coefficient for "Vanilla".
To illustrate this concept, we ran the above ice cream example with 20 respondents. Below is the analysis of those responses. This analysis includes the utilities for each level in addition to the relative importance of each attribute.
The utilities in the last column are the output of regression analysis. Next to each number is small bar chart for visual representation.
Remember, utilities are not an actual unit of measurement, and could be thought of happiness. If we look the above table, the "Cookie Dough" flavor has a utility of 14 and the "Vanilla" flavor has utility value of 7. We could interpret this as "Cookie Dough has double the happiness of Vanilla". In terms of product size, we could say "a medium size gives a little more than 2 units of happiness than a large".
The importance column, is the weighted difference in utilities ranges for the product levels. You can see that flavor has the level with the largest difference of roughly 7. The larger the utility differences for an attribute, the more important they are to consumers. To get a large difference like we see with cookie dough, a lot of respondents had to choose concepts with that flavor. We know the other levels are evenly distributed, meaning that regardless of size or price, cookie dough was a big driving factor in decision making. Here's how you would calculate the importance:
Take the largest number for each level, and sum: 14.11+4.03+5.06 = 23.02
Divide each of the highest levels by this number. The calculation for flavor importance is 14.11 / 23.02 = 61%
Overall, it looks like "Cookie Dough" is the preferred flavor, "Medium" serving size, with a price of "$5 USD". "Vanilla" would also be a good addition to the menu as it bring a high amount of utility. You'll notice that the higher price has a higher utility value than the lower price. Respondents might have a perception that a higher cost has a better taste or better quality. The preference for "Medium" might also be tied to consumers being health conscious. This summarizes why conjoint is so important. Not only did we find the two optimal flavors, we also found the right size, and correct price point. This data would be almost impossible to capture without conjoint analysis.
SurveyKing uses ChoiceModelR, a package in the R statistical program for conjoint analysis. ChoiceModelR calculates a coefficient using logistic regression with maximum likelihood for each attribute level, for each respondent. When the analysis is complete, utilities for each level are averaged. The output of our example can be found in this Excel file.
We color coded the Excel file for each attribute level. You can see row 22 has an average subtotal. This is the average utility for a specific level. The regression equations use effects coding, to ensure each attribute in total sums to 0. Because of this you will notice the excel file contains negative utilities. We shift each number by a constant, to eliminate negatives, and put the baseline to 0. The dark blue flavor columns were adjusted by 5.43 prior to the results being loaded into our dashboard. Having a 0 baseline makes the data easier to interpret.
Data used to populate ChoiceModelR:
Sometimes it's important to analyze different segments, such as gender. To create a conjoint segment, simply add a multiple-choice question to your survey for each segment you wish to study. In the reporting section, you can choose "Conjoint Segment Report". From here, select the appropriate question, and the report will output a data table for each answer. Using the ice cream example, you may notice "Males" prefer "Cookie Dough", while "Females" prefer "Vanilla". These are additional data points to fine tune your marketing efforts.
A powerful benefit of conjoint analysis is quantifying how each concept would fare in the market. We can easily see the product with most utility would be Cookie Dough, Medium, for $5 USD. But what about the top 3 products? Or the bottom 3 products? In the ice cream example, there were 24 hypothetical products. Unique to the SurveyKing platform is the ability to scroll through each concept in ranked order, to see what profiles faired the best or worst (or offer the most utility). The reporting section will automatically include the table shown below:
|1||Cookie Dough||Medium||$5 USD||488.02|
|2||Cookie Dough||Large||$5 USD||436.91|
|3||Cookie Dough||Small||$5 USD||403.10|
|4||Cookie Dough||Medium||$2 USD||381.51|
|6||Cookie Dough||Large||$2 USD||330.40|
|7||Cookie Dough||Small||$2 USD||296.58|
To get these figures from the Excel output file, you could create a table with all possible combinations, and use sum product to calculate to total utility. Here is an example.