MaxDiff (Best–Worst Scaling)

MaxDiff (Maximum Difference Scaling, also called Best–Worst Scaling) asks respondents to pick the best and the worst item from a small set, repeated across several screens. Because every choice is a forced trade-off, MaxDiff produces a far more discriminating preference ranking than a battery of rating scales — there are no "everything is a 5" responses. Use it to prioritize features, messages, claims, package designs, or any list where you need a clean "what matters most" ordering.

This guide covers every MaxDiff setting in the survey builder, what each one does to the respondent experience, and where it takes effect.

---

Adding a MaxDiff question#

MaxDiff is a custom-interface question type, shown in the question palette as ⚖ MaxDiff — "Best/worst trade-off across items." Click or drag it into the section you want, then fill in the item list and open the Settings tab in the Properties panel to configure the design.

A MaxDiff question has two authoring areas:

• The question card (on the canvas) — the prompt, the MaxDiff Items list, and the Columns (Best/Worst labels and colors).
• The Settings tab (Properties panel) — Design of Experiment, Screens & Items, the Top-N Bandit parameters, and Sequential Response.

---

Items and columns#

The item list is the pool of options respondents trade off against each other. Add items one at a time, paste many at once with Bulk Edit, or pull them from an earlier question with Carry Forward Choices. Each item can carry an image via Insert Graphic for visual studies.

The Columns block controls the two selection columns shown on every screen:

• Best label / Worst label — the column headers respondents see (default "Best" and "Worst").
• Best color / Worst color — the highlight colors for each selection (default green #27ae60 for Best, red #c0392b for Worst).

---

Design of Experiment#

The Design of Experiment (DOE) picks the strategy that decides which items appear together on each screen. The three options are mutually exclusive — selecting one disables the others.

Where it takes effect: Standard and Tournament both render in the in-builder Preview. Top-N Bandit falls back to the Standard balanced design in preview, because there is no model result to steer it until responses have been collected — that difference shows up in live fielding and analysis only.

How randomization actually works#

This is the most common question about MaxDiff, so it's worth being precise.

Within a respondent, exposure is balanced first. The design fills each slot with the item that has been shown least often so far. That means every item appears once before any item appears twice — with 12 items at 4 per screen, screens 1–3 cover the entire list before anything repeats. This is intentional and it is what makes the resulting utilities unbiased. It is not a missing shuffle. Which items land together on a screen, and in what order, is random within that constraint.

Across respondents, every respondent gets a different rotation. The design is seeded from the question and the respondent's session, so no two people see the same screens, the same order, or the same item positions. Each individual still gets perfectly even exposure. This is what keeps position and order effects from being confounded across the sample.

Within a session it is stable. A respondent who navigates back, or resumes a partly finished survey, sees the same screens they saw before. Preview uses a fixed session identifier, so the design does not churn while you are authoring the question.

Note for previewing: because preview is deliberately stable, previewing the same question twice shows you the same screens. That is expected and does not reflect what respondents get in the field.

Top-N Bandit parameters#

Choosing Top-N Bandit reveals three extra controls:

• Focus on Top X% (0–1) — the share of top items the bandit should concentrate on (default 0.3, i.e. the top 30%).
• Min respondents to start — the bandit serves a regular balanced design until this many responses are collected, then switches to model-driven sampling (default 50).
• Update every N respondents — how often the model is re-run and the sampling refreshed during fielding (default 50).

These three values are stored on the survey and read by the server-side model and fielding scheduler — they are intentionally managed outside the front-end design code, so the in-builder preview is unaffected by them.

---

Screens & Items#

These two numbers control the size and length of the exercise:

• Number of screens — how many best/worst tasks each respondent completes. More screens means more comparisons per item and a more stable ranking, at the cost of respondent effort.
• Items per screen — how many items appear on each screen for the respondent to choose a Best and a Worst from (typically 4–6).

In the respondent view below, Number of screens = 4 is reflected by the 1 / 4 counter, and Items per screen = 5 is reflected by the five rows shown.

Fit to Screen#

When Fit to Screen is on (default), the widget sizes the question and all of its choices to fit the viewport without scrolling — useful on mobile and for keeping the whole task visible at once. It is a purely visual setting and does not change the data collected.

Show Best column on the right#

By default the Best column is on the left and Worst on the right. Turn Show Best column on the right on to swap them — Best moves to the right, Worst stays on the left. Some study designs prefer this orientation to match a left-to-right "worst → best" reading order.

---

Constraints#

Some item pairs shouldn't be shown together — two variants of the same product, a claim and its own counter-claim, or options that make no sense side by side. Constraints let you declare those pairs, and the design will never seat them on the same screen.

Open the Constraints panel in the Settings tab and click Add constraint. Each constraint is two items from the question's item list. Add as many as you need; remove one with the trash icon.

Things to know:

• Constraints apply to Standard and Top-N Bandit. Tournament builds its screens from an elimination bracket rather than the balanced picker, so it cannot honor them. The panel is hidden when Tournament is selected, rather than accepting settings that would be quietly ignored.
• Balance is preserved. Excluding a partner only affects which items can share a screen — every item still gets even exposure overall.
• The builder warns you if you over-constrain. Constraints reduce the pool of items that can legally appear together. Add enough of them and there may be no way to fill a screen with the configured number of items. The panel runs the real design as you edit and tells you when that happens, so you can remove a constraint or lower Items per screen before fielding.
• Deleting an item cleans up its constraints. Any constraint naming that item stops being shown immediately and is dropped for good the next time the question is saved — deleting an item never blocks a save.

---

Sequential Response#

Sequential Response adds a follow-up question about the item the respondent picked as Best on each screen. It is used to estimate a "None" option in the model — i.e. whether the chosen Best item is genuinely wanted, or merely the least-bad of the set.

When you turn it on you configure:

• Follow-up question text — the prompt shown under the chosen Best item (for example, "Below is the offer you chose as most important. How important is this feature to you in choosing a product?").
• Answers — the response options for the follow-up (for example, Extremely Important / Somewhat Important / Not That Important).
• Label as Success — a checkbox on each answer. Answers marked as Success count as "Best beats None" in the model; unmarked answers count as None beating Best. This is what lets MaxDiff estimate a None threshold rather than just a relative ranking.

In the respondent experience, once a Best (and Worst) is chosen on a screen, the follow-up appears inline with the chosen item and the answer options:

With Sequential Response on, a screen is only considered fully answered once its follow-up has also been answered, and the question is mapped to the sequential analysis model rather than the classic one.

---

Settings reference#

Every MaxDiff setting, what it controls, and where it takes effect. "Respondent widget" means the setting changes what respondents see during fielding/preview; "Back-end model" means it is saved on the question and consumed by the server-side design/modeling layer.

---

Data and analysis#

• Each screen is stored as the items shown plus the chosen Best, chosen Worst, and (when Sequential Response is on) the follow-up rating. Because every response records the items that respondent actually saw, per-respondent design rotation needs no special handling in analysis — and it improves the model, since varied item combinations identify the utilities more sharply than one repeated design would.
• The per-survey analytics response view renders MaxDiff answers screen by screen, honoring the Best/Worst column placement and resolving the sequential answer labels.
• MaxDiff data exports in a standardized stacked format — one row per choice set per respondent, with the item columns plus the selected item (and a selected-rating column when Sequential Response is on) — which is the standard feed into the preference model.

---

Related guides#

• Question Types — overview of all question types and when to use each
• Conjoint — choice-based trade-off between multi-feature product concepts
• Display Blocks & Plugins — other custom-interface question types
• Carry Forward & Piping — reuse MaxDiff items and top picks in later questions
• Analytics & Responses — how responses are displayed and exported