Real behavior is never caused by one variable at a time. Factorial design lets you test several factors at once, and see how they interact.
A factorial experiment is a study with two or more factors, each with discrete ‘levels’, where units are tested across all combinations of those levels. In UX research it matters because behavior is rarely driven by a single variable: factors interact. Instead of holding everything else constant (which biases results to one configuration), you manipulate several factors together as independent variables, so you can measure each factor’s effect, whether they interact, and how. It is the statistical backbone of multivariate testing, and results are typically analyzed with ANOVA.
Key takeaways
- 'Factorial design' and 'factorial experiments' are used interchangeably (not strictly identical, but close enough in practice).
- A factor is a manipulated variable with two or more levels (e.g. color: green/blue/red).
- An interaction means one factor's effect on behavior depends on another factor's level.
- Controlling all other variables biases results to that single configuration; factorial designs avoid this.
- Designs are named by their factors and levels (a 3 x 4 design) and analyzed with ANOVA.
What is factorial design?
Factorial design is a statistics methodology used heavily in cognitive and behavioral psychology, which underpin UX research. Formally, a factorial experiment has two or more factors, each with discrete levels, and its experimental units take on all possible combinations of those levels. The point is to study not just each factor alone but how factors combine.
Behavior is never affected by a single variable. Factors occur together and interact in complex ways.
Core premise of factorial design in UX
Factors, levels and interaction
A factor must be manipulable: the experimenter makes it take different values (presence/absence, much/little/none). Take a button’s color as the independent variable (IV) and its selection rate as the affected behavior (the dependent variable). If you also suspect the button’s position matters but hold it constant (say, top-right), your color results are biased to that one position. An interaction occurs when a factor’s effect depends on the level of another factor. This is why behavioral research uses multivariate (factorial) designs: alternative factors are folded in as manipulated IVs rather than held constant.
The color x position example (a 3 x 4 design)
Combine color (3 levels: green, blue, red) with position (4 levels: top-right, top-left, bottom-right, bottom-left) and you get a 3 x 4 factorial design, recording the button’s selection rate across all 12 combinations:
| Position Color | Green | Blue | Red |
|---|---|---|---|
| Top-right | rate | rate | rate |
| Top-left | rate | rate | rate |
| Bottom-right | rate | rate | rate |
| Bottom-left | rate | rate | rate |
Three ways to assign subjects
| Design type | How subjects are assigned | In the 3 x 4 example |
|---|---|---|
| Independent measures | A separate group of subjects for each combination of factor levels; each group does the task under one combination only. | 12 groups |
| Repeated measures | A single group of subjects performs the task under every combination of values. | 1 group, all 12 |
| Mixed (repeated in some factors) | Subjects experience all levels of one factor but are split into groups for the other factor. | 3 color groups x 4 positions |
Why it matters in UX (and how it's analyzed)
Factorial designs reveal each factor’s separate effect, whether factors interact, and how to describe that interaction. They can hold two, three or more factors, and grow more complex to interpret as factors are added. The data is usually analyzed with Analysis of Variance (ANOVA): two-way, three-way and so on, matching the number of manipulated factors.
An experiment with two or more factors, each with discrete levels, where units are tested across all combinations of those levels, so you can study factors alone and together.
A factor is a manipulated variable (like button color); its levels are the values it takes (green, blue, red). A 3 x 4 design has one 3-level and one 4-level factor.
When the effect of one factor on behavior depends on the level of another factor, for example color mattering differently depending on button position.
Multivariate testing (MVT) is essentially factorial design applied to UX: several variables varied at once to study their isolated and joint effects, unlike single-variable A/B tests.
Usually with ANOVA (Analysis of Variance), in two-way, three-way and higher forms according to the number of manipulated factors.
Need rigorous, multi-factor UX research? Talk to Dorve.
Start a project