Did the discount earn anything?
This store offers first-time visitors 15% off their first order — but one in five of them sees the same page without the code. That holdout turns “orders went up” into a number: the orders the code actually added. Customers are kept out of the test entirely. Switch the visitor below to see both groups, a customer, and the readout.
Built with A/B Testing — see the feature →
Tees, denim and sweatshirts you will wear every day
Heavyweight cotton, straight-leg selvedge and a crew sweatshirt in six colors. Free returns for 30 days.
The heavyweight tee is the only tee I have bought three times. It still looks new.Verified buyer — Portland, OR
A discount, measured against the people who never saw it.
Decide who is eligible
Visitors with no past order who are not on the customer list. Customers are excluded by rule, so they are in neither group.
Hold some of them back
The code goes to 80% of eligible visitors and the other 20% see nothing. The split is by visitor, so each person stays in their group on every visit.
Read the difference
The test is judged on purchases. The holdout’s rate is what would have happened anyway; the gap above it is what the code earned.
Free AI tool · Inspire me
See ideas like this for your own site.
How Personyze measures what a discount actually earns.
The code is shown, not generated
FIRST15 is one code set up in the store’s own checkout. Personyze decides who sees it and where: a banner across the top and a slide-in, capped at once per session.
The holdout sees the same page without it
Switch between the two first-time visitors: hero, bestsellers and review are identical. The only difference is the banner and the slide-in, so the only thing the comparison can measure is the code.
Split by visitor, not by visit
A first-time visitor is assigned once and stays in that group when they come back, so nobody sees the code on Monday and loses it on Tuesday, and every order counts toward one group.
Customers are excluded, not discounted
Anyone with a past order, or on the customer list synced from the store, is kept out of both groups by an Exclude rule. They get a new-since-your-last-order page with bought-together recommendations, and no code.
The readout answers one question
Conversion in each group, the orders the test group would have placed at the holdout’s rate, and the gap between them, with how sure the difference is. If the gap is small, the code is mostly paying people who would have bought anyway. The numbers in this demo are illustrative.
See how it is built, and where else it fits
Questions about this example
How does the holdout group work?
The campaign's traffic split gives the discount to a share of eligible visitors, 80% here, and the remainder is a holdout that sees nothing from the campaign. The split is by visitor, so each person stays in the same group on every visit, and both groups' sessions and purchases are counted.
Does Personyze create the discount code?
No. The code is set up in the store's own checkout. Personyze decides who sees it and where, here a banner and a slide-in shown only to the test group.
Why are returning customers excluded from the test?
They are likely to buy with or without a code, so discounting them costs margin, and including them would blur the comparison. An Exclude rule on purchase history or on a customer list keeps them out of both groups.
Measure your next discount.
Show the code to most first-time visitors, hold a slice back, keep customers out, and read the difference in purchases — before the discount becomes a habit.