B won the test. Not for everyone.
A hotel-booking site tested three weekend heroes, and the destination-led one won overall. Read inside each audience, the same test said something else: phone visitors booked more after the price-led version, and visitors deep in hotel pages after the free-cancellation one. Each audience now sees the version that won it, applied in a click. Switch the visitor below to see all three, and the report the call was made on.
Built with A/B Testing with Auto-pick — see the feature →
Lisbon, Porto or the Algarve — two nights away
Hand-picked hotels in the places people are booking for this weekend.
One A/B test, a winner for each audience.
Test the versions
Three heroes, split by visitor, judged on bookings. Auto-pick is on, so once the leader clears your confidence, session and time thresholds it is declared and deployed.
Read it per audience
The same test is scored inside every audience Audience Discovery found. A version can win the site and lose a group of its visitors — the report says which, and how sure.
Apply the winners
Tick the audiences with a clear winner and apply. The test ends, each of them sees its own winner, and everyone else gets the overall one.
How Personyze turns one A/B test into a winner per audience.
One test, three versions, one overall leader
Only the hero is under test; the search box and the hotels below it are the same in every version. Visitors are split by visitor, the goal is bookings, and Auto-pick is set to the Balanced thresholds — 95% confidence, 1,000 sessions, 14 days — so on its own it would deploy B to everyone.
The report reads the test inside each audience
The audiences are the ones Audience Discovery found in this site’s own data, such as “Device is phone” and “Page views in /hotels at least 3”. They overlap, so each row is the test seen through one audience, never a part of a total. Audience Discovery is an add-on.
“Clear winner” is a high bar
A row is offered for applying only when its leader is not the overall leader, is significant after correcting for how many audiences were asked, leads in both halves of the days, and has enough sessions and conversions. The direct-traffic row is not sure yet, so it is left alone and the US row simply agrees with B.
Applied in a click: each audience gets its winner
Tick the clear winners — up to ten — and apply. The test ends: phone visitors see A, hotel researchers see C, and the first-time desktop visitor sees B, because everyone outside the ticked audiences gets the overall winner.
A visitor in two audiences gets the stronger result
A phone visitor who has also viewed three hotel pages matches both rows. They see the winner whose result was stronger, so nobody is shown two heroes at once.
It can be undone
The apply is saved as a new version of the campaign, marked as made by the optimizer, and stays undoable until the campaign is edited again — the note under the button says so before you click. The numbers in this demo are illustrative.
See how it is built, and where else it fits
Questions about this example
How does Personyze show a winner for each audience?
The A/B tests by audience report scores the same test inside every audience Audience Discovery found in the account's own data: the rate per version, which version leads, and how sure that is. Audiences overlap, so each row is the test seen through one audience. Audience Discovery is an add-on.
When is an audience's winner offered to apply?
Only when it is a clear winner: it is not the overall leader, it is significant after correcting for how many audiences were asked, it leads in both halves of the days, and it has enough sessions and conversions. You tick up to ten such audiences and apply; the test ends, each sees its winner, and everyone else gets the overall winner. A visitor in two of them sees the winner of the stronger result.
What does Auto-pick do, and can an apply be undone?
Auto-pick declares and deploys a test's winner on its own once it clears your thresholds for confidence, sessions and minimum time, such as the Balanced preset of 95%, 1,000 sessions and 14 days. Applying winners per audience saves the campaign as a new version marked as made by the optimizer, and it can be undone for as long as the campaign has not been edited since.
Find out who your winner loses.
Run the test you were going to run anyway, read it audience by audience, and give each group the version that actually works for it — in one click, with Undo.