Personyze — A/B test results

Recommendation

Promote Variation B — Social proof as the winner

B leads on every conclusive metric with no negative side-effects. Both the primary and secondary goals have crossed 95% confidence, supporting KPIs are trending in B's favor, and engagement (page views, time on site) hasn't dropped relative to the baseline.

Trial signups: +26.6% (96%) Demo requests: +37.5% (95%) Bounce rate: −2.14% (97%) No engagement regression

Ask the assistant

Delivery issue detected on Variation B

94.3% view-to-delivered rate · placeholder selector .pricing-cta not found on 552 sessions (5.7%) · likely missing on the redesigned mobile pricing page

Delivery health · per variation

Delivered = system attempted to inject the variation. Viewed = element actually rendered and was visible to the user. A gap usually means a missing placeholder selector or a trigger that didn't fire (e.g. exit-intent on touch devices).

Variation A — Original

Delivered9,660

Viewed9,612

99.5%

Healthy

Variation B — Social proof

Delivered9,660

Viewed9,108

94.3%

Debug placeholder

Variation C — No credit card

Delivered9,662

Viewed9,648

99.9%

Healthy

Control group (holdout)

Eligible3,220

N/A—

No injection

Holdout — sees live page

Data readiness. The primary goal (Trial signups) is conclusive at 96%. Three supporting KPIs are not there yet — at the current ~1,073 visitors/day, keep the test running about 9–13 more days to reach 95% on click-through rate, page views, and time on site.

Test overview3 variations

Total visitors

32,202

over 30 days · ~1,073 / day

Traffic split

Control group (holdout)3,220 · 10.0%

A · Original9,660 · 30.0%

B · Social proof9,660 · 30.0%

C · No credit card9,662 · 30.0%

Daily performance

Control group Holdout

No test shown Visitors see the live page

Control group

3,220 visits · 10.0%

Baseline performance · no changes

Live page (holdout)

3,220 visitors · holdout

Live page 3,220Clicked 248 · 7.69%Closed —

Trial signups 127 · 3.94%

Variation A

Original

9,660 visits · 30.0%

1 action

24,030 executed

Delivered 9,612Clicked 767 · 7.98%Closed 0

Trial signups 407 · 4.21%

Variation B Leader

Social proof

9,660 visits · 30.0%

2 actions · combined

60,720 executed

Delivered 18,216Clicked 1,099 · 6.03%Closed 0

Trial signups 482 · 4.99%

Variation C

No credit card

9,662 visits · 30.0%

2 actions · combined

55,130 executed

Delivered 19,296Clicked 990 · 5.13%Closed 0

Trial signups 450 · 4.66%

Conversion goals3 goals

Goals

Primary goal

Trial signups

30 days · 32,202 visitors

Audience conversioni 4.55% 1,466 signups +26.6% B vs baseline

Control group Holdout

3.94% 127 signups

true baseline

Variation A — Original

4.21% 407 signups

+6.9% 71%95% CI −12% to +28%

Variation B — Social proof Winner

4.99% 482 signups

+26.6% 96% sure95% CI +2% to +53%

Variation C — No credit card

4.66% 450 signups

+18.3% 90% sure95% CI −3% to +41%

Daily conversion-rate trend

A — Original 4.21% B — Social proof 4.99% C — No credit card 4.66%

Secondary goal

Clicks on "Start free trial"

8.42% audience CTR · 2,708 clicks · +9.5% top vs baseline 91% sure

Control group Holdout

7.69% 248 clicks

true baseline

A — Original

7.94% 767 clicks

+3.3% 62%95% CI −9% to +16%

B — Social proof

8.60% 831 clicks

+11.8% 89%95% CI −2% to +27%

C — No credit card Leading

8.69% 839 clicks

+13.0% 91%95% CI −1% to +28%

Daily click-through-rate trend

A — Original 7.94% B — Social proof 8.60% C — No credit card 8.69%

Secondary goal

Demo requests

1.21% audience rate · 388 requests · +24.7% top vs baseline 95% sure

Control group Holdout

0.96% 31 requests

true baseline

A — Original

1.06% 102 requests

+10.4% 68%95% CI −13% to +37%

B — Social proof Winner

1.32% 128 requests

+37.5% 95%95% CI +1% to +78%

C — No credit card

1.21% 117 requests

+26.0% 88%95% CI −4% to +59%

Daily request-rate trend

A — Original 1.06% B — Social proof 1.32% C — No credit card 1.21%

Engagement

Click-through rate i

Audience avg.8.42%

89% sure B leads A (Control)

95% CI −1.5% to +25% vs baseline

~11 days to 95% significance

Control groupHoldout

248 clicks

7.69%

true baseline

A — Original

767 clicks

7.94%

+0.25%62%

B — Social proof

831 clicks

8.60%

+0.91%89%

C — No credit cardLeading

839 clicks

8.69%

+1.00%91%

Daily trend

A — Original 7.94% B — Social proof 8.60% C — No credit card 8.69%

Events 1Apr 7 — Variation B copy refined 2Apr 14 — Google Ads campaign launched 3Apr 21 — Holiday weekend 4Apr 25 — Mobile push test ended

Trending — 95% threshold not yet reached

91% · ~11 days more data needed at current pace

Page views in visit i

Audience avg.5.78 pages

B leads slightly — too close to call

95% CI −5% to +18% vs baseline

~9 days to 95% significance

Control groupHoldout

avg/visit

5.55

true baseline

A — Original

avg/visit

5.62

+0.0752%

B — Social proofLeading

avg/visit

5.92

+0.3771%

C — No credit card

avg/visit

5.79

+0.2462%

Daily trend

A — Original 5.62 B — Social proof 5.92 C — No credit card 5.79

Events 1Apr 7 — Variation B copy refined 2Apr 14 — Google Ads campaign launched 3Apr 21 — Holiday weekend 4Apr 25 — Mobile push test ended

Inconclusive — variations are too close

71% · ≈ 9 days more data needed at current pace

Bounce rate i

Audience avg.35.7%

97% sure B beats A (Control)

95% CI −3.7% to −0.6% vs baseline

Control groupHoldout

1,158 bounces

35.96%

true baseline

A — Original

3,413 bounces

35.32%

−0.64%63%

B — Social proofLeader

3,267 bounces

33.82%

−2.14%97%

C — No credit card

3,362 bounces

34.80%

−1.16%82%

Daily trend

A — Original 35.32% B — Social proof 33.82% C — No credit card 34.80%

Events 1Apr 7 — Variation B copy refined 2Apr 14 — Google Ads campaign launched 3Apr 21 — Holiday weekend 4Apr 25 — Mobile push test ended

Statistical significance reached

97% · 95% threshold met

Time on site i

Audience avg.3m 47s

B leads A (Control) — not significant

95% CI −7% to +27% vs baseline

~13 days to 95% significance

Control groupHoldout

avg dwell

3m 30s

true baseline

A — Original

avg dwell

3m 36s

+6s48%

B — Social proofLeading

avg dwell

3m 51s

+21s68%

C — No credit card

avg dwell

3m 48s

+18s63%

Daily trend

A — Original 3m 36s B — Social proof 3m 51s C — No credit card 3m 48s

Events 1Apr 7 — Variation B copy refined 2Apr 14 — Google Ads campaign launched 3Apr 21 — Holiday weekend 4Apr 25 — Mobile push test ended

Inconclusive — confidence low

68% · ≈ 13 days more data needed at current pace

Decision criteriai

How much do you weight each goal and supporting KPI? The composite ranking updates live — useful when signals conflict and you need to make the trade-off explicit.

Goal weightsi

Trial signupsPrimary 100/100

Demo requests 50/100

CTA clicks 20/100

Bounce ratei 15/100

Page views 10/100

Time on site 10/100

Composite ranking · weighted lift vs. baselinei

Control group

0.0%true baseline

Variation A — Original

+7.6%composite lift

Variation B — Social proof Winner

+27.9%composite lift

Variation C — No credit card

+19.8%composite lift

At these weights, Variation B wins with a composite lift of +27.9% over the baseline. The ranking is the same as the primary goal alone — the secondaries reinforce, not contradict, the primary winner.

Breakdown

Performance by segment

See where each variation wins. Cells show each variation's Trial signups by segment — winning cell per row is highlighted.

Segment	Visits / share	A — Original	B — Social proof	C — No credit card	Leader
Desktop	14,884 visits 46.2% of audience	5.12%baseline	6.21%+21.3%	5.58%+9.0%	B · 96%
Mobile	15,148 visits 47.0% of audience	3.42%baseline	3.89%+13.7%	4.04%+18.1%	C · 91%
Tablet	2,170 visits 6.7% of audience	4.10%baseline	4.78%+16.6%	4.51%+10.0%	B · 64%

Segment	Visits / share	A — Original	B — Social proof	C — No credit card	Leader
United States	18,224 visits 56.6%	4.38%baseline	5.42%+23.7%	4.81%+9.8%	B · 97%
United Kingdom	4,512 visits 14.0%	4.12%baseline	4.68%+13.6%	4.55%+10.4%	B · 88%
Germany	2,884 visits 9.0%	3.92%baseline	4.10%+4.6%	4.45%+13.5%	C · 72%
Canada	2,108 visits 6.5%	4.45%baseline	5.24%+17.8%	4.92%+10.6%	B · 90%
Australia	1,640 visits 5.1%	4.21%baseline	4.86%+15.4%	4.62%+9.7%	B · 78%
Other (24)	2,834 visits 8.8%	3.88%baseline	4.42%+13.9%	4.18%+7.7%	B · 71%

Segment	Visits / share	A — Original	B — Social proof	C — No credit card	Leader
New York, US	3,124 visits 9.7%	4.81%baseline	6.04%+25.6%	5.18%+7.7%	B · 96%
San Francisco, US	2,486 visits 7.7%	5.06%baseline	5.94%+17.4%	5.42%+7.1%	B · 92%
London, UK	2,118 visits 6.6%	4.32%baseline	5.01%+16.0%	4.74%+9.7%	B · 89%
Austin, US	1,724 visits 5.4%	4.16%baseline	4.42%+6.3%	5.04%+21.2%	C · 90%
Toronto, CA	1,456 visits 4.5%	4.52%baseline	5.36%+18.6%	4.96%+9.7%	B · 82%
Berlin, DE	1,288 visits 4.0%	3.84%baseline	4.02%+4.7%	4.46%+16.1%	C · 68%

Segment	Visits / share	A — Original	B — Social proof	C — No credit card	Leader
/pricing	13,608 visits 42.3%	5.34%baseline	6.41%+20.0%	5.78%+8.2%	B · 97%
/pricing/teams	6,422 visits 19.9%	4.18%baseline	5.02%+20.1%	4.62%+10.5%	B · 95%
/landing/saas-trial	5,118 visits 15.9%	3.62%baseline	4.18%+15.5%	4.46%+23.2%	C · 92%
/features/automation	3,884 visits 12.1%	3.91%baseline	4.55%+16.4%	4.32%+10.5%	B · 88%
/blog/* (12)	3,170 visits 9.8%	2.84%baseline	3.42%+20.4%	3.04%+7.0%	B · 76%

Segment	Visits / share	A — Original	B — Social proof	C — No credit card	Leader
Chrome	20,602 visits 64.0%	4.32%baseline	5.18%+19.9%	4.78%+10.6%	B · 96%
Safari	6,762 visits 21.0%	3.88%baseline	4.62%+19.1%	4.20%+8.2%	B · 91%
Firefox	2,576 visits 8.0%	4.46%baseline	4.92%+10.3%	5.18%+16.1%	C · 78%
Edge	1,608 visits 5.0%	4.18%baseline	4.86%+16.3%	4.52%+8.1%	B · 74%
Other	654 visits 2.0%	3.65%baseline	4.12%+12.9%	3.94%+7.9%	B · 58%

Each segment is tested independently — a leader needs 95% confidence to be conclusive within that slice. Learn more Export this view

Variation summary

All variations · all KPIs

Variation	Visits		Click rate	Bounce rate	Page views	Time on site
Control group (holdout)	3,220	3.94% true baseline	7.69% true baseline	35.96% true baseline	5.55 true baseline	3m 30s true baseline
A — Original	9,660	4.21% +6.9%	7.94% +3.3%	35.32% −0.64%	5.62 +0.07	3m 36s +6s
B — Social proofGoal winner	9,660	4.99% +26.6% 96%	8.60% +11.8% 89%	33.82% −2.14% 97%	5.92 +0.37	3m 51s +21s
C — No credit card	9,662	4.66% +18.3% 90%	8.69% +13.0% 91%	34.80% −1.16%	5.79 +0.24	3m 48s +18s

How to read this report. A KPI is conclusive when statistical confidence reaches 95%. Until then we show the leader and the estimated time to significance based on current traffic. When a control group is present it is the true baseline; otherwise the original variation is used as the baseline — every variation's improvement is measured against it. Click any variation to drill into session-level data.

A/B test results — docs

How to read this report, when a result is conclusive, and how each metric is calculated.

1What you're looking at

Every test compares one or more Variations against a Control group — a small holdout that does not see the test. The control gives you a true baseline, so the lift you measure on each variation is real lift versus the un-tested experience, not just A vs B noise.

Primary goal — the conversion you optimize for. Picking a winner is based on this metric.
Secondary goals — extra conversions tracked for context. They don't decide the winner but they help you spot side-effects (e.g., a variation that lifts signups but tanks demos).
Supporting KPIs — engagement signals (bounce rate, click-through, page views, time on site) that explain why a variation is winning or losing.

2When is a result conclusive?

A KPI becomes conclusive once statistical confidence reaches 95%. Below that, the leader is just leading — not winning.

Confidence	What it means
≥ 95%	Result is statistically significant. Safe to call a winner.
90 – 94%	Trending strongly. Wait for more data unless you have business pressure to ship.
< 90%	Inconclusive. Differences may be within natural variance — keep the test running.

Don't peek and call. Stopping a test the moment confidence first hits 95% inflates false positives. We recommend running until at least 14 days have passed AND significance has held for several days.

3How lift is calculated

Each variation's improvement is computed against the Control group (the holdout). For example, if Control converts at 3.94% and Variation B converts at 4.99%, the lift is:

(4.99 − 3.94) / 3.94 × 100 = +26.6%

Confidence comes from a two-sided z-test on conversion rates. The wider the gap and the larger the sample, the higher the confidence.

4Reading the breakdowns

The Breakdown panel splits the primary goal by device, country, city, URL, or browser. Each segment is a small mini-test — a leader needs 95% confidence within that slice to be conclusive.

Watch for segment-level surprises: if Variation B wins overall but Variation C wins on Mobile, you may want to personalize rather than ship a single winner.

5How long does the test need to run?

The "Estimated time to significance" pill on each KPI projects forward at the current traffic rate. It assumes the observed difference holds — if your conversion mix shifts, the estimate updates.

For tests under 1,000 daily visitors, plan on 3–4 weeks.
For tests over 5,000 daily visitors, you can usually call a winner inside 10 days.
Always include at least one full week to cover weekday/weekend cycles.

6Reaching out

Stuck on a result? Start a chat from the Help menu, or email support@personyze.com. Include the test ID and the date range you're looking at — you'll get a response within one business day.

Last updated · April 2026 Open full docs page →