Personyze — URCHIN · A/B Testing

Back

URCHIN

Target
Content
QA
Performance

In last 90 min:

12 visitors

9 impressions

0 clicks

Activate testing On

A/B testing

When on, traffic splits across test groups so you can compare variants. When off, Campaign content is shown to every visitor.

Traffic split

Adjust allocation

Drag a handle on the bar, or click any group's % to edit it directly.

Baseline / control group: give Site original some % — visitors in that slice see your unchanged site and serve as the control to compare A/B/C groups against. Set it to 0% to run without a control.

A · 85%

OG · 15%

Add group

Create a new variant group and split traffic with it.

Assign by

Auto-pick winner Off

Automated deployment

When the test reaches statistical significance, Personyze can automatically promote the winning group to 100% of traffic.

Set test goal — the KPI used to pick the winner.

Calling the winner — conditions that must be met before a winner is declared.

Level of assurance

Statistical significance

How confident we need to be before declaring a winner. Industry standard is 95%. Lower values (90%) reach a winner faster but with more risk of false positives.

Min. sessions

Minimum sample size

Don't call a winner until at least this many sessions have entered the test. Low traffic sites should raise this to avoid premature decisions.

Min. time

Minimum test duration

Don't declare a winner for at least this many days, even if significance is reached earlier. Captures weekday/weekend behavior differences.

days

Auto-adjust rotation

Adaptive traffic allocation

While the test runs, gradually shift more traffic toward the group that's currently performing best. Reaches a confident winner faster but trades off some statistical rigor.

When the test ends — what happens once the conditions are met.

Promote the winner to 100% of traffic

Auto-deploy the winner

Automatically reallocate 100% of traffic to the winning group once significance is reached. Other groups are paused. You can still edit the allocation afterwards.

Email on decision

Separate multiple addresses with commas.

Campaign content

100% of traffic

Shown to every visitor that matches targeting

POLICY STARTS

WITH YOU

Policy Starts with You

Popup · by bmiller@nar.realtor

12 (5.04%) Clicks

238 Impr.

0 (0.00%) Goals

0.31% Visitors

2026

VISIT CAPITOL HILL

LEARN MORE

Visit Capitol Hill

Popup · by bmiller@nar.realtor

18 (6.45%) Clicks

279 Impr.

0 (0.00%) Goals

0.31% Visitors

FOR EVERY

REALTOR®

Every REALTOR®

Popup · by bmiller@nar.realtor

20 (6.68%) Clicks

299 Impr.

0 (0.00%) Goals

0.33% Visitors

Add action

or drop here

Group A

85% %

of traffic

FLASH SALE 50% OFF Ends Sunday

SHOP NOW ›

Flash Sale — 50% Off

Banner · by marketing@brand.com

428 (11.20%) Clicks

3,820 Impr.

94 (21.96%) Goals

5.84% Visitors

More statistics

Add action

or drop here

Site original

Baseline

15% %

of traffic

Your unchanged site. No A/B actions here.
Campaign content still applies.

A/B Test Results

Analyze the performance of your test variations for each of your site's KPIs.

When the statistical significance exceeds 95% for a given KPI, the test result is conclusive. If we have sufficient data to predict a trend, we will provide an estimated time to reach statistical significance for that KPI.

Performance data is displayed for the selected date range, using the date range selector or the 'Date of Last Modification' option.

Performance data updates approximately one hour after visitor sessions end.

Reporting time range:

Total variations

Showing data for

21 days

From

Mar 29, 26

Audience total visits

35,056

Visits with viewed content

18,174

Rotation group A

49.6%

Control group

50.4%

Group: Rotation group A:

Flash Sale — 50% Off

Goal Search Result Page Hits

18.887% audience avg., 84% sure that Control group is better than Rotation group A

Estimated time to reach statistical significance level of 95%: 24 days

Control group

1,700 conversions

18.573%

Rotation group A

1,623 conversions

17.991% 0.582% worst (84% sure)

Goal PDP Add to Cart Click

29.538% audience avg., 77% sure that Control group is better than Rotation group A

Control group

2,885 conversions ($4,132)

31.52%

Rotation group A

2,798 conversions ($4,002)

31.017% 0.503% worst (77% sure)

Products purchased

0.836% audience avg., 79% sure that Rotation group A is better than Control group Leading

Control group

78 conversions ($49,874)

0.852%

Rotation group A

87 conversions ($65,111)

0.964% 0.112% improvement (79% sure)