A/B Test #1 Final Results for Kiierr
Overview
Business Question: What should we have on the banner?
Hypothesis: Simplifying the content and replacing the headline with a more engaging headline will lower users’ potential stress and build more trust.
Variant 1
Changed headline to ‘Guaranteed Hair Growth Improvement within 7 months’ on desktop and ‘Guaranteed Hair Growth within 7 months or your money back!’ on mobile.
Changed ‘FDA…’ statement text and button color to orange. Changed green banner text to lowercase and omitted some clickable text from it.
This variant is just like V1 except for 2 other minor changes. Replaced main CTA wording with ‘See Spring Sale Pricing!’ Changed button to orange. The mobile version doesn’t have 2 boxes (I didn’t remove one). Changed green banner text to lowercase and omitted some clickable text from it.
TEST RESULTS
+0.92% Conversion Rate increase on Users Placed Order metric – Desktop: 4.78% / Mobile: 2.08%
+0.48% Conversion Rate increase on Visits to Checkout Page metric.
Variant 2
This variant is just like V1 except for 2 other minor changes. Replaced main CTA wording with ‘See Spring Sale Pricing!’ Changed button to orange. The mobile version doesn’t have 2 boxes (I didn’t remove one). Changed green banner text to lowercase and omitted some clickable text from it.
TEST RESULTS
+0.30% Conversion Rate increase on Users Placed Order metric – Desktop: 4.13% / Mobile: 1,22%
-0.67% Conversion Rate increase on Visits to Checkout Page metric.
Variant 3
This variant is just like V1 except for 2 other minor changes. Replaced main CTA wording with ‘See Spring Sale Pricing!’ Changed button to orange. The mobile version doesn’t have 2 boxes (I didn’t remove one). Changed green banner text to lowercase and omitted some clickable text from it.
TEST RESULTS
+1.02% Conversion Rate increase on Users Placed Order metric – Desktop: 4.14% / Mobile: 2,3%
-0,58% Conversion Rate increase on Visits to Checkout Page metric.
+1,60% Total CVR increase (based on 2 metrics)
Variant 4
This variant is just like V3 except for 1 other minor change. Changed wording on CTA to ‘SEND ME A CAP!’.
TEST RESULTS
+2,84% Conversion Rate increase on Users Placed Order metric – Desktop: 10.37% / Mobile: 1.25%
-0,57% Conversion Rate increase on Visits to Checkout Page metric.
+3,41% Total CVR increase (based on 2 metrics)
Variant 5
This variant is just like V3 except for 1 other minor change. Changed wording on CTA to ‘See Offer Options’.
TEST RESULTS
+0,57% Conversion Rate increase on Users Placed Order metric – Desktop: 4.89% Mobile: 1.46%
-0,32% Conversion Rate increase on Visits to Checkout Page metric.
The results
There is a clear winner for test 1!
- Variant 4 has the highest conversion rates for Placed Order metric and the Visits to Checkout page metric. Variant 4 also has the highest revenue on mobile.
- Although Variant 1 has higher revenue on desktop, more users are purchasing on mobile. There are also consistently more purchases on mobile. It’s wiser to make the implementation decision from off the mobile data.
- There is differing data (can seem a little unclear), but because of the higher revenue increase on V4 mobile and the high Ecommerce CVR on mobile, and how it users responded to it on desktop (+10.37% CVR on Place an Order metric, see Slide 10), we can confidently recommend to implement Variant 4.
PLACED ORDER METRICS CHARTS
ECOMMERCE METRICS CHARTS
COMBINATION OF DESKTOP AND MOBILE
Users Placed Order:
- Original: 2.12% CVR
- Variant 1: 3.05% CVR
- Variant 2: (BEST): 2.42% CVR
- Variant 3: (BEST): 2.88% CVR
- Variant 4: (BEST): 4.96% CVR
- Variant 5: (BEST): 2.69% CVR
Visits to Checkout Page:
- Original: 4.46% CVR
- Variant 1: 4.94% CVR
- Variant 2: (BEST): 3.79% CVR
- Variant 3: (BEST): 5.04% CVR
- Variant 4: (BEST): 5.03% CVR
- Variant 5: (BEST): 4.78% CVR