Welcome to week two of our six-part blog series entitled “Best of A/B Testing.” Every week we will be showcasing a customer A/B test, along with testing results and learning.
This Tuesday, we are showcasing an A/B testing trifecta (conducted by the ion interactive professional services team) for Paychex, a recognized leader in the payroll, human resource and benefits outsourcing industry.
Which landing page increased lead conversions 31%?
Test 1: Single landing page (Control) vs 2-Step progressive experience (Challenger)
Traffic source: Pay-per click advertising; non-brand “payroll” keywords on Google
Test Type(s): Format, Look & Feel, Innovation
Hypothesis: A 2-step progressive landing experience with warm, small business imagery (target audience) will increase conversion rate over the Control (single landing page).
Results: Version B, the 2-step form experience increased lead conversions 31%.
Learning: Behavioral tracking on the Challenger showed that a ~20% conversion rate was associated to visitors who clicked the “show details” link within the offer content. Based on this learning, test two was proposed to determine the impact of visual emphasis of the “free month” offer on conversion.
Text or graphic — which offer presentation increased conversions 28%?
Test 2: Visual presentation of “free month” offer
Traffic source: Pay-per click advertising; non-brand “payroll” keywords on Google
Test Type: Offer Presentation
Hypothesis: Increasing the visibility of the “free month” (addition of offer graphic) will increase lead conversions.
Results: Addition of the “free month” offer (version B) graphic resulted in a 28% increase in lead conversions.
Learning: A more prominent visual emphasis of the offer resulted in a significant increase in conversion rate. Since the free offer was determined to be a strong driver of conversion, test three was proposed to capitalize on this learning.
What offer placement won at 95% confidence in 24 hours?
Test 3: Offer location/placement
Traffic source: Pay-per click advertising; non-brand “payroll” keywords on Google
Test Type: Offer Placement (location of “free month” offer graphic)
Hypothesis: Moving the “free month” offer closer to the call to action will elicit more clicks and thus drive more conversions.
Results: Surprisingly, moving the offer closer to the call to action did not result in an improvement in lead conversions. The version with the offer graphic was auto-optimized out of testing rotation within 24 hours!
Learning: You need to test everything, all the time! Because, as the third test illustrates, you never know when you’ll be surprised by the results. For us, moving the offer closer to the call to action was a “sure thing” — we were quickly told that indeed, it was not! The visitor has spoken and we move on with our testing. Onward ho!
Paychex offers an ever-growing variety of payroll and human resource products and services that are customized for each of their clients’ business needs. Paychex helps over half a million small to medium-sized businesses with their HR needs.
Using LiveBall and Mongoose Metrics, the Paychex marketing team is able to test and optimize over 250 paid search landing pages based on accurate and complete analytics. Read more.