Day 184: Split Test Results and What They Mean

Today’s update focuses on reviewing the split tests that were run during February.

Even though a large part of the month was spent travelling and not actively working on the funnel, a few useful experiments were still set up and allowed to run.

Looking back at these tests is important because they reveal what is actually moving the numbers. Understanding what works and what doesn’t will help shape the strategy for the next phase of the funnel.

Key Takeaways

Adding a second bonus increased conversions from 2% to 3%.
Reinforcing bonuses at the checkout unexpectedly performed worse.

An in-course upsell generated strong conversions and additional revenue.
Small improvements in conversion and lifetime value continue to be key drivers of profit.

Daily Performance Snapshot

Yesterday was not a particularly strong day in terms of revenue. Sales came in at $778.

Ad spend was roughly £408, which converted to around $454 in advertising costs.

That left a profit of approximately $233 for the day. While still profitable, it sits below the target range of $400 to $500 per day.

Today’s performance so far is also relatively modest. At around 3pm sales were sitting at $372, suggesting another tighter day overall.

Why Split Testing Matters

Despite the quieter sales days, February still provided valuable data because several split tests were running in the background.

These experiments help identify which changes improve conversion rates, reduce acquisition costs, or increase revenue per customer.

Even small improvements can compound significantly over time when they affect every visitor entering the funnel.

Test 1: Adding a Second Bonus

The first test involved adding a second bonus to the main sales page.

The goal was straightforward. Increasing the perceived value of the offer should encourage more visitors to convert, which lowers the cost of customer acquisition.

This test was run properly with roughly 4,000 visitors sent to each version of the page.

The results showed a clear improvement. The control page converted at around 2%, while the page with the second bonus converted at roughly 3%.

Although that might appear small at first glance, it actually represents about a 50% improvement in conversion rate.

Across nearly 100 sales on each version, the data appears strong enough to confirm that the second bonus is now the better-performing version.

Test 2: Reinforcing Bonuses in the Checkout

The second split test was far more subtle.

The sales page already uses a two-step checkout process. Step one collects contact details, and step two collects payment information.

Like most funnels, there is always a noticeable drop-off between people starting the checkout and people completing the purchase.

The idea behind the test was simple. Reinforcing the bonuses again at the payment stage might remind customers of the value they’re receiving and reduce cart abandonment.

However, the early data produced a surprising result.

Unexpected Results from the Checkout Test

Approximately 4,100 visitors were sent to each variation of the checkout page.

The strange part is that the difference appeared earlier than expected in the checkout process.

Both versions of the page should have produced similar percentages for the first step of the checkout because the pages are identical at that stage.

Yet the numbers showed around a 20% difference between the variations.

Because this doesn’t logically align with the page structure, it suggests something unusual might be happening within the data.

Rather than drawing conclusions too early, the next step will be reviewing the individual transactions to understand whether the numbers are accurate.

For now the test will continue running until there is clearer information.

Test 3: Adding an In-Course Upsell

The third experiment involved adding an upsell inside the course itself.

Traditionally, upsells appear immediately after the initial purchase or within follow-up email sequences.

This test added a new opportunity inside the product experience. Once customers logged into the course they purchased, they were presented with an upgrade offer.

The offer price was set at $47 as an introductory test price. The full product normally sells for around $97.

Results from the In-Course Upsell

Although the in-course upsell was only active for a short period in February, the early results were extremely encouraging.

From the group of customers who reached that page, around 30% chose to upgrade.

That conversion rate is significantly higher than typical upsell conversions within most funnels.

In total the test generated approximately $470 in additional revenue during February.

Because that revenue came from customers who had already purchased, it represents a valuable increase in lifetime value.

What Happens Next

The next logical step is to test a higher price point for the in-course upsell.

The plan is to increase the offer from $47 to around $67 during March to see whether conversions remain strong.

If the conversion rate holds at a similar level, the increase in price could significantly improve the total revenue generated from this feature.

Even a small adjustment in price could produce a meaningful increase in monthly profit.

Looking Ahead to March

Despite not spending much time actively managing the funnel in February, these tests still produced useful insights.

The second bonus clearly improved conversions, the checkout test requires further investigation, and the in-course upsell has opened up a new revenue opportunity.

The next phase will involve building on these findings and designing new experiments to close the profit gap that currently exists in the overall yearly target.

Closing Reflection

The key lesson from February is that optimisation rarely comes from a single dramatic change.

Instead, progress comes from a series of small tests, each contributing incremental improvements to conversion rates, revenue per customer, or lifetime value.

By continuing to test and refine these areas throughout March, there is still plenty of opportunity to push the funnel closer to the $240K target.