How to A/B Test Post-Purchase Pages/Upsells on Shopify

In eCommerce, the bigger the impact of your page, the higher the need for A/B testing.

A/B testing helps optimize the most crucial pages that affect your brand and its revenue. And when talking of impactful pages, you simply can’t ignore the post-purchase pages — especially when you have upselling offers on them.

So, in this blog post, we’ll walk you through a step-by-step guide on how to A/B test post-purchase pages on Shopify. This guide will help you boost your revenue through A/B testing on post-purchase upsell offers.

Fundamentals of A/B Testing Post-Purchase Pages and Upsell Offers

Let’s start with some basics first!

What is A/B Testing in eCommerce?

A/B testing is a technique in which two different versions of a webpage are created and displayed to different visitors to identify which one performs better for a defined business goal.

In this article, the specific business goal that we’ll mainly focus on is increasing revenue through upselling offers displayed on post-purchase pages.

What are Post-Purchase Pages in Shopify?

As the name suggests, “post-purchase pages” are shown once a customer completes a purchase. Now, which and how many pages are shown to customers depends on how you set them up in your Shopify admin/app.

By default, once the checkout is completed by the customer, they’re shown a page that serves the purpose of both — the order confirmation and thank-you page.

But it’s up to you as to how you customize the post-purchase flow and experience. You can also display the order confirmation page and thank-you page separately with some app-based customizations.

What’s an Order Confirmation Page in Shopify?

The order confirmation page is displayed as soon as the buyer completes the purchase on a Shopify store. Typically, this page covers the following details: order confirmation number, email marketing consent, order details, links to other important pages (refund policy, privacy policy, terms of service, cancellation, and contact), Shop app download button, and a CTA button to continue shopping.

What’s a Thank-You Page in Shopify?

The thank-you page is shown after the purchase or order confirmation page, and it includes a message of gratitude for the customer. For example, if you’re a small business, you could write — “Thank you for shopping with our small business!”

Many brands leverage these post-purchase pages for upsell opportunities — and that’s where A/B testing comes into the picture to optimize the conversions of those upsell offers.

A/B Testing Post-Purchase Upsell Offer — How Does it Work?

Now, let’s understand this whole idea of A/B testing post-purchase upsell offers with a hypothetical example. Let’s say you’re selling clothing items. On your purchase pages, you’re displaying upsell offers of other complementary products.

As explained in the above example, the total traffic from completed purchases is split into two parts of 50% each. (This is why A/B testing is also known as split testing.) In the original version, customers are shown “Denim Jacket” as an upsell offer — and the other 50% customers are offered a “Hoodie”.

Now, since the hoodie gets a better conversion rate (10%) than the denim jacket offer (5%), it’ll be declared as the winner, and you can implement the changes in your upsell offer.

This is just one A/B testing idea for an upselling offer. Here are some more A/B testing ideas for the upsell page/offer on your Shopify store:

• A/B testing the same upsell product but with different discount offers

• A/B testing the same product with different copy

• A/B testing the product with different images or visuals

• A/B testing a single-product vs a product bundle offer

Learn more: Shopify Post-Purchase Upsell: Definition & How To Best Implement

Why A/B Test Post-Purchase Pages or Upsell Offers?

As we know, post-purchase pages are shown after the purchase is completed. So, you may have questions like: Why should I A/B test the order confirmation page? Or why should I A/B test the thank-you page?

Well, here are the benefits of A/B testing your Shopify post-purchase pages:

• Implement data-driven business decisions

Post-purchase pages shouldn’t be left as they are by default — they must be set up strategically because they impact your brand’s impression and revenue potential. Instead of designing your post-purchase strategy with guesswork, A/B testing helps you make robust decisions backed by data.

• Increase your overall revenue through upsells

If you’re not running upsell promotions on your post-purchase pages, you’re leaving money on the table. And if you’re running upsell promotions but not getting desired results or conversions, you can use A/B testing to refine your post-purchase upsell strategy.

• Enhance the user experience

Post-purchase pages are not just an opportunity to increase your revenue, but they also play a crucial role in building your brand’s connection with your customers. Using A/B testing, you can identify the areas of improvement in your post-purchase experience. 

• Optimize your post-purchase survey

If you’re running a post-purchase survey on your Shopify store but not getting enough responses, you can conduct an A/B test with a different version of the survey and see if you can improve the response rate.

h2>Step-by-Step Guide on How to A/B Test Post-Purchase Pages or Upsell Offers

Now, we’ll go through the detailed guide on how to A/B test the order confirmation page and its upsell offer — using the Sales Funnel feature in GemPages.

The Sales Funnel feature lets you create a complete and robust funnel solution that not only converts your leads into customers — but also increases post-purchase upsell opportunities.

Step 1: Define the Hypothesis for the Test

First things first — define the hypothesis as to which variable you’re going to test.

Defining a hypothesis gives a clear direction and goals to your A/B test. It determines which variable you’re going to change and what result you’re expecting from that change.

Here’s a sample hypothesis for this tutorial:

Theory: Based on the low conversion rate found in post-purchase upsell data, we believe that displaying a different upsell product for all new visitors will entice them to go for the purchase.

Validation: We will know this when we see an increase in the conversion rate of the upsell product.

Outcome: This will be good for our business because it will help us validate the upsell strategy and increase our overall revenue.

You can write your own hypothesis using this sample. Just customize it according to your business case and pain point. Make sure to consider the historical data when creating your A/B test hypothesis. The data will help you define which exact variable is crucial for the test.

Step 2: Create Two Versions of the Upsell Offer

Now, we need to create both versions of the post-purchase upsell offer. Let’s assume you’ve built a sales funnel for home decor products and you want to test two different products in the upsell offer.

Original Version: The upsell product — “Touch Dimming Wireless Table Lamp”

Variant: The upsell product — “Wireless Charging Bedside Lamp Table Lamp”

Important note: Make sure to test only one variable at a time. If you change multiple variables at the same time (e.g., upsell products + CTA button design), you won’t be able to accurately identify as to which variable impacted the test result.

Side note: For those who haven’t tried the GemPages Sales Funnel feature yet — you can create a new sales funnel by clicking on the “Create new funnel” button in the GemPages dashboard.

You’ll find two options:

• Post-purchase offers

• Full Sales Funnel

Learn more: GemPages Sales Funnels - The Ultimate Solution for Maximizing Your Shopify Store Profit 

For this tutorial, let’s assume you already have a funnel set up with either “Post-purchase Offers” or “Full Sales Funnel”. So, let’s go to the “Sales Funnels” tab in the left sidebar and click the edit button on the sales funnel/post-purchase funnel you’ve built.

As you can see, we already have the “Touch Dimming Wireless Table Lamp” added as an upsell product in the funnel. Click on the “Create A/B test” button to proceed.

By default, the A/B testing feature will create another variant of the same product. But we’ll change that because we want to display another product as a variant in this test. Click the “Change product” button below the variant (the second product on the right).

Then, we’ll add the variant product, i.e., “Wireless Charging Bedside Lamp Table Lamp”. Once added, click “Next: Setup discount”.

Pro Tip: Remember to test one variable at a time. Since we’re testing a different product as a variant, we won’t change the discount offer. So, we’ll keep the same discount (20%) on this product offer as well. If you change the product AND the discount offer as well, you won’t be able to identify if the change in the conversion resulted from the product change or the discount change.

Now, both products are ready for the test.

Step 3: Set Up and Run the A/B Test

Before you start the test, if you want to customize the traffic volume for each product variant, you can do so by clicking the settings icon. However, ideally, you should distribute equal traffic to each version. So, we’ll go ahead with 50% distribution for each product.

Lastly, click the “Start A/B test” button, just confirm it again, and your A/B test will be initiated immediately, as long as you have an active funnel. If your funnel is not activated yet, first, you need to activate the funnel to put the test into effect.

That’s it for the A/B testing process for the first upsell offer!

Bonus Step: Creating A/B Test for 2nd Upsell Offer

Now, we’d like you to know that there are more possibilities with the GemPages Sales Funnel feature. You can also add “2nd Upsell and Downsell” offers and A/B test those offers as well.

Select “Specific offers” if you want to offer a specific product as an upsell. Then select the main product you want to offer as an upsell. Set up the discount rate.

Now, your 2nd upsell offer is also ready. 

As we did the A/B test of the main product, you can also conduct an A/B test for the second product.

And it just doesn’t end here! You can follow the same process to create a “Downsell” offer and create an A/B test for that offer as well.

In a nutshell, GemPages Sales Funnel not only levels up your sales funnel game but also optimizes your upselling opportunities.

Step 4: End the Test and Implement the Changes

Run the test for the required timeframe and until you achieve statistical significance.

As a general rule, for an A/B test to be valid, you need to achieve 95% (or at a minimum 90%) of statistical significance. Ideally, run the test for at least two weeks.

Once you’ve identified a clear winner, meeting the mentioned criteria, you can end the test.

You can select the winner inside the GemPages Sales Funnel, and then the chosen version of the post-purchase upsell offer will be live in your sales funnel.

Step 5: Document the A/B Test Insights 

Whenever you’re running A/B tests of any sort, it’s imperative to document the test results and insights. These insights could be quite helpful in your future marketing campaigns.

Document how the customer behavior changes with each version included in the tests. 

You can use a simple spreadsheet or an advanced note-taking tool like Notion to create a database of all your A/B tests. Keep updating the database as and when needed — and don’t forget to go back to previous insights when running new tests.

Final Thoughts on A/B Testing Post-Purchase Pages

Post-purchase pages can be optimized and leveraged to increase your revenue.

However, keep in mind that it’s equally important to deliver the best-in-class user experience. Make sure to create upselling offers mindfully, and write a genuine message of gratitude on your thank-you page.

After all, your goal should be to earn loyal customers who keep coming back!

Last but not least — the “Sales Funnels & Post-Purchase Upsell” feature is available on the GemPages Optimize plan at just $59/month. Or just install GemPages for FREE to create and publish your first page.

To learn more about other eCommerce marketing strategies, tools, and best practices — check out more resources on the GemPages Blog. Also, join the GemPages Facebook community to network and learn from like-minded entrepreneurs and experts.

FAQs About the Post-Purchase A/B Testing