Setting Up A/B Split Testing

A/B testing (also called split testing or bucket testing) is a method of marketing testing by which a baseline control sample is compared to one or a variety of single-variable test samples in order to improve response or conversion rates. In A/B testing, you have two (or more) different situations: Situation 1 and Situation 2. There are two restrictions which will allow you to identify the impact of each change: (1) there should be only one difference between the situations, and (2) the situations should be run simultaneously.  For example, if you change the price of your product in mid-December, it will be difficult to tell whether an increase in purchases is due to the new price or more users shopping for the holiday season. Instead, it would be better to run 50% of your purchase links to one price and 50% to another price, so that you can directly compare the results.

Possible A/B tests using SpringBoard

In SpringBoard, the easiest way to run A/B testing is to take a product that you have already setup and duplicate the product. Once you have duplicated the product, you can make a change to it or its template style.

Some experiments that can easily be tested to determine if they increase profitability include, but are not limited to:

• Changing the price
• Offering an automatic discount
• Showing a coupon field earlier in the order process
• Comparing traditional pricing versus pay what you want pricing
• Altering the product image or description
• Adding or deleting
  • Survey field(s)
  • Cross-sells
  • Up-sells
• Changing the flow of the order process by selecting a different page linking option
• Manipulating the style template
  • Comparing the default style template to a custom style template
  • Adding or removing a main menu or navigation bar
  • Simplifying a header or footer
  • Switching colors, font sizes, etc.
  • Altering style options

FastSpring Setup

It is important to note that this a/b testing setup will only work with page linking options one, three and five because they do not use forms. 

You can take any of these regular FastSpring product links and append a second product link to the query string. This will direct 50% of traffic to the first product and 50% of traffic to the second product. The second product link must be URL encoded, and a helpful, free tool for encoding is provided here. For additional information, see Wikipedia's articles on URL encoding and Percent encoding.

The full URL to link to your product would be 

URL1?b=URL2(encoded)

where you would replace URL1 with your original product link and URL2(encoded) with the encoded version of the variation product link.

An Example - Determining whether a survey field affects sales

If you want to see whether the addition of a survey field affects sales, you can go to SpringBoard's Store Home » Products and Pages, and select a product to copy. Click Duplicate and enter in the Product Naming information. We suggest you add "with survey" to the Name, which is used for management purposes only, and make the Display name the same as your original product. Under Fulfillment Actions, select Reuse from the drop-downs, if desired, and then click Create.

Follow our directions for adding a custom field, such as a survey, to an order or order item notification. 

Next, go to SpringBoard's Store Home » Custom Fields. Click on the survey you just created. Click Edit. Select Product Condition and click Next.

Using the drop-down menu, select the product you recently created (using Duplicate Product) and click Create.

To get the URLs for these products, go to SpringBoard's Store Home » Products and Pages, and select your original product. Choose the page linking option you wish to use. This html link will be URL1.

In our example, URL1 = https://sites.fastspring.com/julie/instant/testcompanyfontmaker40

Then go back and select the duplicate product (the one with the survey), and get the html link for URL2

In our example, URL2 = https://sites.fastspring.com/julie/instant/testcompanyfontmaker40withsurvey

You must then encode URL2. Copy and paste your URL2 into the URL Encoder, and click Encode. This is now URL2(encoded).

The URL you would use, which 50% of the time would go to the original product and 50% of the time would go to the duplicate product that has a survey field as part of the checkout process, is URL1?b=URL2(encoded). In our example, the html link would be

sites.fastspring.com/julie/instant/testcompanyfontmaker40?b=https%3A%2F%2Fsites.fastspring.com%2Fjulie%2Finstant%2Ftestcompanyfontmaker40withsurvey

Since 50% of the time potential customers will reach the original product and 50% of the time they will reach the duplicate product (with the survey as part of the checkout process), all you will need to do is wait until you have some sales data and see if your sales are greater for one option.

Split Testing using JavaScript

If you want to do split testing by adding tracking to the order completion page of your FastSpring store using JavaScript, go to SpringBoard's Store Home » External Tracking. Click Add Tracking Method. 

Select Custom Tracking Method, and click Next. Enter in a Name, such as split testing, for this tracking method (the name is used for management purposes). Under </body> (before), click Click to Define Content. Enter in the JavaScript order tracking code. Do not include <noscript> or <iframe> in your code. Add <div> at the very beginning of the code and add </div> at the very end of the code. Click Next and click Save.

We're Here to Help

If you need assistance with setting up A/B split testing, please open a support ticket.