A/B testing on WordPress: a framework for developers and tutorial

Some months ago, I changed one link in the menu in my website postpaycounter.com. After that, it looked to me more people were purchasing products, i.e. the conversion rate had increased. But how to check whther that was really the case, or if it was just an accident/impression? Use an A/B test, I told myself!

With an A/B test, half of the users are served one version of the page, the one with the old link, and half of them another version of it, the one with the new link in place. When a sale happens, we may then log that as a success for the kind of page that was used, be it the A version or the B one.

In my case, the two versions of the page simply consisted of two different links in the menu, while I wanted the success to be logged when the user purchased something (I use Easy Digital Downloads to handle purchases).

I could find a bunch of plugins that allowed to set up A/B tests, but they all seemed pretty difficult to customize from a developer perspective, and I was already seeing myself wrestling with someone else's code that provide tons of features useless to me, but through which was nearly impossible to interact with Easy Digital Downloads. So I decided to build my own, simple implementation, with the aim of it being tailored to developers rather than users who needed an interface.

An A/B test implementation example

This is an example of how to use the little framework. To set up a test, you only need to provide two functions:

• A_function specifies what should happen to the page that is served to the first half of visitors. In my case, I attach a filter to edit the navigation menu with a custom function.
• B_function works the same for specifying what should happen to the second version of the page. Again, I attach a filter to the navigation menu.

At this point, you can set up a test object, through which you will be able to retrieve the almost-unique alphanumeric test ID (it's a MD5 hash of the setup arguments).

Finally, you need a third function that is going to log successes. You should probably have it attached to some action or filter. In my case, I used an action that fires when a purchase through Easy Digital Downloads happens. The only requirement about the test function is that it should call AB_log_success( $test_id ), that will handle the under-the-hood success logging, basically updating the wp_option (AB_tests) containing the stats.

<?php
global $test_id;
$args = array(
    'label' => 'Menu link',
    'A_function' => 'test_A_function',
    'B_function' => 'test_B_function'
);
$test = new AB_test( $args );
$test_id = $test->get_ABT_id();

function test_A_function() {
    add_filter( 'wp_get_nav_menu_items', 'leave_untouched', 10, 3 );
}
function test_B_function() {
    add_filter( 'wp_get_nav_menu_items', 'change_menu', 10, 3 );
}

function log_success( $download_id, $payment_id, $download_type ) {
    global $test_id;

    AB_log_success( $test_id );
}
add_action( 'edd_complete_download_purchase', 'log_success', 10, 3 );

For the full code, see the GitHub repo!