Get up and running with Chartbeat—full implementation guides for all our products.

Headline Testing

Technical Prerequisites

In order to be compatible with the current Headline Testing implementation, there are two prerequisites that your site must meet:

  1. Cookies must be enabled

    Because Headline Testing uses its own dedicated cookie, it is necessary that you have first-party cookies enabled for your site. If you have set the noCookies configuration variable you will not be able to conduct headline tests.

  2. Average peak of 200 concurrents on homepage

    Average audience size directly relates to how long headline tests take to complete. With a smaller audience size, tests will take longer and a higher percentage of tests may not conclude.

Initial Implementation

Depending on your site design and preferences, there are three implementation options that you may use. The recommended method is to add a synchronous script tag to load the Chartbeat test runner, however we do have an asynchronous option for customers who do not wish to insert a synchronous script on their site.

The Headline Testing code only needs to be implemented on pages that will run headline tests (in most cases landing pages or section verticals). The actual articles that you’re testing the headlines for do not need any additional implementation.

When you implement Headline Testing, there will be a total of two Chartbeat scripts on the pages on which you create headline experiments. To ensure the scripts communicate well with one another, you must make the following three code changes in the original Chartbeat Publishing script.

  1. On the pages where you’re adding the Headline Testing snippet, you must remove the following variables from the standard Publishing code from the <body>.

    _sf_async_config.uid
    _sf_async_config.domain

    Defining these variables in both scripts may create unexpected results in headline tests as well as Dashboard performance.

  2. If you’re setting either the path or useCanonical variable, they should be also moved to the Headline Testing script in the <head> as they are necessary for headline testing to function properly.

    Note: If you’re setting canonical links, you must put the rel="canonical" link element before the Headline Testing script in your the <head>.

  3. Update the beginning of the Pinger script to define the _sf_async_config object:

    var _sf_async_config = _sf_async_config || {};

For more information on all the _sf_async_config variables, you can check out our Chartbeat Publishing Getting Started Guide

Synchronous Implementation

Because Headline Testing uses its own dedicated cookie, it is necessary to have first-party cookies enabled for your site. If you have set the noCookies configuration variable you will not be able to conduct headline tests.

Include the following code snippet before the closing <head> on the pages in your site where you want to conduct headline tests.

<script type="text/javascript">
     var _sf_async_config = _sf_async_config || {};
     /** CONFIGURATION START **/
     _sf_async_config.uid = #####; /** CHANGE TO YOUR CHARTBEAT ACCOUNT ID **/
     _sf_async_config.domain = "yoursite.com"; /** CHANGE THIS **/
     /** CONFIGURATION END **/
     var _sf_startpt = (new Date()).getTime();
</script>
<script src="//static.chartbeat.com/js/chartbeat_mab.js"></script>

Please make sure that you also complete the Initial Implementation instructions.

Planning for Flicker Control

An undesired side effect of dynamic headline testing is the possibility that the end user will see a "flicker" in the test headlines. This would occur when a user sees a default headline suddenly change to a different test headline.

The synchronous implementation is recommended because it is the most straightforward way to ensure that document body is hidden and users don’t see a headline flicker.

To limit headline flicker, the Headline Tester code chartbeat_mab.js relies on the document body to be hidden while the headlines are being replaced. By default, the Headline Tester code will hide the body when chartbeat_mab.js loads. It then unhides the body after all headlines have been replaced, the Tester gives up on finding the headlines, or a one second timeout is reached.

Asynchronous Implementation With Flicker Control

Alternatively we do have an asynchronous implementation option that still hides the document and prevents headline flicker:

Note: This script should be placed before the closing </head>

<style id='chartbeat-flicker-control-style' type='text/css'>
    body {
    visibility: hidden !important;
    }
</style>
<script type='text/javascript'>
    var _sf_async_config = _sf_async_config || {};
    /** CONFIGURATION START **/
    _sf_async_config.uid = #####;
    _sf_async_config.domain = 'yoursite.com';
    _sf_async_config.useCanonical = true;
    /** CONFIGURATION END **/
    var _sf_startpt = (new Date()).getTime();
    // Set a timeout event for 1 second that will remove the body hiding
    // tag from the document if it has not already been removed. This
    // gives the Headline Tester script a total of 1 second to load and run to limit
    // the potential for flicker of headlines.
    // The one second time limit can be adjusted to client preferences.
    window.setTimeout(function() {
        var hider = document.getElementById('chartbeat-flicker-control-style');
        if (hider) {
            hider.parentNode.removeChild(hider);
        }
    }, 1000);
</script>
<script async src = "//static.chartbeat.com/js/chartbeat_mab.js"></script>

With the addition of the client-set style tag and the async attribute on the chartbeat_mab.js script tag, the Headline Tester code can now load asynchronously and ensure that visitors only see test headlines with no flicker.

The Headline Tester will load and replace tested headlines then remove the chartbeat-flickercontrol-style style tag (unhiding the page).

In the very unlikely scenario that the headlines aren’t replaced within the time limit, the page will be unhidden once that limit is reached. The time limit set to 1000 ms by default (second argument of the window.setTimeout function) and is configurable.

Please make sure that you also complete the Initial Implementation instructions.

Asynchronous Implementation Without Flicker Control

If you would still like to implement an asynchronous script and opt out of the flicker control option, you can use the following code snippet before the closing </head> on the pages in your site where you want to conduct headline tests.

<script type="text/javascript">
    var _sf_async_config = _sf_async_config || {};
    /** CONFIGURATION START **/
    _sf_async_config.uid = #####; /** CHANGE TO YOUR CHARTBEAT ACCOUNT ID **/
    _sf_async_config.domain = 'yoursite.com'; /** CHANGE THIS **/
    _sf_async_config.flickerControl = false;
    _sf_async_config.useCanonical = true;
    /** CONFIGURATION END **/
    var _sf_startpt = (new Date()).getTime();
</script>
<script async src="//static.chartbeat.com/js/chartbeat_mab.js"></script>

Note that this option will likely cause the end user will see a "flicker" in the test headlines. This would occur when a user sees a default headline suddenly change to a different test headline.

Please make sure that you also complete the Initial Implementation instructions.

Staging, Subdomains, and Permissions

Staging and QA

There are pros and cons to implementing the Headline Testing tool in a staging environment first, before going into production.

If you’re concerned about page flicker, load time, workflow, or team resources, you may want to test the Headline tool in a staging environment first.

The primary drawback to staging is simulating traffic to actually test the tool. The only way to simulate traffic is to create a test then open an incognito window and visit the page with the test on it. You will see the trial counter increment, then if you click on the headline you’ll see the CTR bar shift, and play percentage go up slightly.

Tracking Subdomains

When tracking click-through-rates across multiple subdomains, you’ll need to set the _sf_async_config.topStorageDomain variable to the top-level domain that you want to store data for. .

_sf_async_config.topStorageDomain = 'top_level_domain';

For example if you’re performing experiments on “news.site.com” and one of the headlines links to a story on the domain “site.com”, you would set the topStorageDomain parameter to “site.com”.

The Headline Tester code uses the following prioritization for storing click-through data.

  1. The value of _sf_async_config.topStorageDomain (if it is set)

  2. _sf_async_config.domain if the value is a subdomain or matches the actual domain.

  3. The default as the actual domain of the current page.

User Permissions

As Chartbeat login seats with permissions to the Headline Testing tool have the ability to affect the actual page that is being served to your audience, we are very strict with granting user permissions. At this time all permissions must be manually added to our back-end, so all account users who need headline testing access must contact their Chartbeat Account Manager.

Technical FAQ

Is there CMS integration?

Not at this time. While changing headlines in your CMS is currently a manual step, it’s actually not entirely necessary as Chartbeat will play winning headlines to all visitors as long as the finished experiment stays on the homepage.

What happens if I change the headline in our CMS during an experiment?

The test will continue. The headline testing javascript looks for experiments by location in the DOM (xpath), href to the story. If the headline changes the test will continue. If the href to the story changes, however, the test will stop.

What browsers will Headline Tests work in?

Tests can only be created in Google Chrome.

Will the synchronous script affect page load time?

Most likely yes but the effect should be imperceptible – approximately 200 ms or 2/10ths of a second.

Why do many of my headline experiments complete at or just after 20 minutes?

If there's no clear winner at the 20 minute mark of an experiement, a winner will be determined using an alternate method called 'soft convergence'. After 20 minutes, if we're 95% confident that the no headline is better by a margin of 25%, the leading headline will win.

Looking for something else? We’re happy to help.