Headline testing is a multi-variate approach to running live headline tests so you can evaluate in real-time which headlines tend to attract more click-through.
The platform uses a multiarmed bandit approach to determine if one headline is more successful at attracting interest from visitors (clicks) and holding the visitors’ attention for at least 15 seconds once they get to the page .
The multi-armed bandit approach, called Thompson Sampling, is different than other multivariate testing methodologies in that it exploits the benefits of the winning headline during the test. Essentially when a headline begins to perform better, Chartbeat will ‘play’ that headline more frequently.
For more about Thompson Sampling, check out this methodology report from our Data Science team.
To actually create a headline experiment you’ll simply select an article on your homepage (or other high traffic page) and input a number of headlines that you want to test. Then, Chartbeat will dynamically serve those different headlines to your audience equally, and as certain headlines begin to receive a higher click-through rate and engagement, they will be served more frequently to your audience while unsuccessful headlines stop being served.
The test ends when the most popular headline is determined, and Chartbeat will automatically serve that headline 100% of the time.