Headline Testing main metrics

 
  1. Trial
    A trial is every time that a test headline is loaded in a page and served to a visitor. For example a headline that ran a total of 2,000 trials was served to 2,000 unique cookies. On the other hand, if an experiment had 5,000 trials it means that headlines were served a total of 5,000 times.
     
  2. Click Through Rate (CTR)
    Percentage of visitors who clicks on a given trial headline (can be lower than 1%).
     
  3. Play Percentage
    The percentage of people are being served a given headline, as well as the confidence that headline is “better”. For example at 65% certainty, a headline is served to 65% of people and has a 65% certainty that it is"better". Note that at 95% play percentage a headline "wins" and is played 100% of the time.
     
  4. Quality Clicks
    When a visitor clicks onto a story and subsequently spends at least 15 seconds of engaged time, it is considered a quality click. Note whenever a visitor reaches the threshold of 15 engaged seconds 

Looking for something else? We’re happy to help.