Trial A trial is every time that a test headline is loaded in a page and served to a visitor. For example a headline that ran a total of 2,000 trials was served to 2,000 unique cookies. On the other hand, if an experiment had 5,000 trials it means that headlines were served a total of 5,000 times.
Click Through Rate (CTR) Percentage of visitors who clicks on a given trial headline (can be lower than 1%).
Play Percentage The percentage of people are being served a given headline, as well as the confidence that headline is “better”. For example at 65% certainty, a headline is served to 65% of people and has a 65% certainty that it is"better". Note that at 95% play percentage a headline "wins" and is played 100% of the time.
Quality Clicks When a visitor clicks onto a story and subsequently spends at least 15 seconds of engaged time, it is considered a quality click. Note whenever a visitor reaches the threshold of 15 engaged seconds