What We Learned from Hundreds of Blog Post Headlines

Last fall, I wrote a post about how to use CoSchedule’s Headline Analyzer to write great blog post headlines.

That got me to thinking: we’ve sold thousands of blog posts; maybe we can use this tool to learn more about our product and our customers. But, seriously: Thanks to CoSchedule and their free online tool, we were able to run a study to investigate this key part of our product.

As a result, we learned a LOT about our service. Among other things, we learned about our writers’ habits, our customers’ preferences, what we can do to improve our product, and how we can help our customers get posts that are more shareable, more clickable, and more likely to drive traffic to their sites.

Why Analyze Blog Post Headlines?

A good headline is a crucial part of a good blog post, and that’s our core product. There have been numerous studies showing that headlines can have a dramatic effect on how often a blog post is read and shared. And that’s important to us! Not only do we want to make sure that our customers are getting the best possible content, but on our open-market platform, it could have an impact on how customers purchase – and evaluate – the content we sell.

Here’s how we set up the study, what we learned, and what’s next.

Max Newhall

The Setup:

To learn more about the blog post headlines that were written on our platform, we took every post submitted to our site over a two-day period and extracted all of the posts that were purchased or declined.

In the end, we took a sample of 279 headlines from 158 different writers and subjected that data to extensive testing.

We ran every headline through CoSchedule’s Analyzer and collected data like:

• Headline type (generic, question, how-to, etc.)
• Sentiment (positive, negative, neutral)
• % words balance (common, uncommon, emotional, power)
• Length (characters and words)

And, of course, we also collected the overall headline score.

In addition to that, we pulled some relevant data about the posts themselves and the writers who submitted them. We added data about things like:

• Whether the post was purchased or declined by a customer
• How the post was rated by the customer (1-5 stars)
• How long a customer waited to purchase or decline the post
• The length of the post (words)
• The level of the writer within our system

Once we had all of this data on a few hundred posts (and their headlines), we were ready to do a deep dive into the world of data analysis.

The Results:

Warning: In order to really understand these results, I highly recommend a quick read of our blog post about how to use CoSchedule’s Headline Analyzer.

The Analyzer has some quirks. For example, it isn’t great at detecting total nonsense (which can get great scores) and it doesn’t “know” a lot of specialized vocabulary (which can cause low scores where they aren’t merited). However, overall we find that these scores are a good indicator of the sharability and clickability of a headline, especially when paired with the other data points.

The average headline in our study, interestingly enough, fell right within CoSchedule’s norms for what makes a great headline. 

CoSchedule’s ideal headline:

• A score of 55+
• 5-9 words (6 is optimal)
• 30-65 characters (55 is optimal)

The average Verblio (formerly BlogMutt) headline in our sample looked like this:

• Total score of 57.44
• 8.9 words long
• 54.92 characters overall

Pretty close, right? Our headlines were a little above the ideal length in word count, according to their guidelines, but exactly on-point in terms of characters.

Findings that matched expectations based on CoSchedule’s scoring algorithm:

• Score:
  • Posts with headlines receiving scores between 60-70 were more likely to be purchased than posts with headlines that scored lower than 60.
  • Posts receiving 5-star reviews were more likely to have headlines receiving above-average scores (57+).

• Type: Posts with list, question, and time headlines performed better on average than posts with generic or how-to headlines.
• Length: Posts with headlines that fell within CoSchedule’s length norms (5-9 words & 30-65 characters) were more likely to be purchased than those with headlines that were too long or too short.
• Word balance: Posts that were purchased or declined in below-average time have higher scores, and more emotional and power words.

What Does It All Mean?

Basically, the findings above provide some evidence that beyond just evaluating shareability in a public context, CoSchedule’s headline analyzer can help give us an idea of whether a customer will want to purchase a certain piece of content. It also tells us that – at least according to their algorithm – most of our writers are on the right track with how they write their headlines.

This makes sense.

Our customers want to buy content that is shareable for their audience, so they are choosing content that meets those needs. Our writers want to sell content, so they are writing content to fit those needs. Win-win, right?

However, we found three specific metrics that indicate to us that we still have some work to do before perfecting the headline.

Further Findings & Takeaways:

1. Blog post headlines did not perform identically across post length categories.

For example, posts in the 600+ word category (in the range of 600-900 words) were significantly more likely to be purchased, and also received higher-than-average scores than shorter or longer posts. 

Headlines for posts at the 250+ word level (250-600 words) tended to receive lower scores on average. Confusingly, this does not correlate with any specific group of writers. Headline scores were not significantly different for high- vs. low-level writers.

We suspect that this might be related to the types of content created for various word lengths. Perhaps, it is more challenging to write a compelling headline for a super-short post?

At any rate, a big takeaway for us here is that while we’re doing great on headlines for some posts, we have room for improvement in other categories.

What’s next?

In the coming weeks, we’ll be working to educate our writers about crafting the best possible headlines.

2. Posts with super-high-scoring headlines (70+) actually had lower-than-average purchase rates.

Although blog post headlines scoring 60-70 seemed to contribute to a post’s purchasability, headlines that most accurately matched CoSchedule’s ideal thresholds did not. There are several possible reasons for this.

Here are some ways that these super-high-scoring blog post headlines differed from the rest:

• WAY higher percentages of emotional words, and higher-than-average percentages of uncommon and power words.
• Slightly shorter than average.
• More likely to have strong positive or negative sentiments.

We think the most likely reason relates to sentiment, as evidenced by our third unusual finding…

3. Customers were significantly more likely to purchase from blog post headlines with a neutral sentiment than a strongly positive or negative sentiment.

This is perhaps our most unusual finding, but it is also one of our most statistically significant. It is unusual in relation to our other findings because research shows that headlines with strong sentiments are actually more likely to be clicked on and shared than those with neutral sentiments.

So, why don’t our customers want to purchase headlines that contain strong sentiment?

One possible explanation is that customers want to write their opinion pieces in-house. this way they can leave the more neutral, evergreen content to be outsourced. However, this doesn’t really fit with our experiences on the whole. We’re writing all of the content published by many of our customers.

Instead, I suspect that this is just a result of a common fear among small-business owners, in particular: controversy.

The wrong kind of controversy can really damage a company, as Uber’s now-former CEO Travis Kalanick has been proving over and over again in recent news. However, the right kind of controversy is a powerful tool for stirring up press and even sales.

Just ask Patagonia, the company that stirred up a whole lot of sentiments with their recent campaign to protect Bears Ears National Monument.

What’s next?

We’ll need to do some more digging into this phenomenon before we can conclusively name the reason behind it. Once we have a clearer idea of WHY this happens, we’ll have a much better idea of WHAT we need to do about it.

We’ll update you if we discover the secret to selling blog post headlines with controversial content along the way.

Thank you so much to CoSchedule, again, for providing the free tool that enabled this study about blog post headlines. As a bootstrapped small business dedicated to making the content world a better place, we applaud your work. We also applaud the educational value and accessibility of your tool to empower small teams like ours.