Marketers at agencies have their hands full. Between juggling deadlines, contractor communications, cross-team functionality, client expectations, and — somehow — still making time for every lunch-and-learn or mandatory midday stretch sesh, there isn’t always a lot of time left over for the nitty gritty. Having all hands on deck to line-edit, fact-check, and proofread every piece of content that comes down the pipeline isn’t sustainable. And if you’re an agency that’s scaling with rapidly increasing content needs, AI content sounds like a godsend.
OpenAI’s content-writing tools are fast, cost-effective, and don’t take sick days. It’s already done its research — 45 TB of compressed plaintext worth of research, that is. And, if you don’t want to fiddle with all the GPT-3 parameters yourself, Jasper is the biggest name in the game for straightforward AI content.
Theoretically, this service should be able to produce sparkling content that is polished and ready to send to clients.
So, does it?
Experiment methodology
I want to see if using Jasper’s content writing services can cut editing time in half. This would be a major time-saver for agencies who currently rely entirely on human-written content. It could also have huge repercussions with how agencies staff, manage, and budget for their teams, particularly for agencies that are scaling quickly.
Hypothesis: Editing times on content written by Jasper will see at least a 50% reduction compared to human-written content.
For this experiment, I’ll be asking Jasper for a real estate article, and I’m doing this for a couple reasons.
Throughout my career, I have ideated, written, edited, and/or quality assessed over 8,000 real estate articles for hundreds of agents nationwide. With about six million words worth of experience in this field, I feel like I’ve had the chance to understand what a client really looks for when they receive a piece of content.
I’m also selecting a real estate topic because successful content in that industry requires a base understanding of the local area, seamless integration of keywords, and an assumed expertise on behalf of the client. It can be tricky for even seasoned writers, and I want to see how AI-written content compares.
Control: Editing one 500 word piece of human-written content takes, on average, fifteen minutes.
Historically, my average editing time per article written by humans is fifteen minutes for a full sweep. What, then, constitutes editing?
- Line-editing for clarity and overall readability
- Checking for internal and external linking best practices
- Fact-checking for accuracy
- Proofreading for grammatical errors and compliance with a style guide
- Quality assessment of the article, including checks for broken links
When all is said and done, I’ll have a piece worthy of posting far and wide.
Results analysis: time, editing difficulty, and gut-feel.
Time is the most straightforward metric in this experiment. If it takes me ten minutes (or less!), this hypothesis will test true. If it takes longer, the hypothesis will be false.
But for the sake of a comprehensive look at what it’s like to turn Jasper musings into agency-ready content, I want to look at two other factors: difficulty of editing and gut-feel.
Editing difficulty will be measured on intensity and complexity of edits. Changing a few commas is a low-lift edit that takes minimal time and effort. Rewriting or restructuring core pieces of the article is a higher lift edit that requires greater focus and higher skill as well as, of course, more time.
Gut-feel is marketing’s yellow-brick road. You trust it will take you where you need to go, even if it feels a bit fantastical. If something feels off, it usually is.
What does agency-ready content look like?
Every client at every agency has different expectations for what high quality content looks like, but I think we can come to a few agreements.
- Content should be free of any spelling or grammatical errors.
- There should be no factual inaccuracies or outdated information.
- The client’s desired tone of voice and style should be emulated.
- Each article should include keywords that align with marketing goals.
If you’ve used AI tools to create content, you probably already see a few potential pitfalls.
Content should be free of any spelling or grammatical errors
AI can typically write a grammatically correct sentence. Save for a few blunders here or there, this shouldn’t be the biggest time-suck of the editing process. This is where, depending on the experience of your writers and their personal blind-spots to things like comma splices, the time to edit human-written content may be longer.
There should be no factual inaccuracies or outdated information
Here’s where the other shoe drops. Jasper content loves to write sentences. That’s what it’s programmed to do. Unfortunately for the sake of reliable, expedited copywriting, it means that AI is perfectly willing to create fabricated factoids that sound reasonable as long as you don’t look too closely or think too long.
It is trained to replicate humanistic writing patterns. Not write articles with real credibility. When you bring the topic of outdated information into account, you reach a particularly interesting impasse. The API that Jasper is built on, OpenAI’s GPT-3, has no concept of current events — it stopped learning early last year.
In an ever-changing market like real estate, this spells trouble.
Humans, on the other hand, can provide up-to-date information so long as they can find it. With real estate writing — and particularly remote real estate writing about somewhere you may or may not have lived or even visited before — the trickiest part is accurately capturing the city as it feels to residents. This includes local closings, seasonality, breaking news, and the ever elusive “community vibe” that requires researching reliable online sources. Otherwise, the writing feels out of touch with reality and the agents who know their area best are disappointed.
Editing human-written content still requires fact-checking, but there’s a natural bias I’d like to acknowledge where we’re inclined to trust that our incredibly capable colleagues have done their fair share of work because that’s why they were hired. Choosing to utilize Jasper’s services counts as hiring them, and I’m interested in seeing how trustworthy this new robotic colleague’s writing turns out to be.
Humans can provide up-to-date information so long as they can find it. With real estate writing, the trickiest part is accurately capturing the city as it feels to residents. This includes local closings, seasonality, breaking news, and the ever elusive “community vibe.” Otherwise, the writing feels out of touch with reality and the agents who know their area best are disappointed.
The client’s desired tone of voice and style should be emulated
As an agency, you’re used to the quick-pivot required to cater to each client’s individual needs. Style and tone of voice are just a few of those. You might write back-to-back articles for a no-nonsense client who is looking for dry, academic market updates and an offbeat client aiming for out-there attractions.
Being a marketing agency means reaching expert-level acting skills. Every piece has to sound like it’s coming straight from the mouth of the client, or it loses credibility. If it’s off-brand, out of sync with their established style, or structured differently than the majority of their other content, you’ll often be asked for revisions or rewrites. This, of course, counts as time spent editing.
AI offers the unique ability to produce content faster and with less overhead, but not with less effort.If you want success with these kinds of articles while using AI to write content, you should anticipate spending time fine-tuning the parameters and training the models on what successful content looks like.
Human Crafted AI content, on the other hand, gives you the best of both worlds — human expertise and oversight, AI speed and pricing, and we’ll train the models for you.
Each article should include keywords that align with marketing goals
And, through it all, the content needs to actually be useful. Your job at a marketing agency is to make your clients’ businesses shine. You’ll have proposed marketing strategies that you now need to live up to, and your clients are chomping at the bit to see the results (and the ROI) you’ve promised.
Keywords are a frequently used tool to help meet these goals. You establish which keywords will be most effective in reaching your target audience (in real estate, this often includes long tail keywords like “houses for sale in West Palm Beach” as well as opportunities to target local SEO opportunities like “best Jersey Shore ice cream shops”) and then include those in your article according to best practices. Bada bing, bada boom. Your content will start to rank. Leads will generate. Everyone is happy.
Of course, a key here is to make sure the keywords don’t sound like keywords. If you (or your robot counterpart) cram keyword phrases into the text willy nilly, you’ll disrupt the flow of reading. It’ll sound unnatural, unprofessional, and under-developed. A competent copywriter will know how to use keywords without them feeling clunky. AI, on the other hand, probably won’t.
Editing an article written using Jasper
Before I start the editing timer, I need Jasper to generate an article. I want to target two keywords: Best bookstores in Charleston and Charleston real estate. I don’t want the blog to be too serious, but I still want readers to think I know what I’m talking about. So, using their Document tool, I put in the following description and watched it work its magic.
Two minutes later, there are five hundred words in front of me. I’ll be honest. If a freelance writer had submitted this article, I would have immediately requested serious revisions.
The produced article includes mention of eight “Charleston bookstores,” but there’s a catch.
- One is a library, not a bookstore
- Two bookstores aren’t located in Charleston
- Five bookstores don’t exist at all
Which leaves one. The article was written about one actual bookstore in Charleston.
I see my mistake, and as someone who understands the importance of providing detailed briefs to writers, I should have anticipated the gobbledygook that Jasper produced. It needed more direction—a lot more.
But at least it started off strong.
Blue Bicycle Books is a great bookstore in Charleston. I’ve been there! My actual, real life, human experience confirms that this article would be remiss Blue Bicycle Books not to mention it. I gave the intro and this first section a little face lift, added an H1, fixed the glaringly incorrect founding date, and went on my merry way. You’ll see my changes below in purple.
This is where the article truly starts to struggle.
The Ollie Store in North Charleston? I believe this refers to Ollie’s Bargain Outlet, a chain of low-priced goods. In Jasper’s defense, Ollie’s Bargain Outlet is located in North Charleston, and you can probably find books there. Unfortunately, it’s not exactly what we’re looking for if we search “Best bookstores in Charleston.”
Next on the list: City Lights Booksellers. This one is painful. City Lights Bookstore is a famed beatnik-era bookstore in San Francisco founded by poet Lawrence Ferlinghetti. So, while it was on the right track (bookstores), it missed Charleston, South Carolina by about 2,700 miles. Bunch of Grapes Bookstore falls to a similar fate. This is a very real bookstore in Vineyard Haven, Massachusetts.
Best Buy Books & Coffee? Pretty sure that’s… Best Buy. The electronics store. And, sure, you can probably buy an e-reader there and maybe even a coffee maker, but it’s not a bookstore.
And then, there’s Charleston Real Estate Bookstore. Here, I’m reminded that AI content generators don’t actually know anything. They were trained on human typing patterns, and because I indicated that I wanted to highlight Charleston real estate… it found a way. It just wasn’t the right way. There is no Charleston Real Estate Bookstore specializing in books about real estate — I checked.
Plus, The Rare Book room at Calhoun Street Books & Collectibles sounds so freaking cool that I’m actually mad that it’s entirely made up.
Last but not least:
The Charleston Library Society tries so hard to be right. This is a super real (super cool) establishment — the country’s third old subscription library, second oldest circulating library, and what the official site calls “the oldest cultural institution in the South.” But it’s not a bookstore! I do want to note, however, that Jasper nailed the founding date here, and I’m impressed.
Ultimately, this entire section needed to be re-researched and rewritten. (Reminder: everything in purple is what I rewrote. The green is what I’d salvaged from Jasper’s text.)
While rewriting, I reworked some of the conclusion and, because AI content can’t cite anything, I needed to make sure links to each featured bookstore had been included. This is a fairly quick exercise, but it tacked on a few seconds of searching for each bookstore.
How long does it take to produce a usable piece of content using Jasper?
Time spent editing: 29 minutes 37 seconds
Editing difficulty: 3/5
Gut-feel: UGH.
Time spent editing an AI-generated article
Thirty minutes to edit a blog isn’t that long. It ended up taking me slightly longer than average to complete this piece. But if this blog was one of hundreds in the pipeline, all requiring similar edits, I’d be truly overwhelmed. Plus, I could do this in my sleep.
Difficulty of editing an article from Jasper
On a scale of one to five — one being the simplest and five being the most challenging — I gave this edit a three. It was not intellectually difficult. Jasper produced clear, easy to read sentences that varied in delivery. I didn’t find a single grammatical or spelling error. But, since I had to rewrite large swaths of the article, it still required me to get my hands dirty. This edit was more complex than a top-level overview or a quick check for misplaced commas. So, three it is.
Gut-feel
It did not feel good to receive a piece of content that was effectively defunct. Having content of this caliber received on any day at a high-producing agency could seriously interrupt workflow and cause backlogs, let alone receiving thousands like this. If low-quality articles like this slip through the cracks, your agency could be facing massive escalations and major churn rates.
Find the content creation method that works for you
This was just one experiment. Maybe this isn’t the best use-case for AI content, and certainly refining the parameters could see improvement on the content output. But if you’re looking for a one-and-done content producer, this might not be the best fit.
Creating content using AI tools like Jasper could certainly be cheaper and faster than working with human-only, but it won’t be better.
At Verblio, we’re all about figuring out the best ways for you to have content you love, whether you’re an agency representing hundreds of clients or a small business reaching local customers. Every day, we’re experimenting with AI content to find out when and how this tool is best used by marketers. Our Human Crafted AI Content solution offers the opportunity to have content created faster and more cost-efficiently than ever without sacrificing quality along the way.
