Legal Content vs. AI Detectors: Addressing False Readings and Ensuring Authenticity

In the same way that computers changed the world and how we do business, so did – or so will – chatGPT and similar platforms. At first glance, we were excited about this tool and what it could do for our work. After months of testing and working with it, we see so many opportunities for this technology to save time ordering lists, pulling data from reports, offering a list of adjectives, styling names, and page titles… The list goes on. But in one way, we are left puzzled and unsure – content writing. 

Ensuring Originality in Legal Content Writing: Is it Possible?

We know the problem is not uniquely ours, but that doesn’t make it any less important. How do we identify AI-written content? At PaperStreet, one of our core principles is to have a writing team of attorneys. 

Every writer who works on content for our clients has a JD. Many of them are retired attorneys. Some are attorneys who practiced for a part of their lives and stepped away to pursue family or other priorities. Some are like our founder, Pete Boyd, and client relations manager are trained attorneys who later thought they could do something else with their lives using their law degree. 

Our writers sign our AI policy, which states they are not to use chatGPT or similar programs for composing content, but how can we be sure they are not generating AI content when the AI content checkers out there are so widely different, or – should we say – inaccurate?

We pay our writers for their time and efforts. Some we have worked with for decades (literally over ten years). As a business, we don’t want to put out subpar content for our clients. We don’t want to pay for AI content. We want to pay for original material composed by writers with an advanced degree. We have high stakes in ensuring that the content we present is original. 

Where Algorithms Fail, Human Editors Prevail

We have extensively tested several platforms for checking for AI content, and even content that our Content Director personally wrote in 2014, well before the existence of AI, comes back showing a high probability of AI content generation. Tests like this make us question whether or not these tools can be trusted. Their algorithms are flawed, so what does that mean for an organization that writes more than 300 pieces of content for lawyers nationwide? How do we gatekeep and ensure the content we’re putting out is original and not tainted by AI-generated junk?

The most reliable method for PaperStreet is human editors. We have a three-person team of editors here with advanced communications and journalism experience. Our Blog Manager is also a former attorney. We believe the best way to determine whether content is AI-generated is for a human to edit that copy. 

In some instances where we have been editing content from the same writers for years, it would be obvious when something was off. Every writer has crutches – little signatures they leave in their adjective choices, sentence structures, and introductory clauses that give them away. Our editorial team knows these little signatures, and their absence would indicate an issue. 

There are also commonalities within AI-generated content. For example, the bullet structure is always the same. With human editors, we can be aware of these commonalities and use them to identify any red flags. 

Exposing the Flaws of Popular AI-Content Detectors

We offer the following example that details how the AI content checkers that are currently on the market are unreliable. It isn’t what we hoped for when this advanced technology entered the workplace. We hoped there would be a reliable check and balance in place that allowed us to maintain credibility and originality. 

As a team of creatives, we hope that the geniuses of the technology world will soon create something that we can depend on to give you 100% assurance that your content from PaperStreet is straight from the mind of a kick-ass attorney writer. 

Our AI-Content Writing Test

In the video below, we show you an authentic writing process. We’ve sped up the video for your sanity, but you can see a writer adding text to the page in real-time. It’s raw and unedited, but it serves our point. The writer never clicks away from the screen or copies from other sources. The text is as original as it can get. 

After demonstrating the writing process, we also show you the test results live. As you can see, the results are mixed and, by our definition, inaccurate.

The Results? 50% Accuracy

We used four of the most popular and high-ranking AI content detection tools. Some are free, and some are paid. Two of the four identified the writing as human text. The other identified it as over 80% AI-generated. Whelp.

• https://www.zerogpt.com/ – 83.24% AI 

• https://copyleaks.com/ – Human text

• https://www.getmerlin.in/ – 83% AI 

• https://app.gptzero.me/– 1% AI 

A Promise From a Team of Creative Hearts

We are a full-service web design and marketing agency. One of our most valued principles is our creativity and originality. It is why we are trusted to brand more than 50 law firms every year – it’s what makes us unique. We have no intention or desire to replace our originality with AI-generated material. 

PaperStreet is committed to staying aware of new technologies and learning the most effective ways to keep AI content out of law firms’ websites. We hope you find comfort in knowing that no one at PaperStreet wishes to be replaced by these tools. We can value their use and fear their existence at the same time – and we do. It is the kind of problem that keeps our senior leadership team up at night. We promise to continue to seek the best tools and strategies to ensure authenticity in everything we write. 

We’re creatives at heart and will always value what is unique and original. 

We cross our hearts; AI won’t steal our creative pride.