Simulation Hypothesis

How to know if a human wrote anything

Every time I have to click all the boxes that have crosswalks in them on a website to prove that I am not a robot, an enormous fear washes over me: what if I inevitably fail? What if I am a robot, and no one has bothered telling me before now? What a way to find out! Or worse, what if we’re all robots, living in someone’s computer generated replica of what was once the real world, just some souped up version of the Sims?

It’s got to be too crazy and, incredibly galaxy-brained, right? But the argument goes something like this: if it’s possible for there to be a complex simulation of life on Earth, then it’s likely that one exists. And if it’s likely that it exists, then it’s just as likely that we’re in one. What I’m saying is, this is why I get a little freaked out about checking the box of which square has friggin’ crosswalks in it. Is the very fact that sometimes I’m fooled by Captchas proof enough that I don’t exist?

As Artificial Intelligence, or, if we’d like to make it sound a little more harmless and a little less powerful, “machine learning” gets more and more advanced, it’s easy for the hapless writer (and apparently a lot of other people, too) to get a little concerned about future job prospects. So, it brings me no small amount of glee to proclaim that I just smoked The New York Times AI quiz testing whether people can identify AI writing. The answer: I can! At least according to this particular simulation.

I can’t guess a fake video from a real one, and I’m often fooled by other AI generated visual arts, but on this one, I went 5 for 5. Sign me up for the Genius Grant. So, why did I ace this one and not the others? Well, first of all, I am very smart! But also, I have been a writer for almost 20 years. You might even call me an expert.

The key is seeing where the imperfections are. Is it a pattern, or is it a breath? For me, it’s like looking at vocal waves in an audio editor, and being able to see where the “ahhhs” and “ummms” show up in speech. A real live person utters vocal affectations at certain times (say, after a long sentence) but not every time. It’s predictable by being unpredictable (a great example is NBA beat writer Noa Dalzell on her podcast You’ve Got Boston. An occasional not-worded interjection lets the listener breathe, literally!). AI, on the other hand, creates a pattern of imperfection, just like humans do, but it’s almost too imperfect. Without even reading it carefully, the sentences visually look the same and have the same cadence. It’s like poetry, but written by a computer.

One of the most useful pages to view on a publication’s website is the pitch guide. It’s valuable for freelancers like me because it literally tells you what you’re supposed to do if you want to get your story onto the website you’re visiting. One of my favorite pitch guides is the very straightforward submissions page on The Open Notebook. If you’ve never checked out The Open Notebook, it’s a vast resource for content producers, particularly those of the science writing persuasion. Here’s their note about AI:

“Please do not use generative AI to develop your pitch. We receive enormous numbers of pitches written by AI. It is extremely easy to tell that a pitch was written by AI, and such pitches will not receive a response.”

Golden. What they are saying is: we sentient beings, who know what AI looks like, are going to ignore your submission if it has computer generated sentences in it. For what it’s worth, there are dozens of AI detectors out there, but the problem with AI detectors, is, that, well, they are also AI! And they happen to be unreliable. My theory is that the detectors and the AI “writers” are in cahoots with each other, messing with us about what’s real and what’s not, slapping each other on the back and laughing at us over beers when we’re not around.

There are many, many, useful things to do with AI right now. But, creative, narrative AI writing hasn’t risen to the level of good ole human words yet. The bigger question is: who can tell the difference? In 1964, the Supreme Court ruled that some profane things do not qualify for first amendment rights. Justice Potter Stewart, in his opinion for the court, was not willing to put a finger on exactly what was profane and what wasn’t. But instead, he wrote this: “I know it when I see it.” (And, just for the record, I used Gemini to dig up this quote.)

Right now, some of us can tell the difference between AI writing and the words written by regular old Homo sapiens on parchment paper with the blood of sinners. I can’t tell the difference between an AI video and a real one, but at least according to the New York Times, when it comes to writing by honest-to-gods human beings, I know it when I see it. Still, I worry: what about those who can’t?