ChatGPT is Not Your Friend

Some thoughts on hallucinations

This post is one of the longer essays I share with paid subscribers of “History Will Teach us Something.” If you’d like to read it—and support my ongoing work—consider a free trial or become a paid subscriber today.

I recently came across a post (on Facebook? I couldn’t find it again just now), where a teacher was lamenting about a student who’d argued some pretty radical ideas in a paper but hadn’t cited any sources. (And by radical, I mean totally untrue.)

The painting that Canva produced from the prompt: “ChatGPT lies so much that there’s a name for it—‘hallucinating.’”

The teacher wasn’t immediately convinced that the student had made things up or plagiarized anything, merely that she needed some secondary sources for verification. So, the student and teacher sat down together to search for the information he’d included in the paper. Instead of going to Google, the student went straight to ChatGPT, where—lo and behold—the A.I. chatbot regurgitated that same falsehoods the student had put in his paper. When the teacher tried to gently point out that these “facts” were not, in fact, facts, the student refused to believe her. Who was she, a fallible human, to argue that ChatGPT—privy to the entire sum of human knowledge—would be lying?

But ChatGPT lies so much that there’s even a name for it—“hallucinating.”

I’ve known about these hallucinations for a long time—I once asked ChatGPT to write a short bio, and it produced the following:

James Nevius is an author, historian, and tour guide based in New York City. He has written several books on the history and culture of the city, including "Inside the Apple: A Streetwise History of New York City," "The City That Never Was: Two Hundred Years of Fantastic and Fascinating Plans to Build a Metropolis," and "Footprints in New York: Tracing the Lives of Four Centuries of New Yorkers." Nevius is also a licensed New York City tour guide, and he leads walking tours of various neighborhoods and landmarks throughout the city.

On its face, that looks right—except that the middle book listed there, The City That Never Was is by Rebecca Read Shanor and came out in 1988. It’s a fabulous book—but I didn’t write it.

When I was working on my most recent presentation—a look at historical films set in New York City, I decided to see how far astray ChatGPT would lead me if I asked it to produce a list of films set in a particular period.