Failing with ChatGPT-5
I was reading Karen Hao's Empire of AI today, when I was inspired to swipe over to my browser and open up ChatGPT for a quick test.
I hadn't looked at the news, as it's Sunday, and so was unaware that ChatGPT-5 had been rolled out and then rolled back in less than 24 hours.
"Oh!" I thought, when I saw the banner announcing the new version. "Interesting. Let's see how this goes."
In the past, I had attempted to use ChatGPT to summarize things. Mostly, my own books, to see if I could use it to identify "hooks" (juicy moments that will resonate with readers) that I may not be aware are "juicy."
I'm not asking ChatGPT to identify the hooks - I'm mostly curious to see what it considers important enough to put in a summary, to see if it's different from the way I would summarize the book. All I'm asking it to do is summarize the text. Which really doesn't seem that hard.
Every time I attempted to do this with earlier versions of ChatGPT, it failed spectacularly while hallucinating wildly. I wondered if ChatGPT-5 might do better (after all, that was specifically what the shiny banner announcing the update said it would do!)
Reader...it did not.
Below is a link to my full conversation with ChatGPT-5, where I asked it to summarize my book Salt. Note for clarity: I nicknamed my ChatGPT instance "Dot" after the AI in my book Dot Slash Magic.