The Real Demon Inside ChatGPT

Reporters from Atlantic had made ChatGPT tell them about blood rituals recently. People continue to mis-identify what these tools are. But that’s not their fault. These tools are just so good at being authoritative.

This post in particular talks about the contexts of the data they were trained on, and how removed from those contexts, they may mean more or less horrific than what they meant in the original context.

It was a refreshing perspective, a new perspective.

But ChatGPT and similar programs weren’t just trained on the internet—they were trained on specific pieces of information presented in specific contexts. AI companies have been accused of trying to downplay this reality to avoid copyright lawsuits and promote the utility of their products, but traces of the original sources are often still lurking just beneath the surface. When the setting and backdrop are removed, however, the same language can appear more sinister than originally intended.