AI Fakes Understanding Puns, New Research Shows
AI chatbots like ChatGPT and Gemini can spit out jokes, but they don’t grasp why puns work. A study from researchers at Cardiff University and Ca’ Foscari University of Venice tested this head-on. Their paper, Pun Unintended: LLMs and the Illusion of Humor Understanding, presented at the 2025 Conference on Empirical Methods in Natural Language Processing, proves large language models (LLMs) rely on memorized patterns, not real comprehension.
How the Researchers Tested AI
The team refined old datasets and built new ones called PunnyPattern and PunBreak. They fed LLMs real puns and tweaked versions where the wordplay vanished, but the structure stayed the same.
Examples they used:
- “Long fairy tales have a tendency to dragon” (pun on “drag on”) became “Long fairy tales have a tendency to wyvern” or “prolong.” AI still called the nonsense versions puns. TechXplore reports accuracy dropped sharply on unfamiliar puns, hitting as low as 20%, worse than random 50% guessing.
- “I used to be a comedian, but my life became a joke” switched to “but my life became chaotic.” Both got flagged as puns. News18 covers this test.
- “Old LLMs never die, they just lose their attention” (playing on AI tech term) changed to “ukulele.” AI spotted a fake phonetic link anyway. India Today details it.
Professor Jose Camacho-Collados from Cardiff called it an illusion: LLMs insist sentences are funny if they mimic pun shapes, even without double meanings or sense.
What This Means for AI
LLMs overconfident on trained puns falter on new ones. They lack phonetics grasp and cultural context needed for true humor. Mohammad Taher Pilehvar noted outputs need a “pinch of salt” for creative tasks like empathy or nuance.
Yahoo News Canada warns writers and marketers: AI wit feels hollow, risks robotic or confusing content.
Boing Boing says AI just repeats training data jokes. Axios calls AI’s sense of humor complicated.
Humans hold the edge on comedy. The team wants to test broader creativity and build self-aware AI that admits limits.
