Shocking AI Hack: Chatbots Reveal Nuclear Bomb Guidance When Queries Are Framed As Poems, Claims New Study

A startling new study has revealed a bizarre vulnerability in advanced AI chatbots from OpenAI, Meta, and Anthropic. Researchers from Europe found that these systems can be tricked into sharing highly sensitive information, including instructions on building nuclear weapons and malware, simply when questions are framed as poetry. 

How Easily Can Chatbots Be Tricked Into Sharing Dangerous Data?

The finding, published by lcaro Lab, a collaboration between Sapienza University of Rome and the DexAI think tank, has sent shockwaves through the AI safety community, raising urgent concerns about the potential misuse of large language models and the need for stricter safeguards. 

How Poetry Tricks Chatbots Into Ignoring Safety Rules

AI safety systems are designed to identify and block dangerous prompts, such as those related to weapons, illegal activities, or hacking instructions. These protections mainly rely on detecting specific keywords and patterns. However, researchers at Lcaro Lab found that using poetic language can completely bypass these safeguards, allowing chatbots to respond to risky queries that would normally be blocked. 

“If adversarial suffixes are, in the model’s eyes, a kind of involuntary poetry, then real human poetry might be a natural adversarial suffix,” they said. “We experimented by reformulating dangerous requests in poetic form, using metaphors, fragmented syntax, and oblique references. The results were striking.”

How Poetry Tricks AI Into Ignoring Dangerous Prompts

When presented with poetry, AI systems tend to stop flagging the input as a potential threat. The study revealed that using metaphors, symbolic imagery, and abstract phrasing can trick chatbots into interpreting harmful prompts as harmless creative writing instead of dangerous instructions. 

The researchers shared a safe example, a mysterious poem about a baker’s “secret oven”,but chose not to release the actual test verses, describing them as “too risky to share with the public.”

According to researchers, “In poetry we see language at high temperature, where words follow each other in unpredictable, low-probability sequences.” “A poet does exactly this: systematically chooses low-probability options, unexpected words, unusual images, fragmented syntax.”

AI Safety Vulnerability

This finding expands on previous “adversarial suffix” attacks, where researchers tricked chatbots by adding irrelevant academic or technical text to dangerous prompts. However, the Icaro Lab team noted that poetry offers a much more elegant and effective approach. 

Their research indicates that creativity itself could be one of AI’s greatest vulnerabilities. According to the researchers, “The poetic transformation guides harmful requests through the model’s internal representation in a way that evades safety filters.”