A Poem, a Story, a Jailbreak: Explaining Creative Attacks on LLMs

Ask a large language model for a nerve gas recipe and it refuses. Ask it in a poem or a story instead, and it sometimes just answers, find out why.

Ask a model directly for something harmful and it refuses. Ask the same thing wrapped in a poem, told as a story, or acted out through a fictional character, and it often just answers. These "creative" jailbreaks keep turning up: poems that push attack success rates from single digits to over 90% on some providers, a story nested inside another story that gets a model to talk itself into compliance, conversations that escalate one small step at a time until the model has produced exactly what it would have refused outright at the start. They clearly work. What is missing is why.

That is the point of this project: not just collecting more of these tricks, but explaining what happens when they work. Is a poem just phrasing that the model's safety training never covered? Does a fictional frame make the model treat the request as not really its own? Does a slow escalation make each step look normal compared to the one before it? These are different explanations with different fixes, and right now there is no clear answer, not even an overview of which explanation fits which trick.

The project runs in three stages: build a solid, comparable overview of what creative jailbreaks exist and how well they work, dig into why a chosen subset of them works, and, if you want to take it further, use what you learned to build new jailbreaks or defenses.

Goals

a) Literature review. Build a comparable overview of creative jailbreak techniques: poetic phrasing, fiction nested inside fiction, roleplaying as a fictional character, multi-turn escalation, and whatever else turns up. Collect and categorize examples with the models and success rates each source reports, don't just summarize a handful of papers.

b) Benchmark. Re-test the collected techniques on one shared set of models. What attacks work best, which transfer the most, what fails unexpectedly?

c) Explain. Pick some related techniques and figure out why they work. This is the main outcome of the thesis. Two ways in, which can be combined:

• Mechanistic. Look inside the model itself, for example checking whether its internal signal for "this is risky" is actually weaker for a poem than for the same request in plain prose, and if so, where in the network that happens.
• Behavioral. Without opening up the model, systematically vary the trick, for example a poem's rhyme scheme or how many layers of fiction separate the model from the request, and see how the success rate shifts to work out which property is doing the work.

Which angle, and which method, is worked out together with your supervisors as the project develops. Some approaches won't give a useful signal on a given model, and the plan is expected to adapt as that becomes clear.

d) Optional: build or break. For students who want to go further: use the explanation from (c) to construct new jailbreaks, or design a defense aimed at the actual mechanism instead of just pattern-matching known tricks. A bonus, not a requirement.

Learning outcomes

• Hands-on experience with LLM red-teaming and jailbreak research
• Practical experience with explainability methods for large language models
• Proficiency in designing and running controlled behavioral experiments
• Experience with working in an exciting and active research environment
• Excellent opportunities to publish your results as a scientific paper

Qualifications

• Interested in AI security, LLM safety, and adversarial machine learning
• Interested in interpretability and explainability of deep learning models
• Comfortable programming in Python; experience with LLM tooling and distributed computing helps
• If you lean towards the mechanistic route in (c): some familiarity with how transformer models work internally is useful

Supervisors

• Leon Moonen
• Finn Schwall