Fabian Grob

For this PhD project, we want to use mechanistic interpretability to better understand why LLMs fail at safety and specific components of it. This can include hallucinations, jailbreaks, sycophancy, scheming, and reward hacking. One of the goals could be to stress test different behaviors and safety aspects under conditions where current detection and intervention methods fail or fall short, and then investigate the underlying causes with the help of mechanistic interpretability. This could then in turn enable us to build new methods that are more robust and generalize better than what we have today.

One option would be to work at the activation level, where recent work has shown it is possible to both detect and intervene on unsafe behavior, for example through harmfulness or deception probes, or through activation steering that directly changes model behavior. However, it is often unclear whether these methods actually target the causal mechanism behind the behaviors, or whether they just pick up on a downstream signal that is easier to read. We want to investigate this question and also extend it to long generations and agentic settings, where safety relevant behaviors typically emerge over multiple steps or longer traces than just a single response.

This connects to Maksym's work on evaluating and reducing risks from LLM agents and to Yonatan's work on causal and mechanistic methods for understanding model internals. We want to bring these together and develop new, scalable methods to explain why safety evaluations succeed or fail, as well as investigate if methods like probes or steering actually generalize.