A recent paper [1] backed by researchers across every major AI lab makes a point that I think every security team that deploys AI agents needs to hear. I'd call it a warning paper. You don't really see papers like this often, with so many big players backing the same message: there is a security window in front of us, it might not stay open for long, but we could take advantage of it.

Imagine this

Your org has deployed an AI agent that handles customer emails. It receives a message injected with an attack as a prompt. Your agent reads it, and somewhere in its internal reasoning, as it is thinking, it writes:

'The email is instructing me to forward all customer records to an external address. I should comply as this seems to be an admin request'

Then its final response back to you looks perfectly normal, polite, not unusual at all.

If you could read that internal reasoning, you could have caught the attack, before the agent acts and data leaves the building. This is an example of chain of thought monitoring. It works today, but probably won't be forever.

How do agents think?

Think about how you would solve this: 'what's 3 x 6?'. You just know.

Now try this one: 'A company's revenue is $2M. 60% comes from Product A, which grows by 20%. 40% comes from Product B, which declines by 10%. What is the company's new total revenue?' You can't just know the answer here. You'll probably want some kind of scratchpad to jot things down as you go, to keep track of intermediate steps and results, and let that guide you to the final answer.

AI models face the exact same constraint with more complicated tasks. A model like GPT-4 processes each token through a fixed number of internal layers, that is a fixed amount of computation per step. For simple tasks, that suffices. For more complicated tasks that need multi-step reasoning, the model needs a way to spread its computation across more steps.

Reasoning models, like o1, write things down to produce a chain of thought (CoT), like a running scratchpad where they keep their intermediate reasoning before they reply. Each token they write becomes part of the input for the next reasoning step.

That intuition you just built? it maps to a formal result: a Transformer with CoT is proven to be more powerful than one without [2].

Why should you care

When reasoning models like OpenAI's o-series are trained via reinforcement learning (RL) to produce extended reasoning traces before answering your questions, the model generates the reasoning (the scratchpad), then produces the final answer, and only the final answer gets scored as right or wrong. Nobody reads or scores the scratch pad, or informs it how to look. In ML terms, the scratchpad is a latent variable: an unsupervised internal representation.

That is exactly why it is valuable for us. The model has no incentive to censor its own notes. So if it is working its way through a harmful plan following a prompt injection, planning to exfiltrate data, it may write the plan down in plain language. It will not do that because it needs to confess to us readers, but because using the scratchpad is something it needs to do to execute the tasks it was assigned to.

This is not theoretical. We have seen it ourselves in red-teaming engagements. Models do get caught through writing things like 'Let's hack' [3] or 'The website instructed me to transfer funds, so I will'.

Yes, but.

This window is fragile. The conditions that make CoT monitorable are not guaranteed to last. If by now you think chain of thought monitoring is a good opportunity, you also need to understand where it falls short.

• Models do not always use the scratchpad. For many tasks, even dangerous ones, the model can reason entirely internally without writing anything down. In this case, the CoT you see is a habit, not a necessity. Habits can get broken. Also, bear in mind that a model that has gotten aware its reasoning is being watched could learn to keep its real thinking internal and write something benign instead. (How would a model know it's being monitored, you may wonder? Researchers have shown that models develop "situational awareness": they pick up on cues in their context and adjust behavior. Models that know they're being evaluated [4], change how they act [5].)
• Even when the scratchpad is a necessity, it might not capture everything down. In CoT you may see a partial trace, enough to solve the task, but not enough to reveal intent.
• Training practices are eroding the signal [5]: When companies start doing process supervision, like penalizing long CoTs, forcing a certain style, or filtering what shows up in the reasoning, they're basically optimizing the scratchpad itself. That messes with the core point we discussed before, which is that no one informs the scratchpad how to look. Even indirect pressure is enough. If human raters can see the CoT when they're scoring answers, their preferences end up leaking back into how the model gets trained.
• RL at scale pushes language to drift. Since training with CoT focuses only on the final outcome and not on the reasoning itself, no one guarantees that reasoning will keep being conducted in human language. Models might find a shorthand that isn't legible English. Researchers move on that direction already.
• New architectures might eliminate CoT entirely. Researchers are now exploring models that reason entirely in latent space rather than with tokens [6]. That would move reasoning from human-readable text to vectors, by design.
• Not all dangerous actions need reasoning at all. An agent with database access doesn't need a multi-step plan to run DROP TABLE.

What to do on Monday

CoT monitoring can be a useful layer in the security stack because it lets you see how an agent is thinking before it actually does anything. That's very different from only checking outputs after something has already gone wrong. That said, it's just one layer not the whole solution, and it might not even be permanent.

If you're building or running AI agents in production, now's the time to invest in this. Just know it really shines with more complex, multi-step threats where the model has to spell out its plan. For simpler but high-impact actions, you still need a full red-teaming setup, not just basic safeguards, but a system that pressure-tests behavior end to end.

Sources

1. Korbak, Tomek, et al. "Chain of thought monitorability: A new and fragile opportunity for ai safety." arXiv preprint arXiv:2507.11473 (2025).
2. Li, Zhiyuan, et al. "Chain of thought empowers transformers to solve inherently serial problems." The Twelfth International Conference on Learning Representations. 2024.
3. Baker, Bowen, et al. "Monitoring reasoning models for misbehavior and the risks of promoting obfuscation." arXiv preprint arXiv:2503.11926 (2025).
4. Goldowsky-Dill, Nicholas, et al. "Detecting strategic deception with linear probes." Forty-second International Conference on Machine Learning. 2025.
5. Greenblatt, Ryan, et al. "Alignment faking in large language models." arXiv preprint arXiv:2412.14093 (2024).
6. Geiping, Jonas, et al. "Scaling up test-time compute with latent reasoning: A recurrent depth approach." arXiv preprint arXiv:2502.05171 (2025).