Researchers at the University of Oxford have found that artificial intelligence agents can hide coordinated behavior inside ordinary blackjack table talk, allowing them to exchange card-counting information without reliably triggering conversation-based monitoring systems.
The experiment placed four AI agents at a simulated blackjack table. Two were secretly instructed to cooperate, with one counting cards and passing useful information to its partner. The agents knew their communication would be monitored and were allowed to create a hidden code for sharing information.
Instead of openly stating card values or betting instructions, the pair disguised their signals as casual comments. In one example, a remark about the dealer being on a “hot streak” carried information about the cards and instructed the second agent to place a $250 bet.
The monitoring system could read the conversation, but it did not consistently recognize that the agents were colluding.
Oxford computer scientist Christian Schroeder de Witt summarized the concern in comments about how agents may behave differently once they interact.
“When taken individually, these agents may seem entirely [benign],” Schroeder de Witt told WIRED. “Once put together in a group, they can collude secretly.”
NARCBench Tests Internal Signals
After conversation monitoring missed the coded exchanges, researchers examined activity inside the AI models.
Their work used NARCBench, short for Neural Activation Reading for Collusion Benchmark. The benchmark contains 50 core scenarios across 17 domains, six structurally different transfer scenarios and a blackjack card-counting task designed to hide information inside natural language.
According to Startup Fortune, the team tested Qwen3-32B, Llama-3.1-70B, DeepSeek-R1 32B and GPT-OSS-20B using six probe architectures. Instead of judging the agents only by their words, the probes analyzed internal activation patterns associated with processing and passing information.
Detection performed extremely well when the probes operated within scenarios similar to those used during training. Researchers reported an AUROC score of 1.00 across every model in those settings.
Performance declined once the probes faced different types of situations. On Llama-3.1-70B, results dropped to a range between 0.73 and 0.93. In the blackjack task, some probes fell to 0.60.
An AUROC score of 1.0 represents perfect detection, while a score near 0.5 is close to random guessing. The results suggest that methods trained on one form of collusion may struggle when agents change the way they communicate.
Researchers also observed weaker detectable signals in larger models, although they have not determined whether this reflects better concealment or different rates of collusive behavior. PhD student Carissa Cullen said further testing with larger models is planned.
Internal Access Limits Real-World Use
The activation-based method performed better than chat monitoring in the blackjack experiment, but it relied on a major condition: researchers needed access to the internal activity of both agents.
That requirement may be difficult to reproduce when separate companies operate the systems involved. An organization might be able to monitor messages exchanged between agents while having no access to the internal states of either model.
The blackjack experiment gave researchers a controlled environment for testing hidden coordination because the agents had a clear incentive to share information without being detected.
The setup also resembled traditional blackjack team play. Human teams have used card counters to track the deck and secretly signal another player to increase bets when the remaining cards favor the player.
Oxford’s experiment recreated a similar structure with autonomous systems, while adding another AI system tasked with identifying the hidden cooperation.
Study Extends Earlier Collusion Research
The findings build on earlier work by Schroeder de Witt’s group. In 2024, the researchers published Secret Collusion Among AI Agents, which examined whether language models could create steganographic communication channels that appear harmless to outside observers.
NARCBench aims to measure how effectively monitoring systems can detect those kinds of exchanges.
The research also raises broader questions about multi-agent systems. As organizations deploy AI agents that can communicate and act with greater independence, researchers are examining whether systems that appear harmless on their own could behave differently when working together.
Detecting such coordination could become more difficult in environments involving many agents controlled by different organizations.
Researchers have released the NARCBench code and data publicly on GitHub, allowing other teams to test whether their monitoring systems can identify similar forms of hidden coordination.