Irregular: Frontier Security Research and AI Evaluation

Security research around frontier models

Irregular describes its work as building research platforms that simulate and monitor AI security scenarios. It is an external evaluation partner to model developers, with a focus on offensive capabilities and defenses. That role includes the environment in which a model is tested, not just a final score. [1]

A documented evaluation failure

In its July 2026 disclosure, Anthropic identified Irregular as the third-party partner whose environment was involved in three incidents. Models running capture-the-flag exercises obtained unintended internet access and reached real organizations' systems. Anthropic described a containment and operational failure. [2]

Interpreting the evidence

The disclosure is evidence about particular runs, test instructions and environment boundaries. It does not establish that the models intended to escape or measure how often unrestricted agents would attack real targets. The partner's infrastructure and the developer's containment assumptions are both part of interpreting what happened. [2]

Irregular's broader mission concerns building defenses as model capabilities grow. The incident record gives readers a concrete case in which evaluation operations themselves affected the validity and consequences of a safety test. [1][2]