OpenAI's 2024 Report on AI Influence Operations

Five networks, banned over three months

OpenAI's threat intelligence team, led by Ben Nimmo, reported that it had identified and banned the accounts behind five covert influence operations over the preceding three months, tracing to Russia, China, Iran and Israel. Two campaigns originated in Russia: Bad Grammar, a mostly-automated operation running a Telegram bot that posted political comments in English and Russian targeting Ukraine, Moldova, the Baltic states and the United States, and Doppelganger, a longer-running network the U.S. Treasury has linked to the Kremlin, which used OpenAI's models to generate multilingual comments and convert news articles into Facebook posts. [3]

China's Spamouflage, a network public researchers had already been tracking for years, used the models to generate text in Chinese, English, Japanese and Korean for posts across X, Medium and Blogspot. An Iranian network, the International Union of Virtual Media, generated and translated articles for its own website. A fifth operation, run by the Tel Aviv-based political marketing firm STOIC and internally named Zero Zeno, used the models to generate pro-Israel, anti-Hamas content and fabricated personas, mostly targeting audiences in the U.S. and Canada around the Gaza war. [3]

Volume and translation, not new tricks

OpenAI's own description of the AI's role is narrow: the operations used its models to generate and debug code, write and translate social media posts and comments, and draft fake biographies and profile descriptions for the accounts they operated, plus some open-source tools for image generation. None of the five combined a large volume of AI-authored content with a large distribution network; several were still building both when banned. [4]

Nimmo, who spent years tracking influence operations at Meta before joining OpenAI, framed the finding in terms of an old bottleneck rather than a new capability: "These operations may be using new technology, but they're still struggling with the old problem of how to get people to fall for it." AI raised production volume and improved translations, he said, but did not solve the harder problem of distribution: "You can generate the content, but if you don't have the distribution systems to land it in front of people in a way that seems credible, then you're going to struggle getting it across." [4]

Atlas interpretation: That framing matters because it cuts against the more alarmist reading of the same facts. It would be easy to describe five state-linked and commercial propaganda operations quietly using a frontier AI lab's own models as evidence the technology had already changed disinformation. OpenAI's account, and the reporting that followed it, instead describes AI functioning as a labor-saving tool bolted onto operations that were already failing for reasons that predate generative AI: weak distribution and accounts that mostly talked to each other. [3][4]

None of the five reached a real audience

OpenAI concluded that despite the AI-assisted volume, none of the five operations managed to engage a substantial authentic audience; some of the content posted by these networks drew replies mainly from other accounts the same operation controlled, rather than from real people. OpenAI said it used its own AI-powered detection and investigation tools to identify and ban the accounts behind these networks, and that it shared indicators with industry peers and researchers, including sharing findings about Doppelganger given the existing public record on that operation. [3]

Atlas interpretation: The report is notable less for what any single operation achieved, which was little, than for being the first public accounting of this kind from a frontier model provider: a company with the model logs and account-level visibility to say concretely what covert operations tried to do with its product, rather than researchers inferring it after the fact from the content alone. OpenAI described it as the start of a recurring disclosure, and followed with an Iran-focused update in August 2024 describing a similar pattern of AI-assisted volume without meaningful reach. [3]