OpenAI Bans Influence Operation Accounts Behind Articles With Fake Bylines
The company blocked accounts tied to two operations originating in Russia and Iran. In the Iranian case, OpenAI says roughly 100 articles written under fabricated bylines were published by real online outlets.
On October 8, OpenAI announced that it had identified two covert influence operations originating in Russia and Iran and banned the ChatGPT accounts involved. Both operations spread messaging about politics and conflict while concealing who was behind the organizations and who had actually written the material. According to OpenAI, ChatGPT was used to draft activity reports and to polish article drafts and emails.
How a fake research institute was used
The Russian operation, dubbed “Dark Clark,” targeted Latin America. OpenAI says the operators used ChatGPT to write activity reports and to create or edit some deceptive content. The company also says the operators appear to have run the Social Research Center, an outfit presented as a local research institute, fronted by a fictitious persona named “Mia Clark.”
Local staff who worked at the institute appear not to have known about the Russian involvement, OpenAI says, adding that it has evidence to that effect. By dressing the operation up as a local institution, the operators hid who was actually behind it.
The Iranian operation, “Bogus Bylines,” pitched English-language articles to online outlets under seven fake bylines. The operators used ChatGPT to refine the drafts and the pitch emails. They also used ChatGPT to generate batches of social media comments in English and Persian, as well as to draft internal reports.
100 articles: what was the real impact?
In connection with the Iranian operation, OpenAI says it confirmed that roughly 100 articles were published or republished across more than a dozen online outlets. Pieces pitched under fabricated bylines made their way through real media outlets and into circulation. On the impact scale the company uses, it rated this article-distribution activity as Category 4, and the confirmed commenting activity as Category 2.
The Russian operation was rated Category 5 on the same scale, the first Category 5 case among the operations OpenAI has disrupted since it began publishing these reports. That said, the company notes that some of the results the Russian operators claimed internally were exaggerated or could not be independently verified.
As for the Iranian operation’s comments, OpenAI says engagement was low within the scope it was able to verify. It added that the view-count methodology the operators themselves used does not demonstrate the comments’ actual effect. The company also said there is no evidence that OpenAI models were used to post the comments automatically.
What stands out in this case is that writing published under fabricated bylines made it into real online outlets. OpenAI says the misuse of AI extended beyond generating text to pitching articles to media outlets. Beyond the number of articles distributed and the categories assigned, how actual readers responded remains unclear.