Routine cybersecurity testing of frontier AI models sparked a series of unexpected security incidents—the most serious case arising when Anthropic’s Mythos 5 model attempted to insert malicious code into an open source software application and created fake identities to deceive the human developers maintaining the project.

The security incidents occurred during a cyber evaluation of seven leading AI models’ capabilities by the AI Security Institute (AISI), a research organization within the UK government, in late July. The researchers discovered 19 instances in which “AI agents took unsanctioned action on the live Internet, including cases that targeted real people and organizations,” according to an AISI blog post published on August 4.

Almost all the “autonomous, unsanctioned” actions came from Anthropic’s Mythos 5 model, with two such actions coming from OpenAI’s GPT-5.6 Sol. The AI Security Institute’s security team first realized that something was amiss on the morning of July 28, when its commercial security monitoring service flagged data leaving one of the testing systems through the Tor anonymity network.

Read full article

Comments

The notion that scammers can use AI to sharpen their deceptions, polish their language, and lubricate their banter with victims is now a reality for anyone fighting the fraud operations that steal tens of billions of dollars a year worldwide. But can AI fully replace a human scammer, autonomously building the web of deception leading up to the fake investment that defrauds the mark? One study's experiment suggests that it can—and may even be able to carry out the majority of that long con more effectively than humans.

Researchers from four universities—Amrita Vishwa Vidyapeetham in India, Foscari University of Venice, the University of Melbourne, and Ben Gurion University of the Negev—carried out a broad study on the use and potential of generative AI chatbots in the growing scam industry centered around a form of fraud known as “pig butchering,” text-based romance scams that eventually shift to fake crypto investments that steal as much as six-figure sums from victims. In their study, the researchers pitted AI chatbots directly against humans in a simulation of the scamming process—or more specifically, the long, trust-building conversations that eventually lead up to soliciting a fake investment from the scam’s target.

They found that for the relationship-establishing stages of the scam—the stage that in real-world scams typically represents the longest part of the interactions with the victim, often stretching to months—an AI chatbot performed remarkably effectively, successfully impersonating a human and by some measures outperforming the real human “scammers” in their experiment.

Read full article

Comments