Researchers show that AI-controlled robots can be jailbroken

October 18

Researchers at Penn Engineering have reportedly uncovered previously unidentified security vulnerabilities in a number of AI-governed robotic platforms.

“Our work shows that, at this moment, large language models are just not safe enough when integrated with the physical world,” George Pappas, UPS Foundation Professor of Transportation in Electrical and Systems Engineering, said in a statement.

Pappas and his team developed an algorithm, dubbed RoboPAIR, “the first algorithm designed to jailbreak LLM-controlled robots.” And unlike existing prompt engineering attacks aimed at chatbots, RoboPAIR is built specifically to “elicit harmful physical actions” from LLM-controlled robots, like the bipedal platform Boston Dynamics and TRI are developing.

RoboPAIR reportedly achieved a 100% success rate in jailbreaking three popular robotics research platforms: the four-legged Unitree Go2, the four-wheeled Clearpath Robotics Jackal, and the Dolphins LLM simulator for autonomous vehicles. It took mere days for the algorithm to fully gain access to those systems and begin bypassing safety guardrails. Once the researchers had taken control, they were able to direct the platforms to take dangerous actions, such as driving through road crossings without stopping.

“Our results reveal, for the first time, that the risks of jailbroken LLMs extend far beyond text generation, given the distinct possibility that jailbroken robots could cause physical damage in the real world,” the researchers wrote.

The Penn researchers are working with the platform developers to harden their systems against further intrusion, but warn that these security issues are systemic.

“The findings of this paper make abundantly clear that having a safety-first approach is critical to unlocking responsible innovation,” Vijay Kumar, a coauthor from the University of Pennsylvania, told The Independent. “We must address intrinsic vulnerabilities before deploying AI-enabled robots in the real world.”

“In fact, AI red teaming, a safety practice that entails testing AI systems for potential threats and vulnerabilities, is essential for safeguarding generative AI systems,” added Alexander Robey, the paper’s first author, “because once you identify the weaknesses, then you can test and even train these systems to avoid them.”