Robotic AI performs successful surgery after watching videos for training

Thursday at 03:03 AM

Watching old episodes of ER won’t make you a doctor, but watching videos may be all the training a robotic surgeon’s AI brain needs to sew you up after a procedure. Researchers at Johns Hopkins University and Stanford University have published a new paper showing off a surgical robot as capable as a human in carrying out some procedures after simply watching humans do so.

The research team tested their idea with the popular da Vinci Surgical System, which is often used for non-invasive surgery. Programming robots usually requires manually inputting every movement that you want them to make. The researchers bypassed this using imitation learning, a technique that implanted human-level surgical skills in the robots by letting them observe how humans do it.

The researchers put together hundreds of videos recorded from wrist-mounted cameras demonstrating how human doctors do three particular tasks: needle manipulation, tissue lifting, and suturing. The researchers essentially used the kind of training ChatGPT and other AI models use, but instead of text, the model absorbed information about the way human hands and the tools they are holding move. This kinematic data essentially turns movement into math the model can apply to carry out the procedures upon request. After watching the videos, the AI model could use the da Vinci platform to mimic the same techniques. It’s not too dissimilar from how Google is experimenting with teaching AI-powered robots to navigate spaces and complete tasks by showing them videos.

Surgical Robot Transformer Demo – YouTube
data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///ywAAAAAAQABAAACAUwAOw==

Watch On

Surgical AI

“It’s really magical to have this model and all we do is feed it camera input and it can predict the robotic movements needed for surgery. We believe this marks a significant step forward toward a new frontier in medical robotics,” senior author and JHU assistant professor Axel Krieger said in a release. “The model is so good learning things we haven’t taught it. Like if it drops the needle, it will automatically pick it up and continue. This isn’t something I taught it do.”

The idea of an AI-controlled robot holding blades and needles around your body might sound scary, but the precision of machines can make them better in some cases than human doctors. Robotic surgery is increasingly common in some instances. A robot performing complex procedures independently might actually be safer, with fewer medical errors. Human doctors could have more time and energy to focus on unexpected complications and the more difficult parts of a surgery that machines aren’t up to handling yet.

The researchers have plans to test using the same techniques to teach an AI how to do a complete surgery. They’re not alone in pursuing the idea of AI-assisted robotic healthcare. Earlier this year, AI dental technology developer Perceptive showcased the success of an AI-guided robot performing a dental procedure on a human without supervision.