The William S. Floyd Jr. Distinguished Chair in Engineering at UC Berkeley, Goldberg and his research team have continued to advance the field.
The group — which includes postdoctoral researchers Minho Hwang and Jeffrey Ichnowski and PhD student Brijen Thananjeyan — has demonstrated how a deep neural network plus 3D-printed depth-sensing markers can train a da Vinci surgical robot to automatically perform peg transfer slightly faster and more accurately than an experienced surgical resident.
They also published a paper in Science Robotics last November about new AI software that allows robots to learn how to rapidly plan smooth motions that increase speed and reduce wear in factories, warehouses…and operating rooms.
Their work encompasses so much more, too. (Here’s a list of published research.)
Goldberg recently consulted with his research team to provide some insights to Medical Design & Outsourcing and MassDevice about where the surgical robotics space could advance in coming years:
MDO: You’ve spoken before of an emerging generation of surgical robots. Tell us more.
Goldberg: Many of the early patents on surgical-assist robots are expiring, so there are several new companies entering the market. For example, Johnson and Johnson purchased Auris, and Medtronic purchased Mazor in the past 2 years. Many new companies that are emerging in Asia — including China and Korea — are also developing a new generation of surgical-assist robots at lower cost and working to introduce some supervised autonomy.
Rather than replacing human surgeons, an emerging new generation of robots will assist surgeons by performing tedious subtasks such as suturing and debridement to improve consistency, reduce fatigue and open the door to long-distance tele-surgery. Advances in AI can be applied to data collected from surgical systems such as Intuitive’s da Vinci to learn underlying control policies for subtasks including cutting, suturing, palpation, dissection, retraction and debridement.
MDO: How can artificial intelligence advance robot-assisted surgery? Give us an example.
Goldberg: We were recently able to automate peg transfer, a common training procedure for minimally invasive surgery, with 99.4% accuracy (357/360 trials) [on a da Vinci Research Kit] — even when the robot tools are switched. This task is challenging because it requires high accuracy, and the cables that drive surgical robots’ joints stretch during motion, which can significantly reduce their accuracy.[We’ve learned] to correct errors at key points in the task visually by using demonstrations from a human teleoperator.
We also present an alternate approach to this task in two papers, one published and one preprint. These approaches use fiducial markers and depth-sensing with deep learning to learn a model of how the robot moves as a function of its past motions, which is a complex function of its cabling properties. Using this learned model for control, we achieve 94-100% accuracy on bilateral and unilateral versions of the task, respectively. Using a trajectory optimization procedure, we are able to roughly match or outperform a human surgeon in terms of speed as well.
MDO: TransEnterix executives see AI as an edge when it comes to competing better with its Senhance system. What do you think of their Intelligent Surgical Unit?
Goldberg: Learning how to position and move the surgical camera without manual adjustment is a very interesting problem. It is tedious for the surgeon to do this herself, so it would be helpful if a robot could anticipate, for example, as a suture advances beyond the field of view and track the camera to move accordingly. This is also subtle because erroneous camera motions could be frustrating for surgeons.
MDO: Overall, what are you most excited about in the robot-assisted surgery space?
Goldberg: I’m hopeful that we can develop systems that can learn to perform specific subtasks such as suturing or debridement faster and more accurately than human surgeons, thus relieving human surgeons of tedium, allowing them to focus on more nuanced aspects of surgery and also to reduce time in the operating room.
This research field is growing rapidly: There are now 30 labs worldwide doing experiments with a research version of the Intuitive surgical-assistant robot, and new hardware and algorithms are being published every month.