Robot Surgery Expanding the Possibilities of Telemedicine

Mamoru Mitsuishi

IT and Telemedicine This may not be imaginary for long. A patient hospitalized in a remote area will be able to receive advanced heart surgery under the hands of experts at a central facility. Tele-micro-surgery robot systems, used in this way, are expected to help people in medically underserved regions. What are the challenges of developing robots for this goal? We talked with Dr. Mamoru Mitsuishi, Professor at Faculty of Engineering, the University of Tokyo, on his study of telemanipulated surgical robots.

Stitching a 1-mm Thick Blood Vessel Over a Distance of 700 km

In September 1997, a remote-controlled surgical robot was operated via a communications network over a distance of 700 km from Tokyo to Okayama, and succeeded in suturing artificial blood vessels with a diameter of 1 mm.

This was the demonstration of the tele-micro-surgery system developed by Professor Mitsuishi and colleagues. An overview of this experiment is schematically shown in Figure 1.

The system consisted of three parts: (1) the surgery site simulating a hospital's operating room with a patient (on this occasion, it was set up in International Conference Hall in Okayama); (2) the operation site, where the surgeon manipulated the machinery (a laboratory in the University of Tokyo); and (3) the communications system connecting the two sites via the Internet.

A microscope at the surgery site captured the views of the field of operation, and sent images to the operation site. Looking at these images shown on a liquid crystal display, the surgeon maneuvered the master manipulator. The slave manipulator at the surgery site, equipped with forceps and other instruments, moved in response to the surgeon's actions to stitch the blood vessel specimens. The communications line used in this experiment had a bandwidth of 1.5 megabits per second. The time lag in image transmission between Tokyo and Okayama was 10 ms, and the delay in control signals was 10 ms. It took 5 to 10 minutes to make each stitch.

More recently, in the experiment in March 2000, the team succeeded in suturing a rat carotid artery with a diameter of 0.3 mm.

Overcoming distance is only one of the promises of robot systems like this. Robots can also expand our abilities to work in microscopic worlds, such as gene manipulation and nanotechnology. Furthermore, they can work in the scenes of disasters and other "extreme environments," which are inaccessible to human beings because of danger, high temperature, vacuum, etc. These robot systems must have several capabilities. First, the robots must have hands to perform work and/or legs to move around. Second, they must have sensor functions to perceive the process of work and the situation in the surroundings. Third, they must be intelligent enough to make decisions based on the perceived information. Finally, they must have communication capabilities for reporting work-related information to human beings and for receiving directions and supports from human beings. Accordingly, the themes of robot study at Professor Mitsuishi's laboratory are centered around the information, communication, and intelligence functions of machines.

The Interface Between Human Senses and Machines

Starting from the study of machine tools and production systems for manufacturing and processing factories, Professor Mitsuishi has been engaged in the development of intelligent production systems for about 15 years. One of his remarkable achievements was the development of a telemanipulated machining system.

His "Telemanufacturing System with Reality Transmission Capability" (see Photo 2) is different from conventional computer-controlled machine tools, which only receives NC programs (numerical data sets describing instructions for the movement of tools) from a computer. The most important feature of his system is the open interface, which enables the operator to monitor the process of machining on-the-fly by means of various sensors.

The machining unit in this system is equipped with a multi-axis force sensor to detect forces generated during machining, as well as a TV camera and a microphone. While these sensors provide real-time monitoring of the process of machining, the operator at a remote site controls the system using either a 3-dimensional control stick, a joystick, or his fingers on the tabletop.

One of the problems inherent in telemanufacturing is the time lag of data transmission (see Table 1). To help overcome this problem, a mechanism was invented that calculates the result of machining from the movement of the joystick, and displays the predicted result before actual execution of work (see Photo 3). The system also delivers information of machining noise, because this information is useful for the judgment of the quality of work. Rather than simply transmitting the actual noise, the system predicts the forces generated during machining and converts them to sound, so that the operator can hear it before the work is done. Tactile sense information is also transmitted to the operator. The joystick has a built-in motor, and the operator can feel its vibration.

Technology for Safe and Accurate Surgery

As demonstrated by this system, the accuracy of telemanipulation is enhanced greatly when the operator is allowed to receive real-life sensory inputs, such as visual, auditory, force, tactile, and thermal senses.

The tele-micro-surgery system, which Dr. Mitsuishi has been working on for 7 or 8 years, also contains a number of mechanisms ensuring safe and accurate surgery.

In the system shown in Figure 1, the movement of the master manipulator is reduced by a scale factor of 20 before it is reflected in the movement of the slave manipulator (see Photo 5). (When the surgeon moves the master manipulator by 10 cm, for example, the slave manipulator moves by 5 mm.) This ensures that the surgeon can perform delicate operations with a good margin of safety.

The slave manipulator has a 6-axis force sensor, and the force information obtained here is fed back to the master manipulator. Using this force-reflection technique, the surgeon can feel the force exerted by the instruments, and also he can hear it after conversion to sound. Because the human auditory sense is more sensitive to the changes in frequency of sound than to the changes in amplitude, this feedback information is presented in the form of frequency changes. The microscope at the surgery site is controlled by a unique interactive mechanism. A small CCD camera at the operation site continuously detects the line of vision and face movements of the surgeon, and this information is used for controlling both the line of vision and the magnification power of the microscope. The liquid crystal display is also motion-controlled, so that the surgeon can always look at it from the right angle.

How does the surgeon know the 3-dimensional positions of objects, such as the blood vessels to be sutured, while looking at images on a 2-dimensional display? To solve this problem, a laser spot ("bright shadow") is projected near the tip of the slave manipulator. The use of this "shadow" greatly facilitates 3-dimensional spatial localization (see Photo 4 and Figure 3).

Telemanipulation systems suffer from another problem. The time lags involved in remote controlling can cause destabilization of the control system, potentially resulting in destruction of the machinery. To ensure the safety of the patient and to prevent damage to the equipment, a hardware fail-safe mechanism has been incorporated in the slave manipulator. When this mechanism detected a load exceeding a specified limit, it moves the tools away to safe positions and shuts down the entire system (see Photo 6).

Promises of Telemanipulation Technology

According to Professor Mitsuishi, telemanipulation technology opens a number of possibilities as follows (see Figure 4):

(1) Reducing the burdens on patients: Telemanipulation can play a vital role in minimally invasive surgery and better aseptic surgery, in which the patient is operated through small holes rather than large cuts in the body. Transmission of images and tactile information can be useful in home diagnosis of aged patients, pregnant women, and infants, minimizing the necessity of hospital visits.

(2) Reducing the burdens on physicians: Telemanipulation can reduce the physician's risk and workload in performing fine surgery. It also prevents infections such as HIV and hepatitis. Telemedicine provides a means for team practice among physicians working at different places. Working as a team improves the choice of treatment and also alleviates the solitude of physicians in remote areas.

(3) Correction of regional disparity in medical services: Patients living in underpopulated areas will be able to receive the same quality of consultation, treatment, and advice as urban inhabitants. (While traditional health-care legislation was based on face-to-face consultation between the physician and the patient, the Ministry of Health and Welfare in 1997 eased restrictions on medical practice mediated by information equipment.)

(4) Enhancement of emergency medical care: Telediagnosis and telesurgery will improve the possibility of providing appropriate initial treatment in emergency cases. A database system allowing prompt access to the medical history of patients from anywhere will be greatly effective, though this involves the problem of privacy protection.

(5) Development of high-speed surgery simulators and advanced medical training: Surgery simulators are interactive systems that simulate the condition of the patient's body undergoing surgery. Given sufficient speed and accuracy, such systems will provide an effective means to compensate the time-lag in telesurgery. They also facilitate pre-operative evaluation of the planned method of operation, as well as training of less experienced surgeons in advanced surgical procedures.

Then, how are we making progress toward practical use of telemanipulated medical care?

"We were the first to demonstrate telemanipulated suturing of blood vessels, but venture companies in the U.S. are one step ahead of us in developing surgical robots for practical use. Japanese development teams are also making various attempts."

"Telecommunication cost is hardly a problem, so long as the data are sent over ISDN or a broad-band network. Rather, a major problem lies in the point system of medical insurance. Under the current system, medical fees for a surgical operation are given to the hospital where the operation was done. The system needs to be revised so that fees are also given to the hospital that teleguided the operation.

"Styling design of machinery is also important. A machine that looks too mechanical will not be popular in practice. It should be designed to convey a sense of safety both to the physicians and to the patients."

The largest problem, however, is the security of private information. "Needless to say, network security is important. In addition, letters on the display can be spied from a distance of 50 m. These problems need careful consideration."

Among various applications, ultrasound (echo) machines seem to be nearest to the goal of practical use. A system connecting the University Hospital and affiliated clinics will become operational this fall, allowing physicians to communicate bidirectionally for better diagnosis of diseases.