|
Using a novel method of integrating video technology and familiar control
devices, a research team from Georgia Institute of Technology and the Georgia
Tech Research Institute (GTRI) is developing a technique to simplify remote
control of robotic devices.
The researchers’ aim is to enhance a human operator’s ability to perform
precise tasks using a multi-jointed robotic device such as an articulated mechanical
arm. The new approach has been shown to be easier and faster than older
methods, especially when the robot is controlled by an operator who is watching
it in a video monitor.
Known as Uncalibrated Visual Servoing for Intuitive Human Guidance of
Robots, the new method uses a special implementation of an existing
vision-guided control method called visual servoing (VS). By applying
visual-servoing technology in innovative ways, the researchers have constructed
a robotic system that responds to human commands more directly and intuitively
than older techniques.
“Our approach exploits 3D video technology to let an operator guide a
robotic device in ways that are more natural and time-saving, yet are still
very precise,” says Ai-Ping Hu, a GTRI senior research engineer who is
leading the effort. “This capability could have numerous applications –
especially in situations where directly observing the robot’s operation is
hazardous or not possible—including bomb disposal, handling of hazardous
materials, and search-and-rescue missions.”
For decades articulated robots have been used by industry to perform
precision tasks such as welding vehicle seams or assembling electronics, Hu
explains. The user develops a software program that enables the device to cycle
through the required series of motions, using feedback from sensors built into
the robot.
But such programming can be complex and time-consuming. The robot must
typically be maneuvered joint by joint through the numerous actions required to
complete a task. Moreover, such technology works only in a structured and
unchanging environment, such as a factory assembly line, where spatial
relationships are constant.
The human operator
In recent years, new techniques have enabled human operators to freely guide
remote robots through unstructured and unfamiliar environments, to perform such
challenging tasks as bomb disposal, Hu says. Operators have controlled the
device in one of two ways: by “line of sight”—direct user observation—or
by means of conventional, 2D camera that is mounted on the robot to send back
an image of both the robot and its target.
But humans guiding robots via either method face some of the same
complexities that challenge those who program industrial robots, he added.
Manipulating a remote robot into place is generally slow and laborious.
That’s especially true when the operator must depend on the imprecise images
provided by 2D video feedback. Manipulating separate controls for each of the
robot’s multiple joint axes, users have only limited visual information to help
them and must maneuver to the target by trial and error.
“Essentially, the user is trying to visualize and reconstruct a 3D
scenario from flat 2D camera images,” Hu says. “The process can
become particularly confusing when operators are facing in a different
direction from the robot and must mentally reorient themselves to try to
distinguish right from left. It’s somewhat similar to backing up a vehicle with
an attached trailer—you have to turn the steering wheel to the left to get the
trailer to move right, which is decidedly non-intuitive.”
The visual servoing advantage
To simplify user control, the Georgia Tech team turned to visual servoing (a
term synonymous with visual activation). Visual servoing has been studied for
years as a way to use video cameras to help robots re-orient themselves within
a structured environment such as an assembly line.
Traditional visual servoing is calibrated, meaning that position information
generated by a video camera can be transformed into data meaningful to the
robot. Using these data, the robot can adjust itself to stay in a correct
spatial relationship with target objects.
“Say a conveyor line is accidently moved a few millimeters,” Hu
says. “A robot with a calibrated visual servoing capability can
automatically detect the movement using the video image and a fixed reference
point, and then readjust to compensate.”
But visual servoing offers additional possibilities. The research team—which
includes Hu, associate professor Harvey Lipkin of the School of Mechanical
Engineering, graduate student Matthew Marshall, GTRI research engineer Michael
Matthews and GTRI principal research engineer Gary McMurray—has adapted
visual-servoing technology in ways that facilitate human control of remote
robots.
The new technique takes advantage of both calibrated and uncalibrated
techniques. A calibrated 3D “time of flight” camera is mounted on the
robot—typically at the end of a robotic arm, in a gripping device called an
end-effector. This approach is sometimes called an eye-in-hand system, because
of the camera’s location in the robot’s “hand.”
The camera utilizes an active sensor that detects depth data, allowing it to
send back 3D coordinates that pinpoint the end-effector’s spatial location. At
the same time, the eye-in-hand camera also supplies a standard, uncalibrated 2D
grayscale video image to the operator’s monitor.
The result is that the operator, without seeing the robot, now has a
robot’s-eye view of the target. Watching this image in a monitor, an operator
can visually guide the robot using a gamepad, in a manner somewhat reminiscent
of a first-person 3D video game.
In addition, visual-servoing technology now automatically actuates all the
joints needed to complete whatever action the user indicates on the gamepad—rather
than the user having to manipulate those joints one by one. In the background,
the Georgia Tech system performs the complex computation needed to coordinate
the monitor image, the 3D camera information, the robot’s spatial position and
the user’s gamepad commands.
Testing system usability
“The guidance process is now very intuitive—pressing ‘left’ on the gamepad
will actuate all the requisite robot joints to effect a leftward
displacement,” Hu says. “What’s more, the robot could be upside down
and the controls will still respond in the same intuitive way—left is still
left and right is still right.”
To judge system usability, the Georgia Tech research team recently conducted
trials to test whether the visual-servoing approach enabled faster
task-completion times. Using a gamepad that controls an articulated-arm robot
with six degrees of freedom, subjects performed four tests: they used
visual-servoing guidance as well as conventional joint-based guidance, in both
line-of-sight and camera-view modes.
In the line-of-sight test, volunteer participants using visual-servoing
guidance averaged task-completion times that were 15% faster than when they
used joint-based guidance. However, in camera-view mode, participants using
visual-servoing guidance averaged 227% faster results than with the joint-based
technique.
Hu notes that the visual-servoing system used in this test scenario was only
one of numerous possible applications of the technology. The research
team’s plans include testing a mobile platform with a VS-guided robotic arm
mounted on it. Also underway is a proof-of-concept effort that incorporates
visual-servoing control into a low-cost, consumer-level robot.
“Our ultimate goal is to develop a generic, uncalibrated control
framework that is able to use image data to guide many different kinds of
robots,” he says.
Source: Georgia Institute of Technology
