This “robot’s eye view” shows how some common household objects appear through the vision system being used in the Perception Challenge. The objects are fuzzy because the cameras have limited resolution. However, the images do provide information on depth (distance of every point on an object). The checkered patterns help to define and verify objects in space. Credit: Courtesy Willow Garage
The National Institute of Standards and Technology (NIST) is
teaming up with Willow Garage, a Silicon Valley robotics research and design
firm, to launch an international “perception challenge” to drive improvements
in sensing and perception technologies for next-generation robots.
“Perception is the key bottleneck to robotics. This
competition will progressively advance solutions to perception problems,
enabling ever wider applications for next-generation adaptive, sensing robots”
says Willow Garage senior scientist Gary Bradski.
The competition will debut at the IEEE International
Conference on Robotics and Automation (ICRA) 2011. It will join three other
competitions: updated versions of two other robotics competitions previously
developed by NIST—the Virtual Manufacturing Challenge and the Micro-Robot
Challenge—and the Modular and Reconfigurable Robot Challenge, a collaborative
effort by the National Aeronautics and Space Administration (NASA) and the
Univ. of Pennsylvania.
The new competition will measure the performance of current
algorithms that process and act on data gathered with cameras and other types
of sensing devices, explains NIST computer scientist Tsai Hong. “There are
hundreds—maybe even thousands—of algorithms that already have been devised to
help robots identify objects and determine their location and orientation,” she
says. “But we have no means for comparing and evaluating these perceptual tools
and determining whether an existing algorithm will be useful for new types of
Willow Garage is putting up cash awards for excellent
performers. The prize money grows exponentially with performance reflecting the
increasing difficulty of each new increment in capability. The top prize is up
to $7,000, awarded for successful completion of all tasks within the allotted
All contestants will receive a common set of about 35
objects for training and tweaking their algorithms. During the competition,
teams will be evaluated on how well their solutions identify and determine the
positions of these 35 objects plus an additional set of 15 objects for
validation. NIST also will inform contestants of the metrics and methods they
are developing for the competition.
Robust perception is a core enabling technology for
next-generation robotics being pursued for a variety of applications. Many of
these applications will require operating in unstructured and cluttered
environments. For anticipated uses ranging from advanced manufacturing to
in-home assistance for the elderly, to search-and-rescue operations at disaster
sites, robots must be able to identify objects reliably and determine their
The practical goals of this and future perception challenges
are to determine what solutions already exist for particular robot-performed
jobs, and to push the entire field to develop more dynamic and more powerful
perception systems critical for next-generation robotics. NIST has designed a
variety of competitions intended to focus research and stimulate innovation in
technology areas critical to improving the capabilities of robots.*
Techniques and metrics demonstrated in these competitions
provide foundations for new standards and test methods for measuring perception
system performance. As is true for the other competitions, the perception
challenge will grow in difficulty with each passing year.
Willow Garage of Menlo Park, Calif., will provide a common
system for testing competitors’ perception algorithms. Visual information and
other environmental data will be gathered and communicated by off-the-shelf
sensing technologies, and will be evaluated on Willow Garage’s Personal Robot 2
The deadline for entering is April 15, 2011, and final
submissions are due May 1, 2011.
For more information on the perception challenge and
instructions for entering, go to: http:/opencv.willowgarage.com/wiki/SolutionsInPerceptionChallenge.