The crystals on the surface of an emery board (top) have light-reflective properties that make their structure difficult to determine. GelSight effectively coats the crystals with a layer of metallic paint, clarifying their structure (bottom). Images: Micah Kimo Johnson |
By
combining a clever physical interface with computer-vision algorithms,
researchers in Massachusetts Institute of Technology’s (MIT’s) Department of
Brain and Cognitive Sciences have created a simple, portable imaging system
that can achieve resolutions previously possible only with large and expensive
lab equipment. The device could provide manufacturers with a way to inspect
products too large to fit under a microscope and could also have applications
in medicine, forensics, and biometrics.
The
heart of the system, dubbed GelSight, is a slab of transparent, synthetic
rubber, one of whose sides is coated with a paint containing tiny flecks of
metal. When pressed against the surface of an object, the paint-coated side of
the slab deforms. Cameras mounted on the other side of the slab photograph the
results, and computer-vision algorithms analyze the images.
In
a 2009 paper, Edward Adelson, the John and Dorothy Wilson Professor of Vision
Science and a member of the Computer Science and Artificial Intelligence
Laboratory, and Micah Kimo Johnson, who was a postdoc in Adelson’s lab at the
time, reported on an earlier version of GelSight, which was sensitive enough to
detect the raised ink patterns on a $20 bill. At this year’s (2011) Siggraph,
Adelson and Johnson, along with graduate student Alvin Raj and postdoc
Forrester Cole, are presenting a new, higher-resolution version of GelSight
that can register physical features less than a micrometer in depth and about
two micrometers across.
Moreover,
because GelSight makes multiple measurements of the rubber’s deformation, with
light coming in at several different angles, it can produce 3D models of an
object, which can be manipulated on a computer screen.
Traditionally,
generating micrometer-scale images has required a large, expensive piece of
equipment such as a confocal microscope or a white-light interferometer, which
might take minutes or even hours to produce a 3D image. Often, such a device
has to be mounted on a vibration isolation table, which might consist of a
granite slab held steady by shock absorbers. But Adelson and Johnson have built
a prototype sensor, about the size of a soda can, which an operator can move
over the surface of an object with one hand, and which produces 3D images
almost instantly.
Adelson
and Johnson are already in discussion with one major aerospace company and
several manufacturers of industrial equipment, all of whom are interested in
using GelSight to check the integrity of their products. The technology has
also drawn the interest of experts in criminal forensics, who think that it could
provide a cheap, efficient way to identify the impressions that particular guns
leave on the casings of spent shells. There could also be applications in
dermatology—distinguishing moles from cancerous growths—and even biometrics.
The resolution provided by GelSight is much higher than is required to
distinguish fingerprints, but “the fingerprinting people keep wanting to talk
to us,” Adelson says, laughing.
Although
GelSight’s design is simple, it addresses a fundamental difficulty in 3D
sensing. Johnson illustrates the problem with a magnified photograph of an
emery board, whose surface, in close-up, looks a lot like marmalade.
“The
optical property of the material is making it very complicated to see the
surface structure,” Johnson says. “The light is interacting with the material.
It’s going through it, because the crystals are transparent, but it’s also
reflecting off of it.”
When
a surface is pressed into the GelSight gel, however, the metallic paint
conforms to its shape. All of a sudden, the optical properties of the surface
become perfectly uniform. “Now, the surface structure is more readily visible,
but it’s also measurable using some fairly standard computer-vision techniques,”
Johnson explains.
GelSight
grew out of a project to create tactile sensors for robots, giving them a sense
of touch. But Adelson and Johnson quickly realized that their system provided
much higher resolution than tactile sensing required.
Once
they recognized how promising GelSight was, they decided to see how far they
could push the resolution. The first order of business was to shrink the flecks
of metal in the paint. “We need the pigments to be smaller than the features we
want to measure,” Johnson explains. But the different reflective properties of
the new pigments required the use of a different lighting scheme, and that in
turn required a redesign of the computer-vision algorithm that measures surface
features.
