The system uses the 2D barcodes attached to various parts of the malfunctioning machine to determine the camera’s position and direction of view and then sends this data to the experts. Photo: Fraunhofer FKIE |
Machines
stretch from one end of the production hall to another, each of them an
important part of the manufacturing process. When one of these complex
pieces of machinery stops working, the on-site technicians grab their
tools and the manual and try to fix it—but sometimes the only solution
is to call the manufacturer for tips on how to get it working again.
The
problem is that giving advice over the telephone is never easy: Do they
mean the screw on the right or the screw on the left? Well, that
depends on which side of the machine you are standing on. Even putting
things down in writing can only get you so far. Often the only choice
left is to fly experts in, sometimes from the other side of the world.
But until they arrive, the machine stays stubbornly offline, perhaps
bringing the entire production process to a halt.
Soon
experts will have an easier and quicker method of supporting the
on-site technicians without having to set foot outside their office: the
augmented reality system developed by researchers from the Fraunhofer
Institute for Communication, Information Processing, and Ergonomics FKIE
in Wachtberg.
This
system allows technicians to record the malfunctioning machine with a
camera fixed to the back of their laptop monitor. The computer is
mounted on a swivel arm so that the technicians can view the screen
while carrying out repairs. An image processing program calculates the
camera’s position and direction of view and sends this information to
the manufacturer over standard telecommunications networks. This enables
the experts to view the machine on their monitor from the same
perspective as the technicians. They can even use the software to write
instructions on specific parts of the machine such as ‘Remove this
screw’. These instructions then pop up on the technicians’ screen on
exactly the same part of the machine. And if a technician walks around
the machine with the laptop, the image moves accordingly—and the
written information stays where it was intended to be, for example
hovering over a specific screw.
Once
the technicians have carried out the experts’ instructions, the pop-up
information can be deleted by simply clicking on it. The system is based
on a chat protocol, which means everyone involved can communicate
either through the chat function or by telephone.
The
researchers managed to minimize the quantity of data transmitted to
allow the system to function over a cellphone network. That means there
is no need for a broadband connection, so technicians can call on
experts even from remote locations such as wind turbines in the middle
of a field or machines in newly industrialized and developing countries.
“We
only transmit location data, not pictures,” says Dr. Thomas Alexander,
who heads up the research team at the FKIE. “At the moment we do that by
attaching 2D barcodes to various parts of the machine. When the repair
technician takes a picture of the malfunctioning machine, the software
on the laptop reads those markings and links them to information in the
database—for example the machine’s identification code and the
position and location of the barcodes that appear in the picture. And
this is the only information that actually gets transmitted.”
Once
the data reaches the manufacturer, another software program links it to
the machine’s CAD data to enable the experts to view the machine from
the same angle as the on-site technician. The researchers have already
developed a prototype of the system, and the next step is to carry out a
study in which users will put the system through its paces. The results
should help the scientists to optimize the system and tailor it more
closely to user requirements.
