The recent success of Pokémon GO made many people very familiar with the concept of “augmented reality”: computer-generated perception blends into the real and virtual worlds. So far, these apps have largely used optical methods for motion detection. Physicists at the German Helmholtz-Zentrum Dresden-Rossendorf (HZDR) working together with colleagues at the Leibniz Institute for Solid State and Materials Research (IFW) and the Johannes Kepler University Linz (JKU) (Austria) have now developed an ultrathin electronic magnetic sensor that can be worn on skin. Just by interacting with magnetic fields, the device enables a touchless manipulation of virtual and physical objects. The results are published in the journal Science Advances(DOI: 10.1126/sciadv.aao2623).

At first glance, the shiny little gold elements look like a modern tattoo. But on this extremely thin, almost invisible foil that sticks to the palm of the hand like a second skin, there are sensors which provide people with a “sixth sense” for magnetic fields. These sensors will enable people to manipulate everyday objects or control appliances both in the physical world and in augmented or virtual reality with mere gestures, similar to how we use a smartphone now. This is the vision nurtured by Dr. Denys Makarov of the Institute of Ion Beam Physics and Materials Research at HZDR.

For the first time, the physicist and his team – together with the groups of Prof. Oliver G. Schmidt at IFW Dresden and Prof. Martin Kaltenbrunner in the Soft Electronics Laboratory at JKU Linz – have now demonstrated that the ultrathin, compliant magnetic field sensors in combination with a permanent magnet are able to sense and process body motion in a room. “Our electronic skin traces the movement of a hand, for example, by changing its position with respect to the external magnetic field of a permanent magnet,” explains Cañón Bermúdez of HZDR, the lead author of the study. “This not only means that we can digitize its rotations and translate them to the virtual world but also even influence objects there.” Using this technique, the researchers managed to control a virtual light bulb on a computer screen in a touchless way.

A virtual lamp

To achieve this result, they set a permanent magnet in a ring-shaped plastic structure emulating a dial. Then, they associated the angle between the wearable sensor and the magnetic source with a control parameter which modulated the intensity of the light bulb. “By coding the angles between 0 and 180 degrees so that they corresponded to a typical hand movement when adjusting a lamp, we created a dimmer – and controlled it just with a hand movement over the permanent magnet,” says Makarov, describing one of the experiments. The researchers were also able to use a virtual dial in the same way. The physicists at Dresden envision that their approach provides a unique alternative for interfacing the physical and the virtual world that goes far beyond what is possible with current technologies.

“To manipulate virtual objects, current systems essentially capture a moving body by optical means,” Makarov explains. “This requires, on one hand, a load of cameras and accelerometers and, on the other hand, fast image data processing. However, usually the resolution is not sufficient to reconstruct fine movements of the fingers. Moreover, because they are so bulky, the standard gloves and glasses hamper the experience of virtual reality.” The skin-like sensors could be a better way of connecting human and machine, according to Martin Kaltenbrunner: “As our polymer foils are not even three micrometers thick, you can easily wear them on your body. Just by way of comparison: a normal human hair is roughly 50 micrometers thick.”

As further experiments have shown, the sensors can also withstand bending, folding and stretching without losing their functionality. Thus, in Oliver G. Schmidt’s opinion, they are suitable for the incorporation into soft, shapeable materials like textiles to manufacture wearable electronics. Makarov sees an additional advantage to the new approach in contrast to optical systems: no direct line of sight between the object and the sensors is necessary. This could open up potential applications in the security industry, as well. Buttons or control panels in rooms which cannot be entered in hazardous situations, for example, could be operated by remote control via the sensors.