Despite recent technology advances, artificial arms and hands still are not able to provide sensory feedback, like the feel of things being touched, or awareness of position and movement. Without this feedback, even the most advanced prosthetic limbs remain numb to users, a factor that impairs effectiveness and their wearers’ willingness to use them.
With support from the Defense Advanced Research Projects Agency (DARPA), and in support of the White House BRAIN Initiative, a collaborative research team has set out to build the world’s first neural system to enable naturalistic feeling and movements in prosthetic hands. The Hand Proprioception and Touch Interfaces (HAPTIX) program aims to provide wounded service members with full and natural dexterous control over advanced prosthetic devices that substitute for amputated hands — the name HAPTIX is a play on the word haptics, referring to the sense of touch.
Managed by Dr. Doug Weber, HAPTIX is being run out of DARPA’s Biological Technologies Office (BTO). The program’s goal is to provide amputees with prosthetic limb systems that feel and function like natural limbs, and to develop next-generation sensorimotor interfaces to drive and receive rich sensory content from these limbs. If successful, the program will provide patients the psychological benefit of having natural sensation in their prosthetic hands, as well as reduction of “phantom limb” pain, a sensation about 80 percent of amputees feel despite the removal of a limb.
“The ultimate goal for HAPTIX is to create a device that is safe, effective and reliable enough for use in everyday activities,” said Weber. “DARPA is partnering with scientists at the Food and Drug Administration to help develop standards for verifying safety and quantifying benefits of this new class of advanced technologies. We hope to streamline the process of validating technologies that can help our military service members and veterans who have been injured while serving our country.”
In Phase 1 of the program, DARPA is evaluating several distinct technical approaches. Those that prove successful would continue into Phase 2, which would integrate selected technology components into a complete HAPTIX test system. The agency plans to initiate take-home trials of a complete, FDA-approved HAPTIX prosthesis system within four years.
The program plans to adapt one of the prosthetic limb systems developed recently under DARPA’s Revolutionizing Prosthetics program to incorporate interfaces that provide intuitive control and sensory feedback to users. These interfaces would build on advanced neural-interface technologies being developed through DARPA’s Reliable Neural-Interface Technology (RE-NET) program.
Technology components
Where appropriate, HAPTIX teams intend to leverage commercially available technologies, such as intramuscular electrodes and lead technologies developed initially for cardiac pacemakers. The program also plans to test advanced microelectrode array and nerve cuff electrode technologies that have been developed over the past two decades with support from the National Institutes of Health (NIH), the Department of Veterans Affairs and DARPA.
DARPA is working with teams led by the following institutions:
- Case Western Reserve University
- Cleveland Clinic
- Draper Laboratory
- Nerves Incorporated
- Ripple LLC
- University of Pittsburgh
- University of Utah
- University of Florida
Gazebo simulation environment
To help teams more quickly and cost-effectively conduct their research, DARPA is providing prosthetics simulation software for testing designs. This software includes a variant of the DARPA Robotics Challenge (DRC) Simulator from the June 2013 Virtual Robotics Challenge — Gazebo — which helped to expedite initial design and evaluation of semi-autonomous robots that could aid in emergency response efforts.
“The DARPA Robotics Challenge Simulator was a big help for DRC, and we immediately saw how adapting its virtual testing environment could benefit HAPTIX research,” Weber said. “The simulator will enable rapid and low-cost development of the HAPTIX technology and also provide amputees with a realistic experience for learning to use their physical prosthesis.”
As the organization maintaining Gazebo, Open Source Robotics Foundation (OSRF) has been tasked with extending the Gazebo environment to simulate prosthetic hands and test environments, and developing both graphical and programming interfaces to the hands. OSRF is officially releasing a new version of Gazebo for use by HAPTIX participants. Highlights of the new release include
- support for OptiTrack motion capture system
- the NVIDIA 3-D vision system
- numerous teleoperation options, including the Razer Hydra, SpaceNavigator, mouse, mixer board and keyboard
- a high-dexterity prosthetic arm
- programmatic control of the simulated arm using Linux, Windows and MATLAB
“Our track record of success in simulation as part of the DARPA Robotics Challenge makes OSRF a natural partner for the HAPTIX program,” said John Hsu, Chief Scientist at Open Source Robotics Foundation. “Simulation of prosthetic hands and the accompanying GUI will significantly enhance the HAPTIX program’s ability to help restore more natural functionality to wounded service members.”
Gazebo is an open source simulator that makes it possible to rapidly test algorithms, design robots and perform regression testing using realistic scenarios. Gazebo provides users with a robust physics engine, high-quality graphics, and convenient programmatic and graphical interfaces. This project also marks the first time Windows and MATLAB users can interact with Gazebo, thanks to new cross-platform transport library.
Teams participating on HAPTIX will have access to a customized version of Gazebo that the Johns Hopkins University Applied Physics Laboratory Modular Prosthetic Limb (MPL) developed under the DARPA Revolutionizing Prosthetics program, as well as representative physical therapy objects used in clinical research environments.
Neural interface systems
Lawrence Livermore National Laboratory (LLNL)’s Neural Tech Group, in collaboration with Case Western Reserve University and the Louis Stokes Cleveland Veterans Administration Medical Center, is developing neural interface systems to measure and decode motor signals recorded in peripheral nerves and muscles in the forearm by using tiny electrodes. Sensory feedback, especially from the hand, is vitally important for many functions, and HAPTIX seeks to create a sensory experience so rich and vibrant that users would want to wear their prostheses full time.
“The HAPTIX project intends to achieve a phenomenal breakthrough in prosthetics never thought possible,” LLNL’s project leader Sat Pannu explained. “Its neural system intends to re-create a range of functions, including a real feeling of touch when holding another person’s hand.”
For these cutting-edge neural interface systems, LLNL will further develop the advanced prosthetic limb systems developed under DARPA’s RE-NET programs, which has made major steps forward in providing a direct and powerful link between user intent and prosthesis control. The HAPTIX program will incorporate sensors, delivered through a patterned stimulation of sensory pathways in peripheral nerves, which provide tactile and proprioceptive feedback to the patient from their hands.
The Revolutionizing Prosthetics and RE-NET programs, combined with the neural interface systems, will allow users to control prosthetic hand movements with their thoughts and will have natural sensations. This means that the bionic hand would be able to perform movements of a human hand and experience pressure, touch and texture.
LLNL reports that one of HAPTIX’s key challenges is identifying stimulation patterning strategies that elicit naturalistic sensations of touch and movement. The ultimate goal is to create a fully implantable device that is safe, reliable, effective and approved for human use.
Pannu and his team of engineers are developing wireless electronic packages for HAPTIX called smart packages. These packages will contain electronics that record and stimulate the peripheral nervous system to control movement and sensation in a patient’s prosthetic hand. Smart packages are designed to miniaturize electronics, normally the size of a third of a cell phone, into a package the size of a watch battery. The electronics would be made of ceramics and titanium, biocompatible materials that seal the package tightly, preventing components from leaking into nerves or human tissue from entering the package.
“The packages have to be really small, so they don’t put any weight or pressure on the nerves,” said Pannu, adding that the smart packages need to bond with the electrodes to function. “We don’t want to damage the nerves.”
BRAIN Initiative collaboration
Three groups of researchers who have received support from the NIH will be funded through the President’s BRAIN Initiative to improve artificial limb technology. As part of HAPTIX, the new awards will be administered by DARPA and will build upon fundamental discoveries made possible by NIH support.
“These projects speak to the power of integrated government-funded science. NIH support enables bright inquisitive scientists to develop whole new areas of research. Now, as part of ambitious DARPA projects, these groups will have the opportunity to turn their discoveries to benefit amputees, particularly those brave men and women who lost their limbs defending our country,” said Walter Koroshetz, M.D., acting director of the NIH’s National Institute of Neurological Disorders and Stroke (NINDS), one of the institutes that supported the research.
“The BRAIN Initiative provides a wonderful platform for collaboration between NIH and DARPA, as well as other federal agencies to work towards the common goal of improving the lives of wounded service members. By joining forces we can quickly move this important technology from the lab to the clinic,” he added.
“This is a wonderful example of how NIH’s investment in the development of novel biomedical technologies opens up a range of possibilities for treating different medical conditions,” said Roderic Pettigrew, Ph.D., M.D., director of the National Institute of Biomedical Imaging and Bioengineering (NIBIB), an NIH institute that supported the research. “We are thrilled to see this technology move forward with new and impactful applications.”
NIH-supported research that will contribute to the HAPTIX effort includes
- Implantable Myoelectric Recording Array for Control of Prostheses
Prosthetic arms that control muscle activity with surface electrodes placed on the skin are available today. However, the resulting movements are slow and not well-controlled, leading many patients to stop using their prostheses. Researchers have developed a wireless device to be implanted directly into muscles that may help improve the reliability of prosthetic limbs. Their goal is to develop a device that will be convenient for patients to use and will create a more natural feeling of prosthesis control. Compared to surface electrodes, the implant should allow users to freely move multiple joints in the prosthetic limb at the same time. The implant collects information from the muscles and sends that information wirelessly to an external receiver built into the prosthetic limb. A prototype of this device showed in early studies that it is safe, requires minimal surgery and effectively records from muscles. The team is developing a smaller version, which will soon be used in clinical studies. - Multisite, Intrafascicular Stimulation for Stance
Good things come in small packages, and Utah Slanted Electrode Array (USEA) technology is no exception. This tiny device, a square grid of 100 microelectrodes, can be implanted in peripheral nerves to communicate with nerve fibers and control muscle movement. This technology was based on a device originally developed for recording signals within the brain. Researchers took the original flat structure and modified it into a 3-D electrode arrangement (the USEA), which is better for recording from peripheral nerves because it ensures uniform coverage of a specific area. The USEA can receive information from a large number of fibers, while activating a select few, resulting in smooth and specific muscle movements - Enhancing Neuroprosthesis Performance with Nerve Cuff Electrodes
More than a decade of NIBIB and NINDS-supported research has led researchers to develop, implant and test high-density electrode nerve cuffs in patients with spinal cord injuries in an effort to return some mobility. The cuffs wrap around major nerves in the arms or legs and stimulate them electrically, which results in muscle contraction and simple movements. These cuffs will now be tested to stimulate nerves in the residual limb directly above a prosthetic arm so that a user can control the prosthesis more naturally and also experience sensation. The significance of the high-density electrode cuff is that multiple electrodes, implanted chronically, allow the nerve to be stimulated in a wide variety of patterns that can be interpreted as sensation at many different locations.
For further information about the HAPTIX program, visit: http://www.darpa.mil/program/hand-proprioception-and-touch-interfaces?