Researchers from the Moscow Institute of Physics and Technology have developed a model for predicting hand movement trajectories based on cortical activity: Signals are measured directly from a human brain. The predictions rely on linear models. This offloads the processor, since it requires less memory and fewer computations in comparison with neural networks. As a result, the processor can be combined with a sensor and implanted in the cranium. By simplifying the model without degrading the predictions, it becomes possible to respond to the changing brain signals. This technology could drive exoskeletons that would allow patients with impaired mobility to regain movement. The paper was published in Expert Systems with Applications, the leading journal in the field of artificial intelligence.
Damage to the spinal cord prevents signals generated by the brain to control limb motion from reaching the muscles. As a result, the patients can no longer move freely. To restore motion, brain cortex signals are measured, decoded, and transmitted to an exoskeleton. Decoding means interpreting the signals as a prediction of the desired limb motion. To pick up high-quality signals, the sensor needs to be implanted directly in the braincase.
A surgical implantation of a sensor with electrodes onto the motor cortex, the area of the brain responsible for voluntary movements, has already been performed. Such a sensor is powered by a compact battery recharged wirelessly. The device comes with a processing unit, which handles the incoming signals, and a radio transmitter relaying the data to an external receiver. The processor heats up during operation, which becomes problematic since it is in contact with the brain. This puts a constraint on consumed power, which is crucial for decoding the signal.
Brain overheating. Image: Lion on helium/MIPT Press Office
Adequately measuring brain signals is only one part of the challenge. To use this data to control artificial limbs, movement trajectories need to be reconstructed from the electrocorticogram — a record of the electrical activity of the brain. This is the point of signal decoding. The research team led by Professor Vadim Strijov from MIPT works on models for predicting hand trajectories based on electrocorticograms. Such predictions are necessary to enable exoskeletons that patients with impaired motor function would control by imagining natural motions of their limbs.
“We turned to linear algebra for predicting limb motion trajectories. The advantage of the linear models over neural networks is that the optimization of model parameters requires much fewer operations. This means they are well-suited for a slow processor and a limited memory,” explains Strijov, the senior author of the paper.
“We solved the problem of building a model that would be simple, robust, and precise,” adds Strijov, who is a chief researcher at MIPT’s Machine Intelligence Laboratory. “By simple I mean there are relatively few parameters. Robustness refers to the ability to retain reasonable prediction quality under minor changes of parameters. Precision means that the predictions adequately approximate natural physical limb motions. To achieve this, we predict motion trajectories as a linear combination of the electrocorticogram feature descriptions.”
In a 2013 study by Yasuhiko Nakanishi et al. published in PLOS One, patients moved three blocks along the sides of a 25-by-25-centimeter square. Signals picked up from the brain of a patient were recorded as electrocorticograms (ECoGs) and matched with the hand movements. Image: Lion_on_helium/MIPT Press Office
Each electrode outputs its own signal represented by a frequency and an amplitude. The frequencies are subdivided into bands. The feature description is a history of corticogram signal values for each electrode and each frequency band. This signal history is a time series, a vector in linear space. Each feature is therefore a vector. The prediction of hand motion trajectory is calculated as a linear combination of feature vectors, their weighted sum. To find the optimal weights for the linear model — that is, those resulting in an adequate prediction — a system of linear equations has to be solved.
However, the solution to the system mentioned above is unstable. This is a consequence of the sensors being located close to each other so that neighboring sensors output similar signals. As a result, the slightest change in the signals that are picked up causes a considerable change in the trajectory prediction. Therefore, the problem of feature space dimensionality reduction needs to be solved.
The authors of the paper introduce a feature selection method based on two criteria. First, the pairs of features have to be distinct, and second, their combinations have to approximate the target vector reasonably well. This approach allows the optimal feature set to be obtained even without calculating the model parameters. Taking into account the mutual positions of the sensors, the researchers came up with a simple, robust, and rather precise model, which is comparable to its analogs in terms of prediction quality.
In their future work, the team plans to address the problem of limb trajectory description in the case of a variable brain structure.
Strijov explains: “By moving around and getting a response from the environment, humans learn. The structure of the brain changes. New connections form, rendering the model obsolete. We need to propose a model that would adapt to the changes in the brain by changing its own structure. This task is far from simple, but we are working on it.”
