Researchers in Japan have developed an inventive approach to address the challenge of implementing artificial intelligence (AI) on resource-constrained edge devices, paving the way for more sophisticated and efficient Internet of Things (IoT) systems.
The team, led by Professor Takayuki Kawahara and Mr. Yuya Fujiwara from the Tokyo University of Science, focused on a type of AI called binarized neural networks (BNNs) known for their efficiency in computing resources.
“BNNs are ANNs that employ weights and activation values of only -1 and +1, and they can minimize the computing resources required by the network by reducing the smallest unit of information to just one bit,” said Kawahara. “However, although weights and activation values can be stored in a single bit during inference, weights and gradients are real numbers during learning, and most calculations performed during learning are real number calculations as well. For this reason, it has been difficult to provide learning capabilities to BNNs on the IoT edge side.”
To overcome this, the researchers proposed a new training algorithm for BNNs called ternarized gradient BNN (TGBNN), which employs ternary gradients during training while maintaining binary weights and activations.
To further optimize the system, the researchers integrated the TGBNN algorithm into a computing-in-memory (CiM) architecture, where calculations occur directly in memory, reducing circuit size and power consumption. This was achieved through a novel magnetic RAM (MRAM) array design, which uses magnetic tunnel junctions to store information in their magnetization state.
The researchers used two different mechanisms to manipulate the stored values in individual MRAM cells: spin-orbit torque and voltage-controlled magnetic anisotropy. These techniques enabled a significant reduction in the size of the calculation circuits compared to conventional methods.
The team tested the performance of their MRAM-based CiM system for BNNs using the MNIST handwriting dataset. Their TGBNN model achieved an accuracy of over 88% while matching the performance of regular BNNs and converging faster during training.
This development holds significant potential for a wide range of applications. For example, wearable health-monitoring devices could become more efficient and less reliant on cloud connectivity. Similarly, smart homes could perform more complex tasks and operate more responsively. The design could also contribute to sustainability goals by reducing energy consumption across various IoT applications.
The researchers believe that this breakthrough could significantly advance the integration of AI into IoT devices, leading to more powerful and efficient edge computing capabilities.
Reference
Title of original paper: TGBNN: Training Algorithm of Binarized Neural Network with Ternary Gradients for MRAM-based Computing-in-Memory Architecture
