[Created with Sora]
The potential rise of AI-based hardware
The growth of compute capabilities at the edge
Tired of data chaos in your lab?
Learn a practical “Crawl-Walk-Run” blueprint to unify R&D data, strategies for getting an ROI for your data efforts, and pave the way for advanced automation and AI. Join experts from Labcorp, Parallel Bio, Pfizer, and the ISS National Lab in our upcoming webinar.
Wednesday, June 11, 2025 | 2:00 PM EDT
Meanwhile, Apple has also invested on building neural engines. Since the A11 Bionic in 2017, the Cupertino-based firm has included a dedicated neural engine in its mobile system-on-chips (SoCs). The latest iterations, such as the one in the Apple M3/M4 chip series, have impressive capabilities. For instance, Apple reports that the M4 (2024) has a 16-core Neural Engine capable of up to 38 trillion operations per second (TOPS). For perspective, the first generation M1 launching in 2020 supported 11 TOPS. Both the M4 chip and the A18 Pro chip in the iPhone 16 Pro series enable a number of generative AI features, related to writing and generating images.
This trend extends beyond dedicated devices, AI is also, increasingly, coming to smartphones, and not just via the cloud. Virtually every high-end smartphone SoC now includes an AI module. Examples include Qualcomm’s Hexagon DSP with tensor extensions, Apple’s Neural Engine, and Google’s Pixel Neural Core. Then there’s Huawei’s Ascend NPU in Kirin chips and Samsung’s Neural Processing Unit in Exynos. In addition, Qualcomm’s 2023-launched Snapdragon 8 Gen 3 (2023) advertised the ability to run a roughly 10 billion parameter language model entirely on the device using its AI engine. That level of compute is sufficient to allow features like generative text or image creation offline locally. Apple has also explored running smaller LLMs locally.
Rounding out the playing field is a fast‑growing cohort of specialist firms, many licensing Arm’s configurable Ethos‑N78 NPU blocks instead of building a full CPU/GPU stack from scratch. Here, players are focused on meeting demand for ultra‑efficient edge inference. IDTechEx forecasts that silicon dedicated to edge AI will balloon to about $22 billion by 2034. Early signs are promising. Israel‑based Hailo packs 26 TOPS of neural performance into a module smaller than a postage stamp while drawing roughly 2–3 W. In any event, packing this much computational power into wearable form factors presents real engineering challenges.
Dealing with heat and other challenges
One challenge AI hardware developers will need to navigate when developing high-powered wearables is heat. Even a few watts of heat can be too uncomfortable when electronics sit on your skin. Meta’s Ray-Ban smart glasses integrate a camera, speakers, and an NPU into a compact frame powered by a 154 mAh battery (about 0.6 Wh), supporting roughly four hours of moderate use. Intensive features like live AI assistance, however, can significantly drain power, reducing battery life to as little as 30 minutes.
Prior heat‑transfer models show a wearable becomes uncomfortable once it dissipates more than about 1.75 W. Humane’s AI Pin learned this the hard way. Staff reportedly iced units before demos because the laser projector overheated the badge, according to Business Insider.
Software fragmentation is another headache. Building an AI-powered app today means navigating a maze of incompatible chip architectures, each demanding its own SDK and optimization approach. A computer vision startup, for instance, might need to retarget their object detection model across Apple’s Neural Engine, Qualcomm’s Hexagon DSP, Samsung’s Xclipse GPU, and Huawei’s Ascend NPU. Complicating matters further, each has different memory layouts, precision formats, and performance characteristics. Still, there are signs of standardization. The industry is converging around common runtimes like ONNX Runtime, TensorFlow Lite, and Apple’s Core ML as portability layers.
Toward a genAI razor-and-blades model
Historically a tech gadget’s price embodied nearly all its profit, with whatever code it ran thrown in as a “free” feature update or two. In some cases, products like the initial generations of the Amazon Echo and similar smart-home hubs and speakers were sold at close to cost, where profits were expected to emerge from software and services. While the strategy may not have worked as intended, costing Amazon billions, as the WSJ noted. Tesla has also pursued a similar strategy with its Full Self-Driving (FSD) offering, an optional package that upgrades its vehicles Autopilot system, providing semi-autonomous driving capabilities. In any event, generative AI hardware will likely follow a similar game plan as Amazon’s Alexa and Tesla’s FSD plays. That is, the money is made on the back a subscription that draws on a stream of model refreshes and cloud inference cycles.
This strategy, however, is uncertain. With DeepSeek and other open-source models achieving performance parity to some advanced models at a fraction of the cost, some analysts argue that genAI itself is becoming commoditized. DeepSeek’s open-sourced R1 model could be a harbinger of commoditization. Several major automakers, including BMW but also Chinese firms like Geely, BYD, and Dongfeng’s Voyah brand, have announced partnerships to incorporate R1 into their vehicles.
Amazon announced Alexa Plus in February 2025, with plans to make it free for Prime members or $19.99/month standalone, but its launch has been delayed and the service has yet to be released. Founded by ex-Apple employees, the startup Humane takes this idea further. Its screen-free AI Pin pairs a $699 badge with a $24/month subscription for cellular data, cloud compute, and unlimited GPT queries. Outside pure AI, Peloton’s growth shows investors how sticky a “device + membership” model can be with fitness classes, performance tracking, and community features in the mix.
Apple has also set a precedent for bundling hardware and software together to optimize sales. Here, perhaps OpenAI’s $6.5 billion deal with Apple veteran Jony Ive is key. “OpenAI, like Meta, doesn’t want to be dependent on smartphone makers—such as Apple—for distribution of its apps,” as Martin Peers pointed out in The Information. But the one factor that could undermine all of this is gadget fatigue. New genAI hardware has to either solve a real user pain point or to offer a dramatically improved experience to thrive. If Altman’s bet is more about sidestepping Big Tech so OpenAI can control distribution, the strategy may not work. Even Apple has had trouble in creating new product classes lately. Its Apple Vision Pro was something of a dud or at least falls well short of runaway‑hit status. Apple reportedly reduced projected demand to 400–450 k down from 700‑800 k.
With genAI hardware, too, if consumers decide they don’t need another gizmo, the companies working on creating a new breed of hardware could end up creating products nobody asks for.
