Apple is reportedly in discussions with a promising Silicon Valley startup that claims to have developed a method for significantly shrinking large artificial intelligence models, enabling them to run directly on an iPhone. This development, if successful, could be a game-changer for Apple’s AI strategy, addressing a key bottleneck in deploying sophisticated AI capabilities on mobile devices.
PrismML, a relatively new company backed by Khosla Ventures and spun out of the California Institute of Technology, recently unveiled compressed versions of Alibaba’s open-source Qwen model. Their breakthrough lies in reducing the model’s footprint from approximately 54 gigabytes to under 4 gigabytes, allowing its 27 billion parameters to operate efficiently on an iPhone 15 or newer devices.
Babak Hassibi, CEO of PrismML, shared with CNBC that Apple, among other tech giants, has been actively evaluating their technology. The focus of these evaluations is on key performance metrics such as speed, energy efficiency, and overall device performance. “They are really evaluating our technology right now,” Hassibi stated, characterizing the ongoing discussions as being in their nascent stages. While the ultimate outcome remains uncertain, he indicated that “things are progressing nicely.”
This news emerges just as Apple has opened the public beta for iOS 17, granting users early access to its revamped Siri. Apple’s ongoing efforts aim to bolster Siri’s competitiveness against established AI assistants from companies like OpenAI and Anthropic. A critical aspect of Apple’s approach is its commitment to processing more personal information and AI tasks directly on the device, a strategy that PrismML’s technology could significantly bolster.
The primary challenge for on-device AI has been the substantial memory and processing power required by the most advanced models, making them unsuitable for smartphones. While Apple can leverage cloud-based models for complex tasks, running AI locally on the iPhone promises several advantages: reduced latency associated with data transmission to remote servers, lower cloud computing expenses, and an enhanced privacy proposition. Furthermore, on-device AI would enable certain functionalities to operate seamlessly, even without an internet connection.
Industry analysts believe that the ability to run smaller, more efficient AI models on a device could unlock new possibilities for Apple. Carolina Milanesi, president and principal analyst at Creative Strategies, suggests that this could lead to more demanding features being integrated directly into the iPhone, such as advanced computational photography, video generation tools, and sophisticated health and fitness monitoring that handle sensitive personal data. “The more you can do on device, the better it is,” Milanesi emphasized, particularly highlighting the user preference for keeping sensitive health information private.
PrismML’s compression technique involves a radical simplification of how AI model parameters are stored, reducing values from 16 bits to a mere one or three possible states. This drastically slashes the memory footprint needed for both storage and operation. Hassibi likens this advancement to the chip industry’s transition from 8-bit to 4-bit computing, but with an even more profound impact. The company claims its compressed models require 10 to 15 times less memory, deliver responses six to eight times faster, and consume significantly less energy compared to their conventional counterparts.
However, Hassibi acknowledges a trade-off: PrismML’s models typically experience a slight dip in overall performance, with factual recall being the first to show a marginal decline before core capabilities like reasoning, math, and coding are affected. PrismML has made two compressed versions of the model available for free, designed to run on a range of consumer devices including iPhones, MacBooks, and PCs.
The underlying technology originated from Hassibi’s research group at Caltech, with the university holding the patents and exclusively licensing them to PrismML. The startup secured a $16.25 million seed funding round in March, with participation from Khosla Ventures and other investors. PrismML has indicated that Google’s open-source Gemma model is next in its pipeline for compression, followed by even larger, frontier models that currently necessitate datacenter-grade hardware. Beyond personal devices, PrismML envisions its technology extending to robotics, autonomous systems, and other applications requiring rapid, cloud-independent decision-making. “It’s very important that the intelligence be local and that it can run fast,” Hassibi reiterated.
Apple’s strategy has long involved a hybrid approach to AI, performing some tasks locally, such as translation and summarization, particularly those involving sensitive personal data. More complex computations are typically offloaded to Apple’s private cloud infrastructure or external models. Horace Dediu, founder of Asymco, posits that Apple’s objective is likely to keep the majority of routine Siri interactions on-device while reserving the most resource-intensive tasks for the cloud. He notes that the advantage extends beyond memory reduction to fitting more capable models within physical device constraints. “They’re trying to figure out how big a model and how clever a model they can fit on the device,” Dediu explained. Localizing common requests offers Apple the benefits of lower latency, enhanced privacy, and potentially reduced licensing and cloud infrastructure costs. Apple’s integrated hardware and software design, encompassing its custom silicon, provides a unique advantage in optimizing AI performance on its devices.
Despite the potential, analysts caution that PrismML’s claims require rigorous validation in real-world scenarios. Tarun Pathak, research director at Counterpoint Research, highlighted that critical factors for success will include the model’s performance with extended prompts, battery consumption during multitasking, and reliability across millions of diverse use cases. “The ultimate test will be millions of queries, thousands of device combinations and robust testing at scale,” Pathak stated. Phil Solis, who leads IDC’s research on client processors, identified power consumption as a significant open question. A continuously active AI model, even if memory-efficient, could substantially drain a smartphone’s battery.
The recent advancements in AI efficiency, exemplified by PrismML’s work, are also fueling a broader debate about the future demand for memory chips and expensive datacenter infrastructure. Memory has emerged as a critical constraint and cost driver across consumer electronics and AI servers. Morgan Stanley forecasts a substantial increase in Apple’s average dynamic random-access memory (DRAM) cost per bit in fiscal 2027, with NAND flash memory costs also expected to rise significantly. The firm anticipates that Apple may increase the starting price of its future iPhone models to maintain profit margins.
PrismML’s technology suggests that a cloud model requiring multiple GPUs could potentially run on a single unit, and models previously confined to servers could be deployed on consumer devices. This could reduce the overall memory and computing capacity needed for specific AI tasks. However, this does not necessarily signal a decline in aggregate chip demand. Gil Luria, an analyst at D.A. Davidson, argues that model compression will not eliminate the need for processors or memory but rather shift their deployment from datacenters to individual devices. “It’s not that you’re not going to need the chip,” Luria remarked. “You’re still going to need the GPU, and you’re still going to need the memory.” He also pointed out that on-device AI can sometimes be less efficient than centralized datacenter processing due to the underutilization of mobile chips. Furthermore, efficiency breakthroughs can spur increased usage rather than cost reduction, as more accessible and faster AI enables new applications and encourages more frequent model execution.
The market has demonstrated sensitivity to any indication of reduced memory demand for AI. Micron’s shares experienced a notable decline in March following Google’s publication of its TurboQuant paper, which detailed methods for extreme memory compression without compromising model performance. While the stock eventually recovered, the episode underscores the market’s focus on memory chip demand. PrismML’s public release offers both consumers and investors an opportunity to scrutinize the practical efficacy of their claims beyond laboratory settings. For Apple, enabling more capable AI directly on the iPhone could be instrumental in enhancing Siri’s functionality while preserving its core tenets of privacy and hardware integration. “The combination of cloud and on-device AI can serve a more complete, efficient, and privacy-centric AI experience,” concluded Counterpoint’s Pathak. “Complex tasks will be offloaded to the cloud, whereas sensitive, latency-critical, and privacy-relevant tasks will be executed on-device.”
Original article, Author: Tobias. If you wish to reprint this article, please indicate the source:https://aicnbc.com/23731.html
