on-device AI

Apple is reportedly in talks with PrismML, a startup developing technology to significantly shrink AI models for on-device iPhone processing. This could revolutionize Apple’s AI strategy by enabling faster, more private, and offline AI capabilities, potentially enhancing Siri and unlocking new features. While PrismML’s methods drastically reduce model size and improve speed, a slight performance trade-off exists. Validation in real-world scenarios and power consumption remain key considerations.

Amazon is prioritizing in-house silicon for its consumer devices to enhance AI capabilities, enabling on-device processing for improved speed, privacy, and responsiveness. This strategy, evident in chips like AZ3, aims to create a more intelligent and interconnected user experience across its hardware ecosystem, unifying devices with an upgraded Alexa. This move positions Amazon to compete with tech giants and anticipates a future of more conversational AI interactions.

While Nvidia’s GPUs dominate the AI chip market, custom ASICs from tech giants like Google and Amazon are gaining traction due to their tailored performance and cost benefits. FPGAs offer reconfigurable solutions, and on-device AI chips from Qualcomm and Apple are enabling real-time processing. Experts highlight growing investments in ASICs and edge AI, potentially shifting the competitive landscape. Despite the competition, Nvidia’s established ecosystem and significant GPU shipments demonstrate its current strength and continuing efforts to maintain supply chain security.

SK hynix has begun shipping mobile DRAM featuring industry-first High-K EMC, significantly improving thermal management. This innovation boosts thermal conductivity by 3.5x and reduces thermal resistance by 47%, mitigating heat issues in on-device AI applications, where rising data transfer rates generate considerable heat. The breakthrough is achieved by incorporating Alumina into Silica-based EMC. SK hynix aims to lead in next-generation mobile DRAM through continued material innovation, enhancing performance and extending device lifespan.

At the China International Supply Chain Expo, Qualcomm showcased 40 years of innovation, highlighting advancements in 5G-A, on-device AI, and collaborations across various sectors. Demonstrations featured Snapdragon-powered devices, including smartphones, PCs, and XR headsets, emphasizing generative AI applications. The company also displayed automotive solutions and IoT applications using the Qualcomm QET1100 platform. Qualcomm reaffirmed its commitment to working with Chinese partners to drive digital transformation and intelligent connectivity across industries.

Google DeepMind has introduced Gemini Robotics On-Device, an offline AI model for robots. This VLA model processes visual data, language, and actions on-device, enabling real-time responses to human commands. Designed for two-armed robots, it interprets natural language instructions. This offers advantages in critical scenarios by removing reliance on cloud connectivity. The model is built on Gemini 2.0 and competes with rivals like Nvidia and OpenAI in the robotics market.

Google launched the AI Edge Gallery, an app enabling on-device AI on smartphones. Sourced from Hugging Face, it allows users to run AI models offline with features like image-based question answering and text manipulation via a “Prompt Lab.” Performance varies based on device hardware and model size. The app is an experimental Alpha version, open-source under Apache 2.0, and encourages user feedback.