Mixture of Experts
-
AI model trained on AMD GPUs achieves milestone
Zyphra, AMD, and IBM have collaboratively developed ZAYA1, a Mixture-of-Experts foundational model, using AMD’s GPUs and platform. Trained on AMD’s Instinct MI300X accelerators within IBM Cloud, ZAYA1 demonstrates comparable or superior performance to established open-source models. Zyphra optimized ROCm for AMD GPUs, focusing on memory capacity and inter-GPU communication. This initiative highlights the viability of AMD-based solutions as a cost-effective alternative to NVIDIA for large-scale AI model training, potentially impacting GPU market dynamics and AI procurement strategies.
-
“Huawei Boosts DeepSeek’s AI Performance: 10% Reduction in Inference Latency Through Expert Optimization”
When it comes to the most talked-about models in recent times, the Mixture of Experts (MoE…
