Ai2 has announced the release of the OLMoE app for iOS, allowing users to run state-of-the-art language models directly on their devices. This is an important step in the development of on-device AI, providing private and secure access to language model capabilities without the need for an internet connection. The app can be downloaded from the Apple App Store or built from source via Ai2’s open repository.
The app is designed for newer Apple devices, including the iPhone 15 Pro and later models, as well as iPads with M-series chips. These hardware requirements are due to the need for eight gigabytes of memory to support the computational demands of the OLMoE model, which is optimized for efficiency using Q4_K_M quantization. Despite this compression, the model maintains high performance with minimal loss in evaluation scores.
OLMoE continues Ai2’s commitment to openness in AI development. The model leverages advancements from previous iterations, such as the Dolmino mid-training mix and Tülu 3 post-training recipe, achieving a 35% improvement in evaluation metrics compared to earlier versions. The app integrates these innovations into an accessible platform for researchers and developers to test real-world applications, improve local AI models, and experiment with open-source code.
Ai2 sees this release as a foundational step toward widespread on-device AI adoption. By eliminating reliance on cloud infrastructure, the OLMoE app ensures data privacy and reliability, showcasing the growing potential of mobile devices as platforms for advanced AI capabilities.
