Apple is going to join the company of artificial intelligence model developers with its own OpenELM family. This is a line of open-source large language models (LLMs) that can run entirely on a single device instead of connecting to cloud servers.
OpenELM is short for Open-source Efficient Language Models. They are aimed at applications on a device. Apple claims that the «family of models aims to empower and strengthen the open research community by contributing to future» research efforts. The models have been pre-trained on publicly available datasets of 1.8 trillion tokens from Reddit, Wikipedia, arXiv.org, and more.
Apple is joining the public AI game with 4 new models on the Hugging Face hub! https://t.co/oOefpK37J9
— clem 🤗 (@ClementDelangue) April 24, 2024
OpenELM consists of small models designed to perform text creation tasks efficiently. The corresponding code is published in the artificial intelligence code community Hugging Face. There are 8 OpenELM models in total: 4 pre-trained and 4 customized by instructions. They cover different parameter sizes – from 270 million to 3 billion parameters.
OpenELM models can run on common user devices such as laptops and smartphones. Apple notes that the tests were conducted on a «Intel i9-13900KF workstation equipped with 64 GB of DDR5-4000 DRAM and an NVIDIA RTX 4090 GPU with 24 GB of video memory running Ubuntu 22.04», as well as a «Apple MacBook Pro with built-in M2 Max system and 64 GB of RAM running macOS 14.4.1».
Apple offers its OpenELM models under the so-called «source code license» and provides various training checkpoints, statistics on model performance, as well as instructions for pre-training, evaluation, and customization of instructions and parameters.
The Sample Code License does not prohibit commercial use or modification, only that «if you redistribute the Apple Software in its entirety and without modification, you must retain this notice and the following text and disclaimer in all such redistributions of the Apple Software».
In addition, the company notes that the «models are available without any security guarantees. Consequently, there is a possibility that these models will produce inaccurate, harmful, biased, or undesirable results in response to user requests».
Source: venturebeat