Apple/OpenELM: Efficient Open-Source Family Language Models (opens in new tab)

OpenELM, a family of Efficient Language Models Developed by Apple, is trending on Hugging Face!

OpenELM offers models with 270M to 3B parameters, pre-trained and instruction-tuned, with Good results across various benchmarks.

My Feedback:

First Phi 3, now OpenELM. It's great to see these small models improving. I know they're not ready for production in all cases, but they're really great for specific tasks.

I see small open-source models as the future because they offer better speed, require less compute, and use fewer resources, making them more accessible and practical for a wider range of applications.

What do you think about this? Do you consider using small opensource. If yes what you are thinking to make?

I am going to use it on my smartphone

ArXiv Paper:

https://arxiv.org/abs/2404.14619

Why'd it drop today? One supposes that instead of pressing shift+delete on their repo they click publish now so they get to write the headline that 2 big tech companies release small language models on the same day.

Huh, They used the pile - that's a pretty interesting choice for a corporate research team?

Article title(h1): OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework

Any idea how much ram this requires?