Tiny is mighty, it seems. This week, tech giants Microsoft and Apple both released open source language models with a small footprint, claiming that despite its compactness, they outperform larger models on text generation benchmarks. These efficient models are designed to run on-device, presumably yielding faster inferences because they don’t have to connect to cloud servers to run.

What’s powerful here — if these language models can run on-device, you truly have AI in your pocket that runs anytime, anywhere, even without the internet. This could potentially open up a whole horizon of use-cases of AI in the physical world — like running constantly in the background of a device like an iphone or accessed where there’s no internet connectivity (like mid-flight or in a war-torn area).

Taking the musings one step further: because it can run locally on a device like a phone, these small LMs could potentially be finetuned or inferenced across a peer-to-peer network. Making it possible to be really collaborative, and a lot cheaper to train and run.

This feels like the cusp of something big. All eyes on WWDC on June 10.

I found the tinyML community is a great resource if you wanted to dig into more of this area of research.

Enjoy the update.

Latest News

1. Microsoft unveils Phi-3 family of compact language models: Microsoft has announced the Phi-3 family of open small language models (SLMs), touting them as the most capable and cost-effective of their size available. The innovative training approach developed by Microsoft researchers has allowed the Phi-3 models to outperform larger models on language, coding, and math benchmarks. The first Phi-3 model, Phi-3-mini at 3.8 billion parameters, is now publicly available in Azure AI Model Catalog, Hugging Face, Ollama, and as an NVIDIA NIM microservice. Despite its compact size, Phi-3-mini outperforms models twice its size.

2. The new Adobe Photoshop gets an in-app image generator, major Generative Fill upgrades: Adobe unveiled the latest Photoshop in beta, which boasts an in-app image generator powered by a new Firefly image model, a more advanced Generative Fill, and other new features and tools to optimize photo editing. Adobe also introduces an impressive AI upscaling project which makes blurry videos look HD: Adobe researchers have developed a new generative AI model called VideoGigaGAN that can upscale blurry videos at up to eight times their original resolution

3. Apple releases OpenELM: small, open source AI models designed to run on-device: Just as Google, Samsung and Microsoft continue to push their efforts with generative AI on PCs and mobile devices, Apple is moving to join the party with OpenELM, a new family of open source LLMs that can run entirely on a single device rather than having to connect to cloud servers. Released on AI code community Hugging Face, OpenELM consists of small models designed to perform efficiently at text generation tasks. There are eight OpenELM models in total – four pre-trained and four instruction-tuned – covering different parameter sizes between 270 million and 3 billion parameters (referring to the connections between artificial neurons in an LLM, and more parameters typically denote greater performance and more capabilities, though not always.

4. Meta announced that multimodal capabilities are rolling out to smart glasses: Meaning it'll be able to understand visual surroundings through the built-in camera. Some use cases include translating text, identifying objects, or providing other context-specific information.

5. Nvidia’s AI Lab introduces Tesmo, a paper demonstrating a method to create interactive scenes and motion via text. For instance, you could create (potentially, interactive) 3D animations via text prompts like “man dodges oncoming traffic”.