Phi-3 is a family of open source small language models developed and made available by Microsoft.
“Small language models are designed to perform well for simpler tasks, are more accessible and easier to use for organizations with limited resources, and they can be more easily fine-tuned to meet specific needs. They are well suited for applications that need to run locally on a device, where a task doesn’t require extensive reasoning and a quick response is needed,” Misha Bilenko, corporate vice president for Microsoft GenAI, wrote in a blog post.
The idea behind developing a model so small was inspired by Microsoft researcher Ronan Elden reading a bedtime story to his daughter, which led him to think “how did she learn this word? How does she know how to connect these words?”
Applying this to AI, Elden wondered what would happen if an AI model was trained just on words that would be understood by a 4-year-old.
Phi-3 comes in a variety of options:
- Phi-3-vision is a 4.2B parameter model that capable of understanding both text and vision
- Phi-3-mini is a 3.8B parameter model, available in 128K and 4K context length options
- Phi-3-small is a 7B parameter model, available in 128K and 4K context length options
- Phi-3-medium is a 14B parameter model, available in 128K and 4K context length options
Phi-3-vision is the first multimodal model in the family, and can generate insights from charts and diagrams. “Phi-3-vision builds on the language capabilities of the Phi-3-mini, continuing to pack strong language and image reasoning quality in a small model,” Bilenko wrote.
According to Microsoft, compared to other models, Phi-3 performs well. For example, Phi-3-small beats GPT-3.5T across a variety of language, reasoning, coding, and math benchmarks, while Phi-3-medium beats out Gemini 1.0 Pro. Additionally, Phi-3-vision outperforms Claude-3 Haiku and Gemini 1.0 Pro V in general visual reasoning tasks, OCR, table, and chart understanding tasks.
All of the Phi-3 models are currently available on Azure AI and Hugging Face.