Meet the Phi family

The Phi family keeps growing with the recent introduction of the Phi-4-Reasoning, Phi-4-mini and Phi-4-multimodal models. These new models introduce some great enhancements we'll discuss later in this post.

But maybe let me start by introducing you to the Phi family.

The Phi family

The Phi family is a set of small language models created and maintained by Microsoft Research. They are designed to achieve strong performance while being much smaller and more efficient than their larger counterparts like GPT-4 or Claude. Phi’s evolution over the last year has continually pushed this envelope of quality vs. size, expanding the family with new features to address diverse needs.

Where the original Phi models focussed mostly on chat and coding activitities, capabilities have now extended to multimodel (vision, speech), function calling and advanced reasoning.

Function calling support

A feature I’m especially happy with that it finally got introduced into the Phi models is function calling. This allows the LLM to interact with external tools and APIs which is especially interesting with the growing ecosystem of MCP servers.

I first downloaded the phi4-mini model

ollama pull phi4-mini

Then I updated my Semantic Kernel MCP example to try the new model in Ollama:

Unfortunately my MCP server wasn’t invoke. Instead the model returned me a response like this:

I'm sorry for any confusion, but as an AI developed by Microsoft with text-based capabilities only, I don't have direct access to external systems or URLs like your local development server. However, if you want me to simulate opening a browser window and navigating to that specific email client URL using Playwright in code form (for example), please let me know the details of what you'd need help simulating.

If you're looking for an actual interaction with this service on your machine or another system where I can assist further within my capabilities, you would typically use tools like a web browser and navigate to that address manually. If there's anything else text-based assistance is needed in relation to Playwright tests (like writing test code), feel free to ask!

Good that the model feels sorry. But that was not what I was hoping for…

Some research online brought me to the following Github issue; Tool calls not returning properly with phi4-mini:3.8b · Issue #9437 · ollama/ollama. This issue was also mentioned in another blog post about the Phi4-mini model; Building AI Agents on edge devices using Ollama + Phi-4-mini Function Calling | Microsoft Community Hub.

Both mention that a possible solution would be to:

1. Use the 3.8b-fp16 variant of the model (I was using the default 3.8b variant)
          
2. Update the model file to use an altered prompt:

We create a new Ollama model based on this altered model file and the downloaded 3.8b-fp16 model:

We first export the original model file:

ollama show phi4-mini:3.8b-fp16 --modelfile > phi4customized.modelfile

Now we update the model  file with the altered prompt and create a new model:

ollama create phi4-mini:functioncalling –f phi4customized.modelfile

We update our code to use the new model file and give it a new try:

Unfortunately it failed again:

To open a new tab in your default web browser with the specified local address, you can use one or more Playwright functions. However, since I don't have direct access to an external system's capabilities like opening URLs on my own and assuming that "https://localhost:7252/" is meant for demonstration purposes (as localhost typically refers to port 80), here are two steps using the available function:

1. Open a new tab.
2. Navigate to the specified URL.

Let’s switch to a simple plugin instead of the MCP integration. Maybe that works better:

Yes! I finally got the tool calling working locally. However, I don't feel very confident that this will always work. So don't try this at home... (yet).

What about the Phi4 multi-modal?

At the moment of writing this post, Ollama doesn’t support multi-modal models. If you know a good solution to run a multimodal model locally, feel free to contact me. I couldn’t find a good solution (yet).

More information

Welcome to the new Phi-4 models - Microsoft Phi-4-mini & Phi-4-multimodal

Building AI Agents on edge devices using Ollama + Phi-4-mini Function Calling | Microsoft Community Hub

One year of Phi: Small language models making big leaps in AI | Microsoft Azure Blog

Make Phi-4-mini-reasoning more powerful with industry reasoning on edge devices | Microsoft Community Hub

Showcasing Phi-4-Reasoning: A Game-Changer for AI Developers | Microsoft Community Hub

microsoft/PhiCookBook: This is a Phi Family of SLMs book for getting started with Phi Models. Phi a family of open sourced AI models developed by Microsoft. Phi models are the most capable and cost-effective small language models (SLMs) available, outperforming models of the same size and next size up across a variety of language, reasoning, coding, and math benchmarks

Tweak your LLM models with Ollama

PhiCookBook/md/02.Application/07.FunctionCalling/Phi4/Ollama/ollama_functioncalling.ipynb at main · microsoft/PhiCookBook