If you've built anything with the Microsoft Agent Framework (MAF), you'll notice that your tool calls can fill up the context window quite fast. A function that returns a list of 50 orders as JSON easily costs you a few hundred tokens on braces, quotes and repeated field names alone. That cost hits you twice: once on the way into the model as tool output, and again on every subsequent turn where that history gets replayed.
That's where TOON (Token-Oriented Object Notation) comes in. Let's explore this.
What TOON actually is
TOON is a line-oriented, indentation-based encoding of the same data model JSON uses. It borrows YAML's indentation for nested objects and CSV's tabular layout for arrays of uniform objects. The trick is in that last part: if you have a list of objects that all share the same fields, TOON declares the field list once and then just streams the values, row by row, instead of repeating every key for every item.
Take a small object like this in JSON:
{
"orders": [
{ "id": 1, "customer": "Acme", "status": "open" },
{ "id": 2, "customer": "Globex", "status": "shipped" },
{ "id": 3, "customer": "Initech", "status": "open" }
]
}
In TOON that becomes:
orders[3]{id,customer,status}:
1,Acme,open
2,Globex,shipped
3,Initech,open
No repeated keys, no braces, no quotes around every string. The [3] tells the parser how many rows to expect and {id,customer,status} is the header, declared once. For uniform tabular data, this is where TOON earns its name: the published benchmarks from the TOON project show 30-60% fewer tokens compared to the equivalent JSON, depending on the tokenizer and how uniform your data is.
Remark: TOON is explicitly designed as an input format for LLMs, not a general-purpose replacement for JSON. You still want JSON (or your own DTOs) everywhere else in your application. TOON only earns its place at the boundary where structured data crosses into a prompt.
How to use this in a MAF agent
In the Microsoft Agent Framework, a tool is just a C# method wrapped with AIFunctionFactory.Create(). MAF takes care of generating the JSON schema for the parameters and, when the LLM calls the tool, feeding whatever your method returns straight back into the conversation as the tool result. By default that "whatever you return" gets serialized to JSON.
So the question becomes: what if the method just returns TOON instead of a plain object or a JSON string?
Turns out, that's easy to achieve. MAF doesn't care what your tool returns as long as it's a string (or something serializable). It doesn't force JSON on you at the tool-result boundary. You control the encoding yourself, inside the method body.
Putting it into action
I used the Toon.NET package here, but any of the TOON encoders on NuGet will do the same job. They all expose a similar Serialize() call that takes your object graph.
dotnet add package Toon.NET
A tool that looks up open orders for a customer, naively, looks like this:
[Description("Get the open orders for a given customer id")]
static List<Orders> GetOpenOrders(string customerId)
{
return orderRepository.GetOpenOrders(customerId);
// MAF will JSON-serialize this for the tool result
}
Swap the return type to TOON:
using Toon;
[Description("Get the open orders for a given customer id")]
static string GetOpenOrders(string customerId)
{
var orders = orderRepository.GetOpenOrders(customerId);
var serializer = new ToonSerializer();
return serializer.Serialize(new { orders });
}
And wire it up on the agent as you normally would:
AIAgent orderAgent = chatClient.AsAIAgent(
instructions: "You help customer service reps look up order status. " +
"Tool results are TOON-encoded: a compact, indented, comma-separated format.",
name: "OrderAgent",
tools: [AIFunctionFactory.Create(GetOpenOrders)]
);
Console.WriteLine(await orderAgent.RunAsync("Does Acme have any open orders?"));
That's it. The LLM sees a TOON block instead of a JSON blob for the tool result, and everything downstream — the reasoning, the follow-up questions, the final answer — works exactly the same, just cheaper.
Tip: Mention the format explicitly in your instructions, like in the example above. Modern models handle TOON fine without being told, since it reads close to YAML/CSV, but a one-line heads-up removes any ambiguity about how to interpret the header row.
Does the schema still work?
Yes! Nothing changes for the input side. AIFunctionFactory.Create() still inspects your method signature and generates a normal JSON Schema for the parameters, because that schema is metadata the model needs to know how to call the tool in the first place. TOON only replaces the output — the tool result that gets appended to the conversation after invocation. Two different concerns, two different formats, and that's exactly why you can mix them without MAF complaining.
When it's not worth it
TOON's savings come from tabular uniformity. If your tool returns a single object, or a small non-uniform blob, the TOON encoding barely differs from compact JSON, and you're adding a dependency for no real gain.
Remark: I'd only reach for this on tools that return lists; order histories, search results, log entries, anything array-of-objects shaped. For a single customer record or a boolean flag, plain JSON is fine and simpler to debug.
The other trade-off: TOON encoders on NuGet right now are mostly encode-only, or young libraries with evolving APIs (MAF itself is still in public preview, so this is a preview-on-preview combination). Pin your package versions and re-check the docs before you ship this into production.
That's it! A small change at the tool-result boundary, and every follow-up turn in the conversation carries a lighter payload.
More information
- TOON specification: github.com/toon-format/spec
- TOON reference implementation and benchmarks: github.com/toon-format/toon
- Microsoft Agent Framework: github.com/microsoft/agent-framework
- Toon.NET on NuGet: nuget.org/packages/Toon.NET