Back from holiday with my batteries charged 100%. Time to continue our journey in the Microsoft.Extensions.AI library. Today we have a look at (Open)Telemetry integration.
This post is part of a blog series. Other posts so far:
- Part I – An introduction to Microsoft.Extensions.AI
- Part II – ASP.NET Core integration
- Part III –Tool calling
- Part IV – Telemetry integration (this post)
Sooner or later you’ll arrive at a moment where you want to better understand what is going on in the interaction between your chat client and the LLM. That is the moment you want to integrate telemetry in your application.
In the Microsoft.Extensions.AI library, this can be done through the the OpenTelemetryChatClient. You can plug this client in by calling the UseOpenTelemetry method on the ChatClientBuilder:
If we now run our application and take a look at the OpenTelemetry data in our Aspire dashboard, we get a lot of useful information on what is going on behind the scenes:
The ChatClient pipeline model
By now you’ve maybe noticed that the ChatClient is in fact composed as a pipeline where every component can add additional functionality. In our example above, we added 2 extra layers of functionality; function invocation and OpenTelemetry tracing:
This means that you can use the same approach to add your own functionality. An example can be found in the documentation where a RateLimitingChatClient is created:
More information
Semantic conventions for generative AI systems | OpenTelemetry