Tracing agents

Determining the reasoning behind your agent's executions is important for troubleshooting and debugging. However, it can be difficult for complex agents for a number of reasons:

• There could be a high number of steps involved in generating a response, making it hard to keep track of all of them.
• The sequence of steps might vary based on user input.
• The inputs/outputs at each stage might be long and deserve more detailed inspection.
• Each step of an agent's runtime might also involve nesting. For example, an agent might invoke a tool, which uses another process, which then invokes another tool. If you notice strange or incorrect output from a top-level agent run, it might be difficult to determine exactly where in the execution the issue was introduced.

Tracing solves this by allowing you to clearly see the inputs and outputs of each primitive involved in a particular agent run, in the order in which they were invoked.

Tracing in the Azure AI Foundry Agents playground

The Agents playground in the Azure AI Foundry portal lets you trace threads and runs that your agents produce. To open a trace, select Thread info in an active thread. You can also optionally select Metrics to enable automatic evaluations of the model's performance across several dimensions of AI quality and Risk and safety.

After selecting Thread info, the screen that appears will be let you view the: thread, run, run steps and any tool calls that were made. You can view the inputs and outputs between the agent and user, as well the associated metadata and any evaluations you selected.

Trace agents using OpenTelemetry and an Application Insights resource

Tracing lets you analyze your agent's performance and behavior by using OpenTelemetry and adding an Application Insights resource to your Azure AI Foundry project.

To add an Application Insights resource, select Observability from the left pane in the Azure AI Foundry portal, select Tracing, and then and create a new resource if you don't already have one.

Once created, you can get an Application Insights connection string, configure your agents, and observe the full execution path of your agent through Azure Monitor. Typically you want to enable tracing before you create an agent.

Observe an agent

First, use pip install to install OpenTelemetry and the Azure SDK tracing plugin.

pip install opentelemetry-sdk
pip install azure-core-tracing-opentelemetry

You will also need an exporter to send results to your observability backend. You can print traces to the console or use a local viewer such as Aspire Dashboard. To connect to Aspire Dashboard or another OpenTelemetry compatible backend, install the OpenTelemetry Protocol (OTLP) exporter.

pip install opentelemetry-exporter-otlp

Once you have the packages installed, you can use one the following Python samples to implement tracing with your agents. Samples that use console tracing display the results locally in the console. Samples that use Azure Monitor send the traces to the Azure Monitor in the Azure AI Foundry portal, in the Observability tab in the left pane for the portal.

Using Azure Monitor

• Basic agent example
• Agent example with function calling
• Example with a stream event handler

Using console tracing

• Basic agent example
• Agent example with function calling
• Example with a stream event handler