Monitor model deployments in Azure AI Foundry Models

When you have critical applications and business processes that rely on Azure resources, you need to monitor and get alerts for your system. The Azure Monitor service collects and aggregates metrics and logs from every component of your system, including Foundry Models deployments. You can use this information to view availability, performance, and resilience, and get notifications of issues.

This document explains how you can use metrics and logs to monitor model deployments in Foundry Models.

Prerequisites

To use monitoring capabilities for model deployments in Foundry Models, you need the following:

• An Azure AI services resource. For more information, see Create an Azure AI Services resource.

  Tip

  If you are using Serverless API Endpoints and you want to take advantage of monitoring capabilities explained in this document, migrate your Serverless API Endpoints to Foundry Models.

• At least one model deployment.

• Access to diagnostic information for the resource.

Metrics

Azure Monitor collects metrics from Foundry Models automatically. No configuration is required. These metrics are:

• Stored in the Azure Monitor time-series metrics database.
• Lightweight and capable of supporting near real-time alerting.
• Used to track the performance of a resource over time.

View metrics

Azure Monitor metrics can be queried using multiple tools, including:

Azure AI Foundry portal

You can view metrics within Azure AI Foundry portal. To view them, follow these steps:

1. Go to Azure AI Foundry portal.

2. Navigate to your model deployment by selecting Deployments, and then select the name of the deployment you want to see metrics about.

3. Select the tab Metrics.

4. You can see a high level view of the most common metrics you may be interested about.

   

5. To slice, filter, or view model details about the metrics you can Open in Azure Monitor, where you have more advanced options.

   

6. Use Metrics explorer to analyze the metrics.

Metrics explorer

Metrics explorer is a tool in the Azure portal that allows you to view and analyze metrics for Azure resources. For more information, see Analyze metrics with Azure Monitor metrics explorer.

To use Azure Monitor, follow these steps:

1. Go to Azure portal.

2. Type and select Monitor on the search box.

3. Select Metrics in the left navigation bar.

4. On Select scope, select the resources you want to monitor. You can select either one resource or select a resource group or subscription. If that's the case, ensure you select Resource types as Azure AI Services.

5. The metric explorer shows. Select the metrics that you want to explore. The following example shows the number of requests made to the model deployments in the resource.

   

   Important

   Metrics in category Azure OpenAI contain metrics for Azure OpenAI models in the resource. Category Models contains all the models available in the resource, including Azure OpenAI, DeepSeek, Phi, etc. We recommend switching to this new set of metrics.

6. You can add as many metrics as needed to either the same chart or to a new chart.

7. If you need, you can filter metrics by any of the available dimensions of it.

   

8. It is useful to break down specific metrics by some of the dimensions. The following example shows how to break down the number of requests made to the resource by model by using the option Add splitting:

   

9. You can save your dashboards at any time to avoid having to configure them each time.

Kusto query language (KQL)

If you configure diagnostic settings to send metrics to Log Analytics, you can use the Azure portal to query and analyze log data by using the Kusto query language (KQL).

To query metrics, follow these steps:

1. Ensure you have configure diagnostic settings.

2. Go to Azure portal.

3. Locate the Azure AI Services resource you want to query.

4. In the left navigation bar, navigate to Monitoring > Logs.

5. Select the Log Analytics workspace that you configured with diagnostics.

6. From the Log Analytics workspace page, under Overview on the left pane, select Logs. The Azure portal displays a Queries window with sample queries and suggestions by default. You can close this window.

7. To examine the Azure Metrics, use the table AzureMetrics for your resource, and run the following query:

   AzureMetrics
   | take 100
   | project TimeGenerated, MetricName, Total, Count, Maximum, Minimum, Average, TimeGrain, UnitName
   

   Note

   When you select Monitoring > Logs in the menu for your resource, Log Analytics opens with the query scope set to the current resource. The visible log queries include data from that specific resource only. To run a query that includes data from other resources or data from other Azure services, select Logs from the Azure Monitor menu in the Azure portal. For more information, see Log query scope and time range in Azure Monitor Log Analytics for details.

Other tools

Tools that allow more complex visualization include:

• Workbooks, customizable reports that you can create in the Azure portal. Workbooks can include text, metrics, and log queries.
• Grafana, an open platform tool that excels in operational dashboards. You can use Grafana to create dashboards that include data from multiple sources other than Azure Monitor.
• Power BI, a business analytics service that provides interactive visualizations across various data sources. You can configure Power BI to automatically import log data from Azure Monitor to take advantage of these visualizations.

Metrics reference

The following categories of metrics are available:

Models - Requests

Models - Latency

Models - Usage

Logs

Resource logs provide insight into operations that were done by an Azure resource. Logs are generated automatically, but you must route them to Azure Monitor logs to save or query by configuring a diagnostic setting. Logs are organized in categories when you create a diagnostic setting, you specify which categories of logs to collect.

Configure diagnostic settings

All of the metrics are exportable with diagnostic settings in Azure Monitor. To analyze logs and metrics data with Azure Monitor Log Analytics queries, you need to configure diagnostic settings for your Azure AI Services resource. You need to perform this operation on each resource.

There's a cost for collecting data in a Log Analytics workspace, so only collect the categories you require for each service. The data volume for resource logs varies significantly between services.