Edit

Share via


Diagnostic settings in Azure Monitor

Diagnostic settings in Azure Monitor allow you to collect resource logs and to send platform metrics and the activity log to different destinations. Create a separate diagnostic setting for each resource you want to collect data from. Each setting defines the data from the resource to collect and the destinations to send that data to. This article describes the details of diagnostic settings, including how to create them and the destinations available for sending data.

Diagram showing collection of activity logs, resource logs, and platform metrics.

The following video walks through routing resource platform logs with diagnostic settings. The following changes were made to diagnostic settings since the video was recorded, but these topics are discussed in this article.

Sources

Diagnostic settings can collect data from the sources in the following table. See each linked article for details on the data collected by that source and its format in each destination.

Data source Description
Platform metrics Automatically collected without configuration. Use a diagnostic setting to sent platform metrics to other destinations.
Activity log Automatically collected without configuration. Use a diagnostic setting to sent activity log entries to other destinations.
Resource logs Aren't collected by default. Create a diagnostic setting to collect resource logs.

Destinations

Diagnostic settings send data to the destinations in the following table. To ensure the security of data in transit, all destination endpoints are configured to support TLS 1.2.

A single diagnostic setting can define no more than one of each of the destinations. If you want to send data to more than one of a particular destination type (for example, two different Log Analytics workspaces), create multiple settings. Each resource can have up to five diagnostic settings.

Any destinations used by the diagnostic setting must exist before the setting can be created. The destination doesn't have to be in the same subscription as the resource sending logs if the user who configures the setting has appropriate Azure role-based access control access to both subscriptions. Use Azure Lighthouse to include destinations in another Microsoft Entra tenant.

Destination Description Requirements
Log Analytics workspace Retrieve data using log queries and workbooks. Use log alerts to proactively alert on data. See Azure Monitor Resource log reference for the tables used by different Azure resources. Any tables in a Log Analytics workspace are created automatically when the first data is sent to the workspace, so only the workspace itself must exist.
Azure Storage account Store for audit, static analysis, or back up. Storage may be less expensive than other options and can be kept indefinitely. Send data to immutable storage to prevent its modification. Set the immutable policy for the storage account as described in Set and manage immutability policies for Azure Blob Storage. Storage accounts must be in the same region as the resource being monitored if the resource is regional.

Diagnostic settings can't access storage accounts when virtual networks are enabled. You must enable Allow trusted Microsoft services to bypass this firewall setting in storage accounts so that the Azure Monitor diagnostic settings service is granted access to your storage account.

Azure DNS zone endpoints (preview) and any Premium storage accounts aren't supported as a destination. Any Standard storage accounts are supported.
Azure Event Hubs Stream data to external systems such as third-party SIEMs and other Log Analytics solutions. Event hubs must be in the same region as the resource being monitored if the resource is regional.

Diagnostic settings can't access event hubs when virtual networks are enabled. You must enable Allow trusted Microsoft services to bypass this firewall setting in storage accounts so that the Azure Monitor diagnostic settings service is granted access to your storage account.

The shared access policy for event hub namespace defines the permissions that the streaming mechanism has. Streaming to Event Hubs requires Manage, Send, and Listen permissions. To update the diagnostic setting to include streaming, you must have the ListKey permission on that Event Hubs authorization rule.
Azure Monitor partner solutions Specialized integrations can be made between Azure Monitor and other non-Microsoft monitoring platforms. The solutions vary by partner. See Azure Native ISV Services documentation for details.

Create a diagnostic setting

You can create a diagnostic setting using any of the following methods.

Use the following steps to create a new diagnostic setting or edit an existing one in the Azure portal.

  1. Where you configure diagnostic settings in the Azure portal depends on the resource:

    • For a single resource, select Diagnostic settings under Monitoring on the resource's menu.

      Screenshot that shows the Monitoring section of a resource menu in the Azure portal with Diagnostic settings highlighted.

    • For one or more resources, select Diagnostic settings under Settings on the Azure Monitor menu and then select the resource.

      Screenshot that shows the Settings section in the Azure Monitor menu with Diagnostic settings highlighted.

    • For the activity log, select Activity log on the Azure Monitor menu and then select Export Activity Logs. Make sure you disable any legacy configuration for the activity log.

      Screenshot that shows the Azure Monitor menu with Activity log selected and Export activity logs highlighted in the Monitor-Activity log menu bar.

  2. Select Add diagnostic setting to add a new setting or Edit setting to edit an existing one. You may need multiple diagnostic settings for a resource if you want to send to multiple destinations of the same type.

    Screenshot that shows adding a diagnostic setting for existing settings.

  3. Give your setting a descriptive name if it doesn't already have one.

    Screenshot that shows Diagnostic setting details.

  4. Logs and metrics to route: For logs, either choose a category group or select the individual checkboxes for each category of data you want to send to the destinations specified later. The list of categories varies for each Azure service. Select AllMetrics if you want to collect platform metrics.

  5. Destination details: Select the checkbox for each destination that should be included in the diagnostic settings and then provide the details for each. If you select Log Analytics workspace as a destination, then you may need to specify the collection mode. See Collection mode for details.

Category groups

You can use category groups to collect resource logs based on predefined groupings instead of selecting individual log categories. Microsoft defines the groupings to help monitor common use cases. If the categories in the group are updated, your log collection is modified automatically. Not all Azure services use category groups. If category groups aren't available for a particular resource, then the option won't be available when creating the diagnostic setting.

If you do use category groups in a diagnostic setting, you can't select individual category types. Currently, there are two category groups:

  • allLogs: all categories for the resource.
  • audit: All resource logs that record customer interactions with data or the settings of the service. You don't need to select this category group if you select the allLogs category group.

Note

Enabling the Audit category in the diagnostic settings for Azure SQL Database does not activate auditing for the database. To enable database auditing, you have to enable it from the auditing blade for Azure Database.

Metrics limitations

Not all metrics can be sent to a Log Analytics workspace with diagnostic settings. See the Exportable column in the list of supported metrics.

Diagnostic settings don't currently support multi-dimensional metrics. Metrics with dimensions are exported as flattened single-dimensional metrics and aggregated across dimension values. For example, the IOReadBytes metric on a blockchain can be explored and charted on a per-node level. When exported with diagnostic settings, the metric exported shows all read bytes for all nodes.

To work around the limitations for specific metrics, you can manually extract them by using the Metrics REST API and then import them into a Log Analytics workspace with the Logs ingestion API.

Deleting diagnostic settings

Delete any diagnostic settings for a resource if you delete or rename that resource, or migrate it across resource groups or subscriptions. If you recreate this resource, any diagnostic settings for the deleted resource could be applied to the new one. This resumes the collection of resource logs as defined in the diagnostic setting.

Controlling costs

There may be a cost for data collected by diagnostic settings. The cost depends on the destination you choose and the volume of data collected. For more information, see Azure Monitor pricing.

Only collect the categories you require for each service. You might also not want to collect platform metrics from Azure resources because this data is already being collected in Metrics. Only configure your diagnostic data to collect metrics if you need metric data in the workspace for more complex analysis with log queries.

Diagnostic settings don't allow granular filtering within a selected category. You can filter data for supported tables in a Log Analytics workspace using transformations. See Transformations in Azure Monitor for details.

Troubleshooting

Metric category isn't supported
You may receive an error message similar to Metric category 'xxxx' is not supported when using a Resource Manager template, REST API, Azure CLI, or Azure PowerShell. Metric categories other than AllMetrics aren't supported except for a limited number of Azure services. Remove any metric category names other than AllMetrics and repeat your deployment.

Setting disappears due to non-ASCII characters in resourceID
Diagnostic settings don't support resource IDs with non-ASCII characters (for example, Preproduccón). Since you can't rename resources in Azure, you must create a new resource without the non-ASCII characters. If the characters are in a resource group, you can move the resources to a new group.

Inactive resources
When a resource is inactive and exporting zero-value metrics, the diagnostic settings export mechanism backs off incrementally to avoid unnecessary costs of exporting and storing zero values. This back-off may lead to a delay in the export of the next non-zero value. This behavior only applies to exported metrics and doesn't affect metrics-based alerts or autoscale.

When a resource is inactive for one hour, the export mechanism backs off to 15 minutes. This means that there is a potential latency of up to 15 minutes for the next nonzero value to be exported. The maximum backoff time of two hours is reached after seven days of inactivity. Once the resource starts exporting nonzero values, the export mechanism reverts to the original export latency of three minutes.

Duplicate data for Application Insights
Diagnostic settings for workspace-based Application insights applications collect the same data as Application insights itself. This results in duplicate data being collected if the destination is the same Log Analytics workspace that the application is using. Create a diagnostic setting for Application insights to send data to a different Log Analytics workspace or another destination.

Next steps