Usage analysis with Application Insights

Application Insights is a powerful observability tool that collects telemetry data to show how users interact with your application. This includes information about which features are most popular, if users achieve their goals, where they drop off, and whether they return later.

These insights help you understand user behavior, identify areas for improvement, and measure the impact of recent changes, allowing you to make data-driven decisions about your next development cycles.

This article covers the following areas:

• Usage analysis with custom events

• Native usage experiences

  • Users, Sessions & Events - Track and analyze user interaction with your application, session trends, and specific events to gain insights into user behavior and application performance.

  • Funnels - Understand how users progress through a series of steps in your application, and where they might be dropping off.

  • User Flows - Visualize user paths to identify the most common routes and areas where users are most engaged or encounter issues.

  • Cohorts - Group users or events by common characteristics to analyze behavior patterns, feature usage, and the impact of changes over time.

• Usage workbook templates

  • User Retention Analysis - Track the frequency and patterns of users returning to your application and their interactions with specific features.

  • User Impact Analysis - Analyze how application performance metrics (for example, load times) influence user experience and behavior, to help you to prioritize improvements.

  • HEART Analysis - Utilize the HEART framework to measure and understand user happiness, engagement, adoption, retention, and task success.

How to get started

Prerequisites

Instrument your application

To collect browser telemetry about the usage of your application, use the Application Insights JavaScript SDK. No server-side instrumentation is required.

To verify if browser telemetry is being collected, run your project in debug mode for a few minutes, then look for results in the Overview pane in Application Insights.

Usage analysis with custom events

Track user interactions with custom events

Use custom events to track important actions that support business goals. Examples include selecting a button, submitting a form, and completing a purchase.

While page views can sometimes represent useful events, they aren't always reliable indicators. For example, a user might open a product page without making a purchase. By tracking specific business events, you can chart users' progress through your site, understand their preferences for different options, and identify where they encounter difficulties or drop out.

Combine custom events with user IDs and session context to enable:

• Tracking behavior across sessions.
• Analyzing conversion funnels based on user actions.
• Segmenting users by how they interact with your app.

Attaching property values to these events allows you to filter or split them during inspection in the portal. Each event also includes a standard set of properties, such as an anonymous user ID, allowing you to trace the sequence of activities of individual users.

How to log custom events

Events can be logged from the client side of the application using either the Click Analytics Autocollection plug-in or trackEvent:

appInsights.trackEvent({name: "incrementCount"});

You can also log server-side custom events using the Azure Monitor OpenTelemetry Distro. For more information, see Add and modify Azure Monitor OpenTelemetry for .NET, Java, Node.js, and Python applications.

To learn how to use custom events with the Application Insights SDK (Classic API), see custom events and properties.

Slice and dice custom events

In the Users, Sessions, and Events tools, you can slice and dice custom events by user, event name, and properties. Whenever you're in any usage experience, select the Open the last run query icon to take you back to the underlying query.

You can then modify the underlying query to get the specific information you're looking for. Here's an example of an underlying query about page views.

// average pageView duration by name
let timeGrain=5m;
let dataset=pageViews
// additional filters can be applied here
| where timestamp > ago(1d)
| where client_Type == "Browser" ;
// calculate average pageView duration for all pageViews
dataset
| summarize avg(duration) by bin(timestamp, timeGrain)
| extend pageView='Overall'
// render result in a chart
| render timechart

Native usage experiences

Users, Sessions, and Events

Three of the Usage panes use the same tool to slice and dice telemetry from your application from three perspectives. By filtering and splitting the data, you can uncover insights about the relative use of different pages and features. Find out when people use your application, what pages they're most interested in, where your users are located, and what browsers and operating systems they use.

• Users tool: Counts the numbers of unique users that access your pages within your chosen time periods. Users are counted by using anonymous IDs stored in browser cookies. A single person using different browsers or machines is counted as more than one user.

• Sessions tool: Tabulates the number of user sessions that access your site. A session represents a period of activity initiated by a user and concludes with a period of inactivity exceeding half an hour or after 24 hours of continuous use.

• Events tool: How often are certain pages and features of your application used? A page view is counted when a browser loads a page from your app, provided you instrumented it.

  A custom event represents one occurrence of something happening in your application. It's often a user interaction like a button selection or the completion of a task. You insert code in your application to generate custom events or use the Click Analytics extension.

Query for certain users, sessions, or events

Explore different groups of users, sessions, or events by adjusting the query options at the top of each pane.

Clicking View More Insights displays the following information:

Determine feature success with A/B testing

If you're unsure which feature variant is more successful, run an A/B test and let different users access each variant.

To set up an A/B test, attach unique property values to all the telemetry sent by each variant. With OpenTelemetry, this can be done by adding a custom property to a span. For more information, see Add and modify Azure Monitor OpenTelemetry for .NET, Java, Node.js, and Python applications.

If you're using the Application Insights SDK (Classic API), use a telemetry initializer instead. For more information, see custom events.

After the A/B test, filter and split your data on the property values so that you can compare the different versions. Measure each version's success, then transition to a unified version.

Funnels

Understanding the customer experience is of great importance to your business. If your application involves multiple stages, you need to know if customers are progressing through the entire process or ending the process at some point. The progression through a series of steps in an application is known as a funnel.

You can use Application Insights funnels to gain insights into your users and monitor step-by-step conversion rates. Selecting a step shows additional step-specific details.

Create a funnel

Before you create a funnel, decide on the question you want to answer. For example, you might want to know how many users view the home page, view a customer profile, and create a ticket.

1. On the Funnels tab, select Edit.

2. Choose your Top Step.

   

3. To apply filters to the step, select Add filters. This option appears after you choose an item for the top step.

4. Then choose your Second Step and so on.

   Note

   Funnels are limited to a maximum of six steps.

5. Select the View tab to see your funnel results.

6. To save your funnel to view at another time, select Save at the top. Use Open to open your saved funnels.

User Flows

The User Flows tool visualizes how users move between the pages and features of your site. It's great for answering questions like:

• How do users move away from a page on your site?
• What do users select on a page on your site?
• Where are the places that users churn most from your site?
• Are there places where users repeat the same action over and over?

The User Flows tool starts from an initial custom event, exception, dependency, page view, or request that you specify. From this initial event, User Flows shows the events that happened before and after user sessions. Lines of varying thickness show how many times users followed each path.

Special Session Started nodes show where the subsequent nodes began a session. Session Ended nodes show how many users sent no page views or custom events after the preceding node, highlighting where users probably left your site.

Create a user flow visualization

To begin answering questions with the User Flows tool, choose an initial custom event, exception, dependency, page view, or request to serve as the starting point for the visualization:

1. On the User Flows pane, select Edit or Select an event.

2. From the Initial event dropdown list, select a custom event, exception, dependency, page view, or request.

   

3. Select Create graph.

The Step 1 column of the visualization shows what users did most frequently after the initial event. The items are ordered from top to bottom and from most to least frequent. The Step 2 and subsequent columns show what users did next. The information creates a picture of all the ways that users moved through your site.

Edit a user flow visualization

By default, the User Flows tool randomly samples only the last 24 hours of page views and custom events from your site. You can increase the time range and change the balance of performance and accuracy for random sampling on the Edit menu.

If some of the page views, custom events, and exceptions aren't relevant to you, select X on the nodes you want to hide. After selecting the nodes you want to hide, select Create graph. To see all hidden nodes, select Edit and look at the Excluded events section.

If page views or custom events you expect to see in the visualization are missing that:

• Check the Excluded events section on the Edit menu.
• Use the plus buttons on Others nodes to include less-frequent events in the visualization.
• If the page view or custom event you expect is sent infrequently by users, increase the time range of the visualization on the Edit menu.
• Make sure the custom event, exception, dependency, page view, or request you expect is set up to be collected by the Application Insights SDK in the source code of your site.

If you want to see more steps in the visualization, use the Previous steps and Next steps dropdown lists above the visualization.

Example questions you can answer with user flows

Select one of the following examples to expand the section.

Cohorts

A cohort is a set of users, sessions, events, or operations that have something in common. In Application Insights, cohorts are defined by an analytics query. In cases where you have to analyze a specific set of users or events repeatedly, cohorts can give you more flexibility to express exactly the set you're interested in.

Cohorts vs basic filters

You can use cohorts in ways similar to filters. But cohorts' definitions are built from custom analytics queries, so they're much more adaptable and complex. Unlike filters, you can save cohorts so that other members of your team can reuse them.

You might define a cohort of users who have all tried a new feature in your app. You can save this cohort in your Application Insights resource. It's easy to analyze this saved group of specific users in the future.

Create a cohort

Your team defines an engaged user as anyone who uses your application five or more times in a given month. In this section, you define a cohort of these engaged users.

1. Select Create a Cohort.

2. Select the Template Gallery tab to see a collection of templates for various cohorts.

3. Select Engaged Users -- by Days Used.

   There are three parameters for this cohort:

   • Activities: Where you choose which events and page views count as usage.
   • Period: The definition of a month.
   • UsedAtLeastCustom: The number of times users need to use something within a period to count as engaged.

4. Change UsedAtLeastCustom to 5+ days. Leave Period set as the default of 28 days.

   Now this cohort represents all user IDs sent with any custom event or page view on five separate days in the past 28 days.

5. Select Save.

   Tip

   Give your cohort a name, like Engaged Users (5+ Days). Save it to My reports or Shared reports, depending on whether you want other people who have access to this Application Insights resource to see this cohort.

6. Select Back to Gallery.

What can you do by using this cohort?

Open the Users tool. In the Show dropdown box, choose the cohort you created under Users who belong to.

Important points to notice:

• You can't create this set through normal filters. The date logic is more advanced.

• You can further filter this cohort by using the normal filters in the Users tool. Although the cohort is defined on 28-day windows, you can still adjust the time range in the Users tool to be 30, 60, or 90 days.

These filters support more sophisticated questions that are impossible to express through the query builder. An example is people who were engaged in the past 28 days. How did those same people behave over the past 60 days?

More cohort examples

Select one of the following examples to expand the section.

Usage workbook templates

User Retention Analysis

The User Retention Analysis workbook helps you understand user engagement by tracking how often users return to your application and interact with specific features. It reveals patterns across user cohorts, such as differences in return rates between users who win or lose a game, offering actionable insights to improve user experience and guide business decisions.

By analyzing user cohorts based on their actions within a given timeframe, you can:

• Understand what specific features cause users to come back more than others.
• Detect potential retention issues.
• Form data-driven hypotheses to help you improve the user experience and your product strategy.

Use the User Retention Analysis workbook

To access the workbook, go to the Workbooks pane in Application Insights and select User Retention Analysis under the Usage category.

The visualizations include:

Overall Retention: A summary chart of the user retention percentage over the selected time frame.

Retention Grid: Displays the number of users retained. Each row represents a cohort of users who performed any event in the time period shown. Each cell in the row shows how many of that cohort returned at least once in a later period. Some users might return in more than one period.

Insights Cards: Highlight the top five initiating and returning events to help pinpoint key engagement drivers.

Use the retention controls at the top of the workbook to:

• Define a specific time range.
• Select different combinations of events to narrow the focus on specific user activities.
• Add filters on properties, for example, to focus on users in a particular country or region.

User Impact Analysis

Impact Analysis discovers how any dimension of a page view, custom event, or request affects the usage of a different page view or custom event.

One way to think of Impact is as the ultimate tool for settling arguments with someone on your team about how slowness in some aspect of your site is affecting whether users stick around. Users might tolerate some slowness, but Impact gives you insight into how best to balance optimization and performance to maximize user conversion.

Analyzing performance is only a subset of Impact's capabilities. Impact supports custom events and dimensions, so you can easily answer questions like, How does user browser choice correlate with different rates of conversion?

The User Impact Analysis workbook

To use the User Impact Analysis workbook in Application Insights, navigate to the Workbooks pane and locate it listed under the Usage category.

1. From the Selected event dropdown list, select an event.
2. From the analyze how its dropdown list, select a metric.
3. From the Impacting event dropdown list, select an event.
4. To add a filter, use the Add selected event filters tab or the Add impacting event filters tab.

How does the User Impact Analysis workbook calculate conversion rates?

Under the hood, the User Impact Analysis workbook relies on the Pearson correlation coefficient. Results are computed between -1 and 1. The coefficient -1 represents a negative linear correlation and 1 represents a positive linear correlation.

The basic breakdown of how user impact analysis works:

• Let A = the main page view, custom event, or request you select in the Selected event dropdown list.
• Let B = the secondary page view or custom event you select in the impacts the usage of dropdown list.

Impact looks at a sample of all the sessions from users in the selected time range. For each session, it looks for each occurrence of A.

Sessions are then broken into two different kinds of subsessions based on one of two conditions:

• A converted subsession consists of a session ending with a B event and encompasses all A events that occur before B.
• An unconverted subsession occurs when all As occur without a terminal B.

How Impact is ultimately calculated varies based on whether we're analyzing by metric or by dimension. For metrics, all As in a subsession are averaged. For dimensions, the value of each A contributes 1/N to the value assigned to B, where N is the number of As in the subsession.

Example questions you can answer with user impact analysis

Select one of the following questions to expand the section.

HEART - Five dimensions of customer experience

This section describes how to enable and use the HEART Workbook in Azure Monitor. The HEART workbook is based on the HEART measurement framework, which was originally introduced by Google. Several Microsoft internal teams use HEART to deliver better software.

Overview

HEART is an acronym that stands for happiness, engagement, adoption, retention, and task success. It helps product teams deliver better software by focusing on five dimensions of customer experience:

• Happiness: Measure of user attitude
• Engagement: Level of active user involvement
• Adoption: Target audience penetration
• Retention: Rate at which users return
• Task success: Productivity empowerment

These dimensions are measured independently, but they interact with each other.

• Adoption, engagement, and retention form a user activity funnel. Only a portion of users who adopt the tool come back to use it.
• Task success is the driver that progresses users down the funnel and moves them from adoption to retention.
• Happiness is an outcome of the other dimensions and not a stand-alone measurement. Users who progressed down the funnel and are showing a higher level of activity are ideally happier.

Prerequisites

• Azure subscription: Create an Azure subscription for free

• Application Insights resource: Create an Application Insights resource

• Click Analytics: Set up the Click Analytics Autocollection plug-in.

• Specific attributes: Instrument the following attributes to calculate HEART metrics.

  Source	Attribute	Description
  customEvents	user_AuthenticatedId	Unique authenticated user identifier
  customEvents	session_Id	Unique session identifier
  customEvents	appName	Unique Application Insights app identifier
  customEvents	itemType	Category of customEvents record
  customEvents	timestamp	Datetime of event
  customEvents	operation_Id	Correlate telemetry events
  customEvents	user_Id	Unique user identifier
  customEvents ¹	parentId	Name of feature
  customEvents ¹	pageName	Name of page
  customEvents ¹	actionType	Category of Click Analytics record
  pageViews	user_AuthenticatedId	Unique authenticated user identifier
  pageViews	session_Id	Unique session identifier
  pageViews	appName	Unique Application Insights app identifier
  pageViews	timestamp	Datetime of event
  pageViews	operation_Id	Correlate telemetry events
  pageViews	user_Id	Unique user identifier

  ¹: To emit these attributes, use the Click Analytics Autocollection plug-in via npm.

• If you're setting up the authenticated user context, instrument the following attributes:

  Source	Attribute	Description
  customEvents	user_AuthenticatedId	Unique authenticated user identifier.

Open the workbook

To use the HEART workbook in Application Insights, navigate to the Workbooks pane and locate the Product Analytics using the Click Analytics Plugin category. You only have to interact with the main workbook, HEART Analytics - All Sections. This workbook contains the other six workbooks as tabs.

Confirm that data is flowing

To validate that data is flowing as expected to light up the metrics accurately, select the Development Requirements tab.

If data isn't flowing as expected, this tab shows the specific attributes with issues.

Workbook structure

The workbook shows metric trends for the HEART dimensions split over seven tabs. Each tab contains descriptions of the dimensions, the metrics contained within each dimension, and how to use them.

The tabs are:

• Summary: Summarizes usage funnel metrics for a high-level view of visits, interactions, and repeat usage.
• Adoption: Helps you understand the penetration among the target audience, acquisition velocity, and total user base.
• Engagement: Shows frequency, depth, and breadth of usage.
• Retention: Shows repeat usage.
• Task success: Enables understanding of user flows and their time distributions.
• Happiness: We recommend using a survey tool to measure customer satisfaction score (CSAT) over a five-point scale. On this tab, we provide the likelihood of happiness via usage and performance metrics.
• Feature metrics: Enables understanding of HEART metrics at feature granularity.

How HEART dimensions are defined and measured

Happiness

Happiness is a user-reported dimension that measures how users feel about the product offered to them.

A common approach to measure happiness is to ask users a CSAT question like How satisfied are you with this product? Users' responses on a three- or a five-point scale (for example, no, maybe, and yes) are aggregated to create a product-level score that ranges from 1 to 5. Because user-initiated feedback tends to be negatively biased, HEART tracks happiness from surveys displayed to users at predefined intervals.

Common happiness metrics include values such as Average Star Rating and Customer Satisfaction Score. Send these values to Azure Monitor by using one of the custom ingestion methods described in Custom sources.

Engagement

Engagement is a measure of user activity. Specifically, user actions are intentional, such as clicks. Active usage can be broken down into three subdimensions:

• Activity frequency: Measures how often a user interacts with the product. For example, users typically interact daily, weekly, or monthly.

• Activity breadth: Measures the number of features users interact with over a specific time period. For example, users interacted with a total of five features in June 2021.

• Activity depth: Measures the number of features users interact with each time they launch the product. For example, users interacted with two features on every launch.

Measuring engagement can vary based on the type of product being used. For example, a product like Microsoft Teams is expected to have a high daily usage, which makes it an important metric to track. But for a product like a paycheck portal, measurement might make more sense at a monthly or weekly level.

Adoption

Adoption enables understanding of penetration among the relevant users, who you're gaining as your user base, and how you're gaining them. Adoption metrics are useful for measuring:

• Newly released products.
• Newly updated products.
• Marketing campaigns.

Retention

A retained user is a user who was active in a specified reporting period and its previous reporting period. Retention is typically measured with the following metrics.

Task success

Task success tracks whether users can do a task efficiently and effectively by using the product's features. Many products include structures that are designed to funnel users through completing a task. Some examples include:

• Adding items to a cart and then completing a purchase.
• Searching a keyword and then selecting a result.
• Starting a new account and then completing account registration.

A successful task meets three requirements:

• Expected task flow: The intended task flow of the feature was completed by the user and aligns with the expected task flow.
• High performance: The intended functionality of the feature was accomplished in a reasonable amount of time.
• High reliability: The intended functionality of the feature was accomplished without failure.

A task is considered unsuccessful if any of the preceding requirements isn't met.

Set up a custom task by using the following parameters.

Frequently asked questions

Does the initial event represent the first time the event appears in a session or anytime it appears in a session?

The initial event on the visualization only represents the first time a user sent that page view or custom event during a session. If users can send the initial event multiple times in a session, then the Step 1 column only shows how users behave after the first instance of an initial event, not all instances.

Some of the nodes in my visualization have a level that's too high. How can I get more detailed nodes?

Use the Split by options on the Edit menu:

1. Select the event you want to break down on the Event menu.

2. Select a dimension on the Dimension menu. For example, if you have an event called Button Clicked, try a custom property called Button Name.

I defined a cohort of users from a certain country/region. When I compare this cohort in the Users tool to setting a filter on that country/region, why do I see different results?

Cohorts and filters are different. Suppose you have a cohort of users from the United Kingdom (defined like the previous example), and you compare its results to setting the filter Country or region = United Kingdom:

• The cohort version shows all events from users who sent one or more events from the United Kingdom in the current time range. If you split by country or region, you likely see many countries and regions.

• The filters version only shows events from the United Kingdom. If you split by country or region, you see only the United Kingdom.

How do I view the data at different grains (daily, monthly, or weekly)?

You can select the Date Grain filter to change the grain. The filter is available across all the dimension tabs.

How do I access insights from my application that aren't available on the HEART workbooks?

You can dig into the data that feeds the HEART workbook if the visuals don't answer all your questions. To do this task, under the Monitoring section in Application Insights, select Logs, and query the customEvents table. Some of the Click Analytics attributes are contained within the customDimensions field.

An example query is shown here:

customEvents
| where isnotnull(customDimensions.actionType)
| extend parentid=tostring(customDimensions.parenId),
pagename=tostring(customDimensions.pageName),
actiontype=tostring(customDimensions.actionType)
| project actiontype,parentid,pagename,
user_AuthenticatedId,user_Id,session_Id,itemType,timestamp

To learn more about Logs in Azure Monitor, see Azure Monitor Logs overview.

Can I edit visuals in the workbook?

Yes. To learn how to edit workbook templates, see Azure Workbooks templates.

Next steps

• Check out the GitHub repository and npm Package for the Click Analytics Autocollection plug-in.
• Learn more about the Google HEART framework.
• To learn more about workbooks, see the Workbooks overview.