Quickstart: custom question answering (CQA)

This quickstart guides you through the essential steps needed to create, test, and deploy a custom question answering (CQA) project in the Azure AI Foundry. Whether you're transitioning from Language Studio or starting from scratch, this quickstart is for you. It provides clear and actionable instructions to achieve a fast and successful CQA project deployment.

Prerequisites

Before you get started, you need the following resources and permissions:

• An active Azure subscription. If you don't have one, create one for free.
• Requisite permissions. Make sure the person establishing the account and project is assigned as the Azure AI Account Owner role at the subscription level. Alternatively, having either the Contributor or Cognitive Services Contributor role at the subscription scope also meets this requirement. For more information, see Role based access control (RBAC).
• An Azure AI Foundry resource or an Azure AI Language resource.
• An Azure AI Search resource (required for accessing CQA). For more information on how to connect your Azure AI Search resource, see Configure connections in AI Foundry
• A Foundry project created in the Azure AI Foundry. For more information, see Create an AI Foundry project.

Get started

1. Navigate to the Azure AI Foundry.

2. If you aren't already signed in, the portal prompts you to do so with your Azure credentials.

3. Once signed in, you can create or access your existing projects within Azure AI Foundry.

4. If you're not already at your project for this task, select it.

Create your CQA fine tuning task

In the Azure AI Foundry, a fine-tuning task serves as your workspace for your CQA solutions. Previously, a fine-tuning task was referred to as a CQA project. You might encounter both terms used interchangeably in older CQA documentation.

1. After you select the Azure AI Foundry project to use for this quickstart, select fine-tuning from the left navigation menu.

   

2. From the main window, select the AI Service fine-tuning tab and then the + Fine-tune button.

   

3. From the Create service fine-tuning window, choose the Custom question answering tab and then select Next.

   

4. Select your Connected Azure AI Search resource from the Create CQA fine tuning task window. For more information, see Configure Azure resource connections.

5. Next, complete the Name and Language fields. For this project, you can leave the Default answer when no answer is returned field as is (No answer found).

6. Select the Create button.

Add a CQA knowledge base source

A CQA knowledge base is a structured set of question-and-answer pairs optimized for conversational AI. The knowledge base uses natural language processing to interpret user queries and return context-aware, accurate answers from a specific dataset.

1. From the Getting Started menu, select Manage sources.

   

2. From the main window, select the + Add source drop-down menu.

3. From the drop-down menu you can select Add chit chat, Add URLs, or Add Files.

   

4. For this project, let's choose Add chitchat.

5. From the Add new source window, let's choose Friendly.

   

6. Finally, select Add. It may take a few minutes for the source to be created.

7. Once created, the source is listed in the Manage sources window.

   

Test your knowledge base

1. Select Test knowledge base from the Getting Started menu.

2. Type the following in the Type your question field and then select Run.

     Hello! How are you doing today?
   
   

3. In the inspection interface, you can review the response confidence level and choose the most suitable answer.

   

Deploy your knowledge base

Deploying a CQA knowledge base means publishing your curated question-and-answer content as a live, searchable endpoint. This process moves your project from a testing phase to a production environment enabling client applications to use it for various projects and solutions, including chatbots.

1. Once your inspection is complete, choose the Deploy knowledge base section from the Getting Started menu.

2. Select the Deploy button first from the Deploy knowledge base main window and then from the Deploy this project pop-up window. It takes a few minutes to deploy.

3. After deployment is complete, your deployed project is listed in the Deploy knowledge base window.

That's it! Your Custom Question Answering (CQA) knowledge base provides a natural language interface to your data, allowing users to interact with information in a conversational manner. By deploying this solution, you can create advanced chatbots and interactive agents that comprehend user questions, supply precise answers, and adjust to changing informational requirements.

Prerequisites

• The current version of cURL. Several command-line switches are used in the quickstarts, which are noted in the cURL documentation.
• Azure subscription - Create one for free
• Custom question answering requires a Language resource with the custom question answering feature enabled to generate an API key and endpoint.
  • After your Language resource deploys, select Go to resource. You need the key and endpoint from the resource you create to connect to the API. Paste your key and endpoint into the code later in the quickstart.
• Create a Language resource with Azure CLI and provide the following properties: --api-properties qnaAzureSearchEndpointId=/subscriptions/<azure-subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Search/searchServices/<azure-search-service-name> qnaAzureSearchEndpointKey=<azure-search-service-auth-key>
• An existing project to query. If you have not setup a project, you can follow the instructions in the Language Studio quickstart. Or add a project that uses this Surface User Guide URL as a data source.

Setting up

Create environment variables

Your application must be authenticated to send API requests. For production, use a secure way of storing and accessing your credentials. In this example, you will write your credentials to environment variables on the local machine running the application.

To set the environment variable for your Language resource key, open a console window, and follow the instructions for your operating system and development environment.

• To set the LANGUAGE_KEY environment variable, replace your-key with one of the keys for your resource.
• To set the LANGUAGE_ENDPOINT environment variable, replace your-endpoint with the endpoint for your resource.

setx LANGUAGE_KEY your-key

setx LANGUAGE_ENDPOINT your-endpoint

export LANGUAGE_KEY=your-key

export LANGUAGE_ENDPOINT=your-endpoint

export LANGUAGE_KEY=your-key

export LANGUAGE_ENDPOINT=your-endpoint

Query a project

Generate an answer from a project

To query a custom question answering project with the REST APIs and cURL, you need the following information:

The cURL command is executed from a BASH shell. Edit this command with your own resource name, resource key, and JSON values and size of JSON.

curl -X POST -H "Ocp-Apim-Subscription-Key: $LANGUAGE_KEY" -H "Content-Type: application/json" -d '{
  "question": "How much battery life do I have left?"
  }'  '$LANGUAGE_ENDPOINT.cognitiveservices.azure.com/language/:query-knowledgebases?projectName={YOUR_PROJECT_NAME}&api-version=2021-10-01&deploymentName={DEPLOYMENT_NAME}'

When you run the code, if you're using the data source from the prerequisites you get an answer that looks as follows:

{
"answers": [
    {
      "questions": [
        "Check battery level"
      ],
      "answer": "If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.",
      "confidenceScore": 0.9185,
      "id": 101,
      "source": "https://support.microsoft.com/en-us/surface/how-to-use-your-surface-pen-8a403519-cd1f-15b2-c9df-faa5aa924e98",
      "metadata": {},
      "dialog": {
        "isContextOnly": false,
        "prompts": []
      }
    }
  ]
}

The confidenceScore returns a value between 0 and 1. Consider the confidence score as a percentage by multiplying it by 100. For example, a confidence score of 0.9185 indicates that the custom question-answering system is 91.85% confident that its response is correct based on the project information.

If you want to exclude answers where the confidence score falls below a certain threshold, you can add the confidenceScoreThreshold parameter.

curl -X POST -H "Ocp-Apim-Subscription-Key: $LANGUAGE_KEY" -H "Content-Type: application/json" -d '{
  "question": "How much battery life do I have left?",
  "confidenceScoreThreshold": "0.95",
  }'  '$LANGUAGE_ENDPOINT.cognitiveservices.azure.com//language/:query-knowledgebases?projectName=Sample-project&api-version=2021-10-01&deploymentName={DEPLOYMENT_NAME}'

Since we know from our previous execution of the code that our confidence score is: .9185 setting the threshold to .95 results in the default answer being returned.

{
  "answers": [
    {
      "questions": [],
      "answer": "No good match found in KB",
      "confidenceScore": 0.0,
      "id": -1,
      "metadata": {}
    }
  ]
}

Query text without a project

You can also use custom question answering without a project with the prebuilt custom question answering REST API, which is called via query-text. In this case, you provide question answering with both a question and the associated text records you would like to search for an answer at the time the request is sent.

For this example, you only need to modify the variables for API KEY and ENDPOINT.

curl -X POST -H "Ocp-Apim-Subscription-Key: $LANGUAGE_KEY" -H "Content-Type: application/json" -d '{
"question":"How long does it takes to charge a surface?",
"records":[
{"id":"doc1","text":"Power and charging.It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. It can take longer if you\u0027re using your Surface for power-intensive activities like gaming or video streaming while you\u0027re charging it"},
{"id":"doc2","text":"You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface."}],
"language":"en",
"stringIndexType":"Utf16CodeUnit"
}'  '$LANGUAGE_ENDPOINT.cognitiveservices.azure.com/language/:query-text?&api-version=2021-10-01'

This example returns a result of:

{  
"answers": [
    {
      "answer": "Power and charging.It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it",
      "confidenceScore": 0.9118788838386536,
      "id": "doc1",
      "answerSpan": {
        "text": "two to four hours",
        "confidenceScore": 0.9850527,
        "offset": 27,
        "length": 18
      },
      "offset": 0,
      "length": 243
    },
    {
      "answer": "It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it",
      "confidenceScore": 0.052793052047491074,
      "id": "doc1",
      "answerSpan": {
        "text": "longer",
        "confidenceScore": 0.6694634,
        "offset": 11,
        "length": 7
      },
      "offset": 109,
      "length": 134
    },
    {
      "answer": "You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface.",
      "confidenceScore": 0.017600709572434425,
      "id": "doc2",
      "answerSpan": {
        "text": "USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. The USB port on the power supply is only for charging",
        "confidenceScore": 0.1544854,
        "offset": 15,
        "length": 165
      },
      "offset": 0,
      "length": 280
    }
  ]
}

Use this quickstart for the custom question answering client library for .NET to:

• Get an answer from a project.
• Get an answer from a body of text that you send along with your question.
• Get the confidence score for the answer to your question.

Reference documentation | Package (NuGet) | Samples | Library source code

Prerequisites

• Azure subscription - Create one for free
• The Visual Studio IDE or current version of .NET Core.
• Custom question answering requires a Language resource with the custom question answering feature enabled to generate an API key and endpoint.
  • After your Language resource deploys, select Go to resource. You need the key and endpoint from the resource you create to connect to the API. Paste your key and endpoint into the code later in the quickstart.
• Create a Language resource with Azure CLI and provide the following properties: --api-properties qnaAzureSearchEndpointId=/subscriptions/<azure-subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Search/searchServices/<azure-search-service-name> qnaAzureSearchEndpointKey=<azure-search-service-auth-key>
• An existing project to query. If you don't have a project, you can follow the instructions in the Language Studio quickstart. Or add a project that uses this Surface User Guide URL as a data source.

Setting up

Create environment variables

Your application must be authenticated to send API requests. For production, use a secure way of storing and accessing your credentials. In this example, you will write your credentials to environment variables on the local machine running the application.

To set the environment variable for your Language resource key, open a console window, and follow the instructions for your operating system and development environment.

• To set the LANGUAGE_KEY environment variable, replace your-key with one of the keys for your resource.
• To set the LANGUAGE_ENDPOINT environment variable, replace your-endpoint with the endpoint for your resource.

setx LANGUAGE_KEY your-key

setx LANGUAGE_ENDPOINT your-endpoint

export LANGUAGE_KEY=your-key

export LANGUAGE_ENDPOINT=your-endpoint

export LANGUAGE_KEY=your-key

export LANGUAGE_ENDPOINT=your-endpoint

CLI

In a console window (such as cmd, PowerShell, or Bash), use the dotnet new command to create a new console app with the name question-answering-quickstart. This command creates a simple "Hello World" C# project with a single source file: program.cs.

dotnet new console -n question-answering-quickstart

Change your directory to the newly created app folder. You can build the application with:

dotnet build

The build output should contain no warnings or errors.

...
Build succeeded.
 0 Warning(s)
 0 Error(s)
...

Within the application directory, install the custom question answering client library for .NET with the following command:

dotnet add package Azure.AI.Language.QuestionAnswering

Query a project

Generate an answer from a project

The following example allows you to query a project using GetAnswers to get an answer to your question.

You need to update the code and provide your own values for the following variables:

From the project directory, open the program.cs file and replace with the following code:

using Azure;
using Azure.AI.Language.QuestionAnswering;
using System;

namespace question_answering
{
    class Program
    {
        static void Main(string[] args)
        {

            // This example requires environment variables named "LANGUAGE_KEY" and "LANGUAGE_ENDPOINT"
            Uri endpoint = new Uri("LANGUAGE_ENDPOINT");
            AzureKeyCredential credential = new AzureKeyCredential("LANGUAGE_KEY");
            string projectName = "{YOUR-PROJECT-NAME}";
            string deploymentName = "production";

            string question = "How long should my Surface battery last?";

            QuestionAnsweringClient client = new QuestionAnsweringClient(endpoint, credential);
            QuestionAnsweringProject project = new QuestionAnsweringProject(projectName, deploymentName);

            Response<AnswersResult> response = client.GetAnswers(question, project);

            foreach (KnowledgeBaseAnswer answer in response.Value.Answers)
            {
                Console.WriteLine($"Q:{question}");
                Console.WriteLine($"A:{answer.Answer}");
            }
        }
    }
}

While we're hard coding the variables for our example. For production, consider using a secure way of storing and accessing your credentials. For example, Azure key vault provides secure key storage.

After updating Program.cs and substituting in the correct variable values. Run the application with the dotnet run command from your application directory.

dotnet run

The response looks as follows:

Q: How much battery life do I have left?
A: If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.

For information on confidence scores, add the following print statement underneath the existing print statements:

Console.WriteLine($"Q:{question}");
Console.WriteLine($"A:{answer.Answer}");
Console.WriteLine($"({answer.Confidence})"); // add this line

If you execute dotnet run again, you now receive a result with a confidence score:

Q:How much battery life do I have left?
A:If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.
(0.9185)

Consider the confidence score as a percentage by multiplying it by 100. For example, a confidence score of 0.9185 indicates that the custom question-answering system is 91.85% confident that its response is correct based on the project information.

If you want to exclude answers where the confidence score falls below a certain threshold, you use AnswerOptions to add the ConfidenceScoreThreshold property.

QuestionAnsweringClient client = new QuestionAnsweringClient(endpoint, credential);
QuestionAnsweringProject project = new QuestionAnsweringProject(projectName, deploymentName);
AnswersOptions options = new AnswersOptions(); //Add this line
options.ConfidenceThreshold = 0.95; //Add this line

Response<AnswersResult> response = client.GetAnswers(question, project, options); //Add the additional options parameter

Since we know from our previous execution of the code that our confidence score is: .9185 setting the threshold to .95 results in the default answer being returned.

Q:How much battery life do I have left?
A:No good match found in KB
(0)

Query text without a project

You can also use custom question answering without a project with GetAnswersFromText. In this case, you provide custom question answering with both a question and the associated text records you would like to search for an answer at the time the request is sent.

For this example, you only need to modify the variables for endpoint and credential.

using Azure;
using Azure.AI.Language.QuestionAnswering;
using System;
using System.Collections.Generic;


namespace questionansweringcsharp
{
    class Program
    {
        static void Main(string[] args)
        {

            Uri endpoint = new Uri("https://{YOUR-ENDPOINT}.cognitiveservices.azure.com/");
            AzureKeyCredential credential = new AzureKeyCredential("YOUR-LANGUAGE-RESOURCE-KEY");
            QuestionAnsweringClient client = new QuestionAnsweringClient(endpoint, credential);

            IEnumerable<TextDocument> records = new[]
            {
                new TextDocument("doc1", "Power and charging.It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. " +
                         "It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it"),
                new TextDocument("doc2", "You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. " +
                         "The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface."),
            };

            AnswersFromTextOptions options = new AnswersFromTextOptions("How long does it takes to charge a surface?", records);
            Response<AnswersFromTextResult> response = client.GetAnswersFromText(options);

           foreach (TextAnswer answer in response.Value.Answers)
            {
                if (answer.Confidence > .9)
                {
                    string BestAnswer = response.Value.Answers[0].Answer;

                    Console.WriteLine($"Q:{options.Question}");
                    Console.WriteLine($"A:{BestAnswer}");
                    Console.WriteLine($"Confidence Score: ({response.Value.Answers[0].Confidence:P2})"); //:P2 converts the result to a percentage with 2 decimals of accuracy. 
                    break;
                }
                else
                {
                    Console.WriteLine($"Q:{options.Question}");
                    Console.WriteLine("No answers met the requested confidence score.");
                    break;
                }
            }

        }
    }
}

To run the code, replace the Program.cs with the contents of the script block and modify the endpoint and credential variables to correspond to the language resource you created as part of the prerequisites.

In this case, we iterate through all responses and only return the response with the highest confidence score that is greater than 0.9. To understand more about the options available with GetAnswersFromText.

Use this quickstart for the custom question answering client library for Python to:

• Get an answer from a project.
• Get an answer from a body of text that you send along with your question.
• Get the confidence score for the answer to your question.

Package (PyPI) | Samples | Library source code

Prerequisites

• Azure subscription - Create one for free
• Python 3.x
• Custom question answering requires a Language resource with the custom question answering feature enabled to generate an API key and endpoint.
  • After your Language resource deploys, select Go to resource. You need the key and endpoint from the resource you create to connect to the API. Paste your key and endpoint into the code later in the quickstart.
• Create a Language resource with Azure CLI and provide the following properties: --api-properties qnaAzureSearchEndpointId=/subscriptions/<azure-subscription-id>/resourceGroups/<resource-group-name>/providers/Microsoft.Search/searchServices/<azure-search-service-name> qnaAzureSearchEndpointKey=<azure-search-service-auth-key>
• An existing project to query. If you don't have a project, you can follow the instructions in the Language Studio quickstart. Or add a project that uses this Surface User Guide URL as a data source.

Setting up

Install the client library

After installing Python, you can install the client library with:

pip install azure-ai-language-questionanswering

Query a project

Generate an answer from a project

The example allows you to query a project using get_answers to get an answer to your question. You can copy this code into a dedicated .py file or into a cell in Jupyter Notebook/Lab.

You need to update the code and provide your own values for the following variables.

from azure.core.credentials import AzureKeyCredential
from azure.ai.language.questionanswering import QuestionAnsweringClient

endpoint = "https://{YOUR-ENDPOINT}.cognitiveservices.azure.com/"
credential = AzureKeyCredential("{YOUR-LANGUAGE-RESOURCE-KEY}")
knowledge_base_project = "{YOUR-PROJECT-NAME}"
deployment = "production"

def main():
    client = QuestionAnsweringClient(endpoint, credential)
    with client:
        question="How much battery life do I have left?"
        output = client.get_answers(
            question = question,
            project_name=knowledge_base_project,
            deployment_name=deployment
        )
    print("Q: {}".format(question))
    print("A: {}".format(output.answers[0].answer))

if __name__ == '__main__':
    main()

While we're hard coding the variables for our example. For production, consider using a secure way of storing and accessing your credentials. For example, Azure key vault provides secure key storage.

When you run the code, if you're using the data source from the prerequisites you get an answer that looks as follows:

Q: How much battery life do I have left?
A: If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.

For information on confident scores add the following print statements:

print("Q: {}".format(question))
print("A: {}".format(output.answers[0].answer))
print("Confidence Score: {}".format(output.answers[0].confidence)) # add this line 

You receive a result with a confidence score:

Q: How much battery life do I have left?
A: If you want to see how much battery you have left, go to **Start  **> **Settings  **> **Devices  **> **Bluetooth & other devices  **, then find your pen. The current battery level will appear under the battery icon.
Confidence Score: 0.9185

Consider the confidence score as a percentage by multiplying it by 100. For example, a confidence score of 0.9185 indicates that the custom question-answering system is 91.85% confident that its response is correct based on the project information.

If you want to exclude answers where the confidence score falls below a certain threshold, you can modify the AnswerOptions to add the confidence_threshold parameter.

        output = client.get_answers(
            confidence_threshold = 0.95, #add this line
            question = question,
            project_name=knowledge_base_project,
            deployment_name=deployment
        )

Since we know from our previous execution of the code that our confidence score is: .9185 setting the threshold to .95 results in the default answer being returned.

Q: How much battery life do I have left?
A: No good match found in KB
Confidence Score: 0.0

Query text without a project

You can also use custom question answering without a project with get_answers_from_text. In this case, you provide custom question answering with both a question and the associated text records you would like to search for an answer at the time the request is sent.

For this example, you only need to modify the variables for endpoint and credential.

import os
from azure.core.credentials import AzureKeyCredential
from azure.ai.language.questionanswering import QuestionAnsweringClient
from azure.ai.language.questionanswering import models as qna

endpoint = "https://{YOUR-ENDPOINT}.cognitiveservices.azure.com/"
credential = AzureKeyCredential("YOUR-LANGUAGE-RESOURCE-KEY")

def main():
    client = QuestionAnsweringClient(endpoint, credential)
    with client:
        question="How long does it takes to charge a surface?"
        input = qna.AnswersFromTextOptions(
            question=question,
            text_documents=[
                "Power and charging. It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. " +
                "It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it.",
                "You can use the USB port on your Surface Pro 4 power supply to charge other devices, like a phone, while your Surface charges. " +
                "The USB port on the power supply is only for charging, not for data transfer. If you want to use a USB device, plug it into the USB port on your Surface.",
            ]
        )


        output = client.get_answers_from_text(input)

    best_answer = [a for a in output.answers if a.confidence > 0.9][0]
    print(u"Q: {}".format(input.question))
    print(u"A: {}".format(best_answer.answer))
    print("Confidence Score: {}".format(output.answers[0].confidence))

if __name__ == '__main__':
    main()

You can copy this code into a dedicated .py file or into a new cell in Jupyter Notebook/Lab. This example returns a result of:

Q: How long does it takes to charge surface?
A: Power and charging. It takes two to four hours to charge the Surface Pro 4 battery fully from an empty state. It can take longer if you're using your Surface for power-intensive activities like gaming or video streaming while you're charging it.
Confidence Score: 0.9254655838012695

In this case, we iterate through all responses and only return the response with the highest confidence score that is greater than 0.9. To understand more about the options available with get_answers_from_text, review the AnswersFromTextOptions parameters.