What are tools in Azure AI Foundry Agent Service?

To empower your AI agent with grounded data or the capability to take actions and automating workflows, the Foundry Agent service provides a wide range of built-in tools, such as Grounding with Bing Search, Azure AI Search, Azure Logic Apps, as well as third-party parter tools, such as Tripadvisor. This page is designed to provide an overview of tools provided in the Foundry Agent Service.

Knowledge tools

To keep your AI agent informed with richer context from various data sources. The Foundry Agent Service has covered a wide range of data types:

• private data: Azure AI Search, File Search, Microsoft Fabric, and more
• public web data: Grounding with Bing Search
• licensed data: Tripadvisor, Morningstar
• unstructured data: Azure AI Search, File Search
• structured data: Microsoft Fabric and more

Action tools

To streamline workflows with your AI agent with capabilities to take actions. The Foundry Agent Service provides different action tools for you with different level of flexibility, control, and ease of integration:

• Azure Logic Apps: Low-code / no-code solution to add a workflow to your AI Agent
• OpenAPI Spec tool: Bring an existing OpenAPI specification of a service API you want to add to your AI agent, with no or minor changes.
• Function calling: Write your own custom, stateless functions to define the expected behaviors.
• Azure Functions: Write and manage your own custom, stateful functions.

How does a tool work in the Foundry Agent Service?

Tools are optional capabilities you can add to your AI agent for AI models to decide and pick based on the user query and context. When a user sends a query, the AI model identifies the intent with the context and potentially rewrites the user query. Then the AI model decides which tools to be called for each run. For example, if you add both the Grounding with Bing Search tool and the Azure AI Search tool to your agent and ask "what is the weather in Seattle today?", the model will identify your intent to ask about real-time information and more likely to invoke the Grounding with Bing Search tool.

You can add tools at the agent, thread, or run level. By providing tools at a narrower level, the tool resources will override tool resources at a broader level. For example, tool resources at the run level override tool resources at thread level. Currently, you can add multiple tools but you can add one instance of each of the following tools: File Search, Azure AI Search, Grounding with Bing Search, Grounding with Bing Custom Search, Microsoft Fabric, and other tools under knowledge section.

When a user sends a query to the agent, it will create a thread, run, and message. For each run, the AI model decides what tools to invoke based on the user intent and available tool resources. Based on the tool outputs, the AI model might decide to invoke another tool or call the same tool again to get more context. For example, when you use Grounding with Bing Search tool, you might see multiple Bing Search queries when tracing a thread. This means the AI model actually calls the Grounding with Bing Search tool multiple times with different queries to get more information. If you want to learn more about what tools are called and how the AI model invokes them, check the run step details.

There are various ways to influence how your AI agent invokes tools:

• The tool_choice parameter: Most deterministic way of controlling which (if any) tool is called by the model. By default, it is set to auto, which means the AI model will decide. If you want to force the model to call a specific tool, you can provide the specification of this tool, for example

  run = project_client.agents.runs.create_and_process(
        thread_id=thread.id, 
        agent_id=agent.id,
        tool_choice={"type": "bing_grounding"}  # specify the tool to use
        )
  

• The instructions parameter: Nondeterministic. Use the instructions to help the AI model understand your use case and the purposes of each tool. You want to tell the AI model what information or actions each tool can do. For example "use the AI Search tool <tool_name> for product related information, use the Fabric tool <tool_name> for sales related information." Sometimes the user query can be responded by the model's base knowledge or by the tools, you want to provide instructions like "use the tool outputs to generate a response, don't use your own knowledge."

Prerequisites

• A created agent
• Make sure your AI model has enough Tokens-Per-Minute (TPM) allocated. We recommend having a minimum of 30k TPM. You can change the TPM allocation by going to models + endpoints in the AI Foundry portal and edit your model.

Built-in tools

The Foundry Agent Service provides the following built-in tools. You can use them with the REST API, SDK, and Azure AI Foundry portal.

Non-Microsoft tools

The following tools are authored by third-party partners. Use the links below to view the documentation and code samples.

Best Practices

Use system instruction to help model invoke the right tool

In order for the model to understand which tools to use, you want to provide detailed instruction for the model to describe when and how to use the tool. You might want to consider providing the following information:

• Primary Objective: what is the objective of this agent? what is the goal of related tasks? what are the expected outcomes?
• Your responsibilities: what tasks you expect the agent to perform. For example, calling Grounding with Bing Search tool to get the latest information about local events.
• Inputs you may receive: what inputs do you expect the agent to receive?
• For each tool:
  • The tool name
  • A description of the tool
  • Triggers: when do you expect this tool to be called? What type of information will be searched? What will queries contain?
  • An example of a query

For example, you might provide tool instructions like the following for the Grounding with Bing Search tool:

Grounding with Bing Search tool

• Use: Gather external trends or news to enrich the post with real-time insights.
• Trigger this when:
  • The user asks to reference recent data or competitive context.
  • Example: "Can you reference the latest industry trends?" or "What are competitors doing?".