AI-901 lesson 8: agents in Foundry

A model on its own answers from what it learned in training and what the prompt contains. An agent in Microsoft Foundry wraps a model deployment with three additions: instructions that define its role, tools that let it take actions or look things up, and optionally knowledge it can consult. The model decides when a tool is needed and uses the result in its answer. The agent is created and kept inside the Foundry project, and a client application calls the agent by name, not the model deployment directly. AI-901 covers the single-agent case only: one agent, built and tested in the portal.

What an agent adds to a model

The figure above shows the pieces. Instructions play the part of a system prompt that belongs to the agent and travels with it, so every caller gets the same persona and rules. The model deployment is the language model doing the reasoning, picked from the deployments in the project. Tools are capabilities the agent can invoke. Foundry’s built-in tools include web search, file search, and code interpreter, and custom tools can be added through functions, OpenAPI descriptions or MCP servers. Knowledge is data the agent can draw on, such as uploaded files it searches to ground its answers, which is the RAG idea from the previous lesson applied to an agent.

The difference from a plain chat deployment is action: an agent can search documents, run code, or call a business function, then fold the result into its reply. The topic page AI agents explained discusses the wider idea.

Foundry’s service for this is Foundry Agent Service. The portal path in this course creates a prompt agent, which is defined purely by configuration, with Foundry hosting and running it and no application code or containers to maintain. Hosted agents (custom code that Foundry runs and scales for you) and multi-agent orchestration go beyond this scope.

Creating and testing in the portal

The order of steps is what the exam checks:

1. Have a model deployed in the project, because an agent needs a deployment to reason with.
2. Create the agent in the portal and give it a name.
3. Write its instructions: role, scope, tone and what to do when it cannot help.
4. Choose the model deployment.
5. Add tools and, if needed, knowledge, such as attaching files for file search.
6. Test in the agents playground by holding a multi-turn conversation, checking whether it follows its instructions and uses tools when it should.
7. Adjust and retest. Changes are saved as new versions of the agent.

The lightweight client for an agent

Once the agent behaves, an application can use it. The client connects to the project endpoint with an identity, as in the chat client, but now it targets the agent by its name. It starts a conversation, sends the user’s message, and reads the response, which may already include the results of tool calls the agent made behind the scenes. Sending further messages in the same conversation keeps the history, so follow-up questions work. Instructions and tools live in the agent, so improving it later does not require redeploying the app.

An illustrative sketch of such a client in Python, with placeholders for the endpoint and agent name:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

project = AIProjectClient(endpoint="<project-endpoint>", credential=DefaultAzureCredential())
openai = project.get_openai_client(agent_name="<agent-name>")
conversation = openai.conversations.create()
reply = openai.responses.create(conversation=conversation.id, input="What is the returns policy?")
print(reply.output_text)

SDK method names change between versions, and the exam tests the steps (connect to the project, target the agent, hold a conversation, read the reply), not the exact names.

Worked example: a returns assistant

A 200-person online shop wants an assistant that explains its returns policy and, when asked, works out refund amounts. An admin creates one agent. The instructions say to answer in a friendly tone, only about returns, and to ask for an order total when a refund is requested. The policy PDF is attached for file search so answers are grounded in the real policy, and code interpreter is enabled so refund arithmetic is computed rather than guessed. In the agents playground the admin checks an answer against the PDF and asks for a refund on a discounted order. Then the shop’s web chat widget calls the agent by name, and when the policy changes only the PDF is replaced.

Easy to confuse

• Agent vs model deployment. The deployment generates text. The agent is the configured unit that uses a deployment together with instructions and tools.
• Tool vs knowledge. A tool performs an action or query, while knowledge is content it searches.

Key points

• An agent combines a model deployment, instructions and tools or knowledge, and it can decide when to use a tool.
• Prompt agents are defined by configuration and run by Foundry without custom code.
• Create the agent, set instructions, add tools or knowledge, then test in the agents playground.
• A client app calls the agent by name through the project endpoint and keeps history in a conversation.
• Behaviour changes are made in the agent, not in the client application.

Check yourself

This lesson continues the implementation domain (55-60% of AI-901) and links back to the Foundry implementation recap, where single-agent creation and the agent client are summarised.