When a multi-agent AI system moves from a laptop to an enterprise environment, the language model is rarely the first thing to break. The tools around it are. A schema changes, a token expires, an app restarts in the middle of a conversation, and the agent quietly stops doing its job.

This article walks through a small reference implementation that deals with those problems using four pieces: the Model Context Protocol (MCP) for tools, Microsoft Foundry for the agent, Azure Cosmos DB for session state, and OpenTelemetry for tracing. Microsoft Foundry was called Azure AI Foundry until recently, so you will see both names in older docs and posts.

The problem with glue code

Picture a supply chain assistant that checks stock in SAP, reviews contracts, and raises purchase requests. In a typical first version, every integration is a hand-written wrapper: a function that calls the API, a JSON parser for the response, and a schema pasted into the agent definition. It works until the API changes. Then the agent starts returning wrong answers, and nothing in your logs says why.

Four things tend to go wrong at the same time:

  • Tool schemas are hardcoded in the agent, so they drift from the real API.
  • Session state lives in memory and disappears when the container restarts.
  • Approval steps are custom code that every workflow re-implements.
  • Logs are scattered, so nobody can tell where the latency or the token cost came from.

MCP addresses the first problem by putting each tool behind its own server. The agent asks the server what it can do, gets a typed schema back, and calls the tool over a standard protocol. The tool can then change, scale, and be secured on its own.

What we are building

The example covers one slice of the supply chain scenario: a Foundry agent that checks plant inventory through an MCP server, saves its session state to Cosmos DB, and emits OpenTelemetry traces. A Procurement agent or a Contract Auditor would follow the same pattern, but they are not part of the code here. The request flow looks like this:

  1. A user message or an ERP event reaches the Foundry agent.
  2. The agent discovers the tools exposed by the remote MCP server.
  3. When the agent calls a tool, Foundry pauses for approval if the tool is configured to require it.
  4. The app loads the session from Cosmos DB, runs the turn, and saves the new state using the document's ETag.
  5. Each turn is wrapped in an OpenTelemetry span and exported to Application Insights.

Prerequisites

  • Python 3.10 or newer.
  • A Microsoft Foundry project with a model deployment (the example uses gpt-4o; use whatever you have deployed).
  • An Azure Cosmos DB for NoSQL account with a container partitioned on /plantId.
  • An Application Insights resource.
  • Packages: mcp, pydantic, azure-identity, azure-ai-projects (install with --pre, because the agent APIs used here are in the 2.x preview), azure-cosmos, and azure-monitor-opentelemetry.

The agent APIs are still moving, so pin your package versions and check the release notes when you upgrade.

The code is split into four files, one per step:

  • sap_mcp_server.py for the MCP server
  • foundry_agent.py for the agent definition
  • state_store.py for Cosmos DB session state
  • telemetry.py for tracing and the turn loop

Step 1: Build the MCP server

The server below uses FastMCP and exposes one tool. The data is a hardcoded dictionary so the example runs anywhere. In a real deployment the function would call your SAP OData endpoint.

Python
# sap_mcp_server.py
from mcp.server.fastmcp import FastMCP
from pydantic import BaseModel

mcp = FastMCP("SAP-Enterprise-Inventory-Service")

# Mock data for the demo. Replace with a call to your SAP gateway.
STOCK = {
    ("PLANT-1020", "MAT-9920"): {"on_hand": 450, "reorder": 500},
    ("PLANT-3040", "MAT-9920"): {"on_hand": 1200, "reorder": 300},
}


class InventoryResponse(BaseModel):
    plant_id: str
    material_number: str
    stock_on_hand: int
    reorder_threshold: int
    backorder_status: bool


@mcp.tool()
async def query_sap_inventory(plant_id: str, material_number: str) -> InventoryResponse:
    """Return current stock level and reorder threshold for a material at a plant."""
    record = STOCK.get((plant_id, material_number), {"on_hand": 0, "reorder": 100})
    return InventoryResponse(
        plant_id=plant_id,
        material_number=material_number,
        stock_on_hand=record["on_hand"],
        reorder_threshold=record["reorder"],
        backorder_status=record["on_hand"] < record["reorder"],
    )


if __name__ == "__main__":
    mcp.run(transport="streamable-http")

Host this somewhere Foundry can reach over HTTPS, for example Azure Container Apps. The example server has no authentication. Before you point it at real data, put authentication in front of it, and decide how Foundry will present credentials to it.

Step 2: Attach the server to a Foundry agent

The agent definition lists the MCP server as a tool. Authentication to the Foundry project uses DefaultAzureCredential, so there are no keys in the code.

Python
# foundry_agent.py
import os

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import MCPTool, PromptAgentDefinition

project_client = AIProjectClient(
    endpoint=os.environ["AZURE_AI_FOUNDRY_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
)

sap_tool = MCPTool(
    server_label="sap-inventory-gateway",
    server_url=os.environ["SAP_MCP_SERVER_URL"],
    require_approval="always",
)

agent = project_client.agents.create_version(
    agent_name="ProcurementSwarmOrchestrator",
    definition=PromptAgentDefinition(
        model="gpt-4o",
        instructions=(
            "You are an enterprise procurement agent. "
            "Check stock levels with the SAP inventory tool before suggesting a purchase order."
        ),
        tools=[sap_tool],
    ),
)

print(f"Agent created: {agent.name}, version {agent.version}")

A note on require_approval="always". The only tool here reads data, so approving every call is cautious to the point of being annoying. I left it on because it shows where the human gate sits. In practice you would require approval for tools that change something, such as creating a purchase order, and let read-only tools run freely.

Step 3: Keep session state in Cosmos DB

Approvals can take hours. If the session lives in process memory, a restart loses it. Cosmos DB fixes that, and its ETag gives you optimistic concurrency: a save succeeds only if nobody else changed the document since you read it. That prevents two workers from overwriting each other's progress.

In the Python SDK, that check is done with etag and match_condition on a replace call.

Python
# state_store.py
from datetime import datetime, timezone

from azure.core import MatchConditions
from azure.cosmos.aio import CosmosClient
from azure.cosmos.exceptions import (
    CosmosAccessConditionFailedError,
    CosmosResourceNotFoundError,
)
from azure.identity.aio import DefaultAzureCredential


class AgentStateStore:
    def __init__(self, container):
        self.container = container

    async def load(self, session_id: str, plant_id: str):
        """Return (state, etag), or (None, None) for a new session."""
        try:
            item = await self.container.read_item(item=session_id, partition_key=plant_id)
        except CosmosResourceNotFoundError:
            return None, None
        return item["state"], item["_etag"]

    async def save(self, session_id: str, plant_id: str, state: dict, etag: str | None = None) -> str:
        document = {
            "id": session_id,
            "plantId": plant_id,
            "state": state,
            "lastUpdated": datetime.now(timezone.utc).isoformat(),
            "ttl": 2592000,  # 30 days
        }
        if etag is None:
            result = await self.container.create_item(document)
        else:
            try:
                result = await self.container.replace_item(
                    item=session_id,
                    body=document,
                    etag=etag,
                    match_condition=MatchConditions.IfNotModified,
                )
            except CosmosAccessConditionFailedError:
                raise RuntimeError(f"Session {session_id} was changed by another worker.")
        return result["_etag"]

To open the store, create the client and the credential inside async with blocks so both are closed cleanly:

Python
async with DefaultAzureCredential() as credential:
    async with CosmosClient(endpoint, credential=credential) as client:
        container = client.get_database_client("agents").get_container_client("AgentSessions")
        store = AgentStateStore(container)
        # run your turns here

Two details are easy to miss. The per-item ttl only takes effect if the container has a default TTL turned on (a value of -1 enables TTL without expiring items by default). And the async client needs the async credential from azure.identity.aio, which is why the imports differ from Step 2.

Step 4: Trace every turn with OpenTelemetry

Call configure_azure_monitor once at startup. After that, wrap each agent turn in a span and record the model and token usage, so you can see cost and latency per turn in Application Insights.

Python
# telemetry.py
import os

from azure.monitor.opentelemetry import configure_azure_monitor
from opentelemetry import trace

configure_azure_monitor(
    connection_string=os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"]
)

tracer = trace.get_tracer("enterprise.agent.swarm")


async def run_turn(store, session_id: str, plant_id: str, user_message: str):
    state, etag = await store.load(session_id, plant_id)
    state = state or {"history": []}

    with tracer.start_as_current_span("agent.turn") as span:
        span.set_attribute("gen_ai.provider.name", "azure.ai.openai")
        span.set_attribute("gen_ai.request.model", "gpt-4o")
        span.set_attribute("enterprise.plant_id", plant_id)

        # call_agent is your code: it sends the message to the Foundry agent,
        # handles any approval requests, and returns the reply and token usage.
        reply, usage = await call_agent(user_message, state)

        span.set_attribute("gen_ai.usage.input_tokens", usage.input_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", usage.output_tokens)

    state["history"].append({"user": user_message, "agent": reply})
    await store.save(session_id, plant_id, state, etag)
    return reply

The attribute names follow the OpenTelemetry GenAI semantic conventions. Earlier versions used gen_ai.system and gen_ai.usage.prompt_tokens, and you will still see those in older examples, so check which version your tooling expects.

This trace covers the agent side. To see the MCP server's work inside the same trace, instrument the server with OpenTelemetry as well and make sure the W3C traceparent header is propagated on the call. I did not wire that up here.

Custom wrappers compared with MCP

Area Custom tool wrappers MCP with Foundry
Tool discovery Schemas copied into the agent definition The agent asks the server for its tools and schemas
Reuse Tied to one SDK or framework Any MCP-capable client can use the same server
Isolation Runs inside the agent process with its credentials Runs as a separate service you can secure on its own
Human approval Custom code in each workflow A require_approval setting on the tool

Security notes

MCP gives you a clean boundary, but it does not make tool use safe by itself. Tool descriptions and tool results are text that the model reads, so a compromised or malicious server can try to steer the agent through them. Treat third-party MCP servers like any other dependency: review them, restrict which tools the agent may call, and keep approvals on for anything that writes data. Input validation and prompt-injection defenses still belong in your design.

A note for .NET developers

The example is in Python, but the same architecture applies on .NET. There is an official MCP C# SDK for building servers and a .NET client library for Foundry projects, and Cosmos DB ETags and OpenTelemetry work the same way there. If readers are interested, I can follow up with a C# version.

What to take away

  1. Put each enterprise integration behind an MCP server so tools can change without touching the agent.
  2. Store session state outside the process, and use ETags so concurrent workers cannot silently overwrite each other.
  3. Record model, token usage, and latency on every turn from day one. It is much harder to add later.
  4. Require approval for tools that change data, and secure the MCP server like any production API.

Resources