Mastering the Foundry Model Catalog, Model Router, and Foundry Local

Imagine hiring a world-renowned heart surgeon and paying them $800 an hour to put band-aids on paper cuts and sort incoming mail. They will do a great job sorting the mail, but you will go bankrupt by Tuesday.

Teams make this exact mistake when building AI software. They wire their entire application to a massive, expensive frontier reasoning model, and then use that same model to classify simple yes/no user replies, format dates, and extract zip codes. Not only does the cloud bill skyrocket, but users also wait five seconds for a response that a smaller model could generate in 200 milliseconds.

Microsoft Foundry solves this problem in three ways:

1. A Single Catalog of 11,000+ Models: You can run OpenAI (gpt-5, gpt-5-mini), Anthropic (Claude Opus 4.6, Claude Sonnet 4.6 with Zero Data Retention), xAI (Grok 4.0), Meta Llama, Mistral, and Cohere models through the exact same API contract.
2. Model Router: A smart dispatcher deployment that reads each incoming prompt and automatically sends simple questions to fast, low-cost models while escalating complex coding or math prompts to deep reasoning models.
3. Foundry Local: A lightweight runtime that runs models directly on Windows, macOS, Linux, or Windows Server hardware with zero cloud cost and complete offline privacy.

How the Model Router Works Under the Hood

When you deploy Model Router from the Microsoft Foundry portal (Build > Models), it gives you a single deployment name, such as model-router. Your application sends every prompt to that one deployment name.

Incoming User Prompt
        |
        v
[ Foundry Model Router ] (Analyzes complexity in ~15ms)
        |
        +---> Simple classification / summary ---> [ Fast Mini Model ] (Low cost, ~200ms)
        |
        +---> Multi-step coding / deep logic  ---> [ Frontier Reasoning Model ] (High accuracy)

You do not need to write if/else routing rules in your C# or Python code. The router inspects the prompt complexity and picks the right model automatically, cutting token costs by 40% to 60% on mixed traffic.

Enforcing Reliable JSON with Structured Outputs

Before we look at code, there is one feature every developer needs when calling models in production: Structured Outputs.

If you want to send a model's output into a SQL database or an API, you cannot rely on a prompt that says "Please reply in valid JSON." Eventually, the model will add extra commentary like "Sure, here is your JSON:" and crash your parser. Structured Outputs locks the model's token generation to a strict JSON schema so it is mathematically impossible for the response to violate your C# class or Python Pydantic model.

C# (.NET) Example: Calling Models with Strict JSON Schemas

using System;
using System.Text.Json;
using Azure.Identity;
using Azure.AI.Projects;
using OpenAI.Responses;

#pragma warning disable OPENAI001

var projectEndpoint = "https://your-resource.services.ai.azure.com/api/projects/your-project";

AIProjectClient projectClient = new(
    endpoint: new Uri(projectEndpoint),
    tokenProvider: new DefaultAzureCredential());

// Get the standard OpenAI Responses client from your Foundry project
OpenAIResponseClient responseClient = projectClient.ProjectOpenAIClient
    .GetProjectResponsesClientForModel("model-router");

// Define the exact JSON Schema the model must follow
BinaryData ticketSchema = BinaryData.FromString("""
{
    "type": "object",
    "properties": {
        "severity": { "type": "string", "enum": ["Low", "Medium", "High", "Critical"] },
        "department": { "type": "string" },
        "summary": { "type": "string" }
    },
    "required": ["severity", "department", "summary"],
    "additionalProperties": false
}
""");

ResponseCreationOptions options = new()
{
    TextOptions = new ResponseTextOptions
    {
        TextFormat = ResponseTextFormat.CreateJsonSchemaFormat(
            jsonSchemaFormatName: "support_ticket",
            jsonSchema: ticketSchema,
            jsonSchemaIsStrict: true)
    }
};

ResponseResult response = responseClient.CreateResponse(
    "Customer reports that checkout payments are failing for all EU credit cards since 09:00 UTC.",
    options);

Console.WriteLine(response.GetOutputText());
// Guaranteed output shape:
// {"severity":"Critical","department":"Payments Engineering","summary":"Checkout payments failing for all EU credit cards since 09:00 UTC."}

Python Example: Model Router + Pydantic Structured Outputs

from pydantic import BaseModel
from typing import Literal
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

PROJECT_ENDPOINT = "https://your-resource.services.ai.azure.com/api/projects/your-project"

class SupportTicket(BaseModel):
    severity: Literal["Low", "Medium", "High", "Critical"]
    department: str
    summary: str

project = AIProjectClient(
    endpoint=PROJECT_ENDPOINT,
    credential=DefaultAzureCredential(),
)

openai_client = project.get_openai_client()

# Call the Model Router deployment and enforce the Pydantic schema
response = openai_client.responses.parse(
    model="model-router",
    input="Customer reports that checkout payments are failing for all EU credit cards since 09:00 UTC.",
    text_format=SupportTicket,
)

ticket: SupportTicket = response.output_parsed
print(f"Severity: {ticket.severity} | Dept: {ticket.department}")
print(f"Summary: {ticket.summary}")

Running Offline at the Edge with Foundry Local

What if you are building software for a hospital, a factory floor, or a developer laptop that has no internet connection or strict data sovereignty rules? Or what if you want your developers to build and test AI features locally without spending a dime on cloud tokens?

That is where Foundry Local comes in. It installs a local CLI and local inference server (ONNX Runtime GenAI and vLLM with automatic GPU tuning) right on your machine.

Step 1: Start a Model Locally in Two Commands

Open your terminal on Windows, macOS, or Linux:

# List models optimized for your machine's GPU/CPU
foundry model list

# Download and start Phi-4-mini or Qwen3 on a local OpenAI-compatible port
foundry model run phi-4-mini

Step 2: Point Your Existing Code to Foundry Local

Because Foundry Local exposes a standard OpenAI v1 endpoint on localhost, you can use the exact same OpenAI SDK by changing only the base_url:

from foundry_local import FoundryLocalManager
from openai import OpenAI

# Automatically starts the local service and downloads the model if needed
manager = FoundryLocalManager("phi-4-mini")

client = OpenAI(
    base_url=manager.endpoint,
    api_key=manager.api_key  # Local placeholder token
)

response = client.chat.completions.create(
    model=manager.get_model_info("phi-4-mini").id,
    messages=[{"role": "user", "content": "Explain TCP three-way handshake in 3 bullet points."}]
)

print(response.choices[0].message.content)

Choosing Your Cloud Deployment SKU: Standard vs. PTU vs. Data Zones

When you move from local testing to deploying models in the Microsoft Foundry cloud portal, you have to pick a deployment type. Picking the wrong one can cause unexpected throttling or compliance violations:

Architect Tip for Anthropic Claude Models in Foundry: If you deploy Claude models in Microsoft Foundry for regulated workloads (finance, healthcare, legal), enable Zero Data Retention (ZDR) on your deployment so prompts and completions are never retained after the API call finishes.