Amazon CloudWatch Omni: How to Monitor AI Agents and Cloud Apps Together

AI agents introduce a different observability challenge compared with traditional applications.

A normal application may have a relatively predictable request path:

User
  |
  v
API
  |
  v
Database
  |
  v
Response

An AI agent can execute a much longer workflow:

User
  |
  v
Agent
  |
  +--> LLM
  |
  +--> Tool
  |
  +--> Database
  |
  +--> External API
  |
  +--> Another model
  |
  v
Final Response

When something becomes slow or fails, developers need to determine which part of the workflow caused the problem.

Was the model slow?

Did a tool timeout?

Was the database query expensive?

Did the agent retry an operation?

Was a cloud service unavailable?

This is where centralized observability becomes important.

Amazon CloudWatch provides monitoring and observability capabilities across AWS workloads. CloudWatch cross-account observability, often associated with CloudWatch's broader observability capabilities, can help teams bring telemetry from multiple AWS accounts and applications into a unified monitoring experience.

For AI agents, the same principle can be extended across the infrastructure supporting the agent: applications, compute resources, APIs, databases, queues, and model-related services.

This article explains how to design an observability architecture for AI agents using Amazon CloudWatch and how to connect agent-level telemetry with traditional cloud monitoring.

Why AI Agent Observability Is Different

Traditional application monitoring often focuses on metrics such as:

• CPU utilization

• Memory usage

• Request latency

• HTTP errors

• Database latency

These metrics remain important for AI applications, but they are not enough.

An AI agent introduces additional dimensions:

• Model latency

• Token consumption

• Tool execution time

• Number of agent steps

• Tool failures

• Retry count

• Prompt size

• Output size

• Workflow completion rate

• Human approval time

A useful agent trace might look like:

Agent Request
      |
      +---- LLM Call: 420 ms
      |
      +---- Search Tool: 180 ms
      |
      +---- Database Query: 75 ms
      |
      +---- LLM Call: 650 ms
      |
      +---- Email Tool: 230 ms
      |
      v
Completed

Without this information, developers may see only:

POST /api/agent -> 1.8 seconds

That does not explain why the request took 1.8 seconds.

A Unified CloudWatch Architecture

A practical architecture can connect application telemetry with infrastructure telemetry:

                    User
                      |
                      v
              +---------------+
              | Agent App      |
              +---------------+
                      |
             +--------+--------+
             |        |        |
             v        v        v
           LLM      Tools    Database
             |        |        |
             +--------+--------+
                      |
                      v
              CloudWatch
             /     |      \
            /      |       \
        Logs     Metrics    Traces
            \      |       /
             \     |      /
              +----+-----+
                   |
                   v
             Dashboards
              & Alerts

The goal is to correlate application-level agent behavior with infrastructure-level health.

What Should Be Monitored?

A production AI agent should expose telemetry at several levels.

This layered approach makes troubleshooting much easier.

CloudWatch Logs for Agent Activity

Logs are useful for capturing discrete events.

For example:

logger.LogInformation(
    "Agent started. RequestId={RequestId}, Agent={AgentName}",
    requestId,
    agentName);

When a tool starts:

logger.LogInformation(
    "Agent tool started. RequestId={RequestId}, Tool={ToolName}",
    requestId,
    toolName);

And when it completes:

logger.LogInformation(
    "Agent tool completed. RequestId={RequestId}, Tool={ToolName}, DurationMs={DurationMs}",
    requestId,
    toolName,
    stopwatch.ElapsedMilliseconds);

Structured logging is preferable to unstructured messages because fields such as RequestId, ToolName, and DurationMs can later be queried and aggregated.

A useful log event might contain:

{
  "requestId": "req-4821",
  "agent": "OrderAssistant",
  "tool": "OrderSearch",
  "durationMs": 182,
  "status": "success"
}

Avoid putting prompts, credentials, access tokens, or sensitive customer information into logs.

Metrics Are Better for Trends

Logs explain individual events.

Metrics help answer questions about the system as a whole.

For an AI agent, useful custom metrics include:

AgentRequests
AgentFailures
AgentDuration
ToolDuration
ToolFailures
AgentRetries
TokensConsumed

For example:

AgentRequests = 12,400
AgentFailures = 183
Failure Rate = 1.48%

The exact calculation can be performed by the monitoring platform or application analytics layer.

The important part is to emit consistent measurements.

Measuring Agent Step Duration

Consider an agent that executes five steps:

Step 1: Understand request
Step 2: Search documents
Step 3: Query database
Step 4: Call external API
Step 5: Generate response

The application can record duration for each step.

var stopwatch = Stopwatch.StartNew();

try
{
    await ExecuteToolAsync(cancellationToken);
}
finally
{
    stopwatch.Stop();

    logger.LogInformation(
        "Tool execution completed. Tool={Tool}, DurationMs={DurationMs}",
        toolName,
        stopwatch.ElapsedMilliseconds);
}

This makes it possible to identify the slowest part of the workflow.

For example:

Document Search     120 ms
Database Query       80 ms
External API        950 ms
LLM Generation      700 ms

The external API would deserve investigation before optimizing the database query.

Distributed Tracing

Logs and metrics become significantly more useful when requests can be correlated.

Consider:

Request ID: 7f29

Agent
 |
 +-- LLM Call
 |
 +-- Search Tool
 |     |
 |     +-- Database
 |
 +-- Email Tool
       |
       +-- Email API

Distributed tracing can represent these operations as related spans.

Conceptually:

Agent Request
├── Model Call
├── Search Tool
│   └── Database Query
└── Email Tool
    └── External API

This allows engineers to move from an overall slow request to the specific operation responsible for the delay.

For .NET applications, OpenTelemetry can be used to instrument application code and export telemetry to supported observability systems.

Correlation IDs

A simple correlation strategy is essential.

Generate or propagate a request identifier:

var requestId = Activity.Current?.TraceId.ToString()
                ?? Guid.NewGuid().ToString();

Then include it in logs and relevant application events.

For example:

RequestId: 9e13

Agent started
Tool started: SearchOrders
Tool completed: SearchOrders
LLM call started
LLM call completed
Agent completed

Now a developer can search for one identifier rather than trying to reconstruct a workflow from thousands of unrelated log messages.

Monitoring Tool Calls

Tools are one of the most important areas to monitor because agents can invoke them dynamically.

Suppose an agent has:

SearchOrders
GetCustomer
GetInventory
SendEmail
CreateTicket

Track each tool independently.

This immediately highlights unusual behavior.

For example, a sudden increase in GetInventory latency may explain an increase in overall agent latency.

Monitoring Agent Retries

Agents can produce repeated tool calls because of failures, transient errors, or workflow behavior.

Track retries explicitly.

Agent Requests
     |
     +---- First attempt
     |
     +---- Retry
     |
     +---- Retry
     |
     v
Final Result

A high retry rate can increase:

• Latency

• Infrastructure usage

• API traffic

• Model calls

• Cost

A useful metric is:

Retry Rate =
Number of Retried Operations
/
Total Operations

Do not assume retries are always bad. They can be valuable for transient failures. The important issue is whether retry behavior is controlled and justified.

Monitoring Model Calls

AI agents may invoke a model multiple times during one workflow.

For example:

User Request
    |
    v
LLM Call 1 -> Planning
    |
    v
Tool
    |
    v
LLM Call 2 -> Interpretation
    |
    v
Tool
    |
    v
LLM Call 3 -> Final Response

A single user request therefore does not necessarily equal one model request.

Monitor:

• Number of model calls per workflow

• Model latency

• Input tokens

• Output tokens

• Errors

• Timeouts

• Model selection

This can reveal unexpectedly expensive workflows.

Connecting Logs, Metrics, and Traces

The three major observability signals have different purposes.

Logs
 |
 +--> What happened?

Metrics
 |
 +--> How often is it happening?

Traces
 |
 +--> Where did it happen in the request?

For example:

Metric

Agent p95 latency increased from 900 ms to 2.1 s.

Trace

External API span increased from 150 ms to 1.3 s.

Log

External API timeout occurred for endpoint X.

Together, these signals provide a much clearer diagnosis than any one of them alone.

CloudWatch Dashboards for AI Agents

A dedicated dashboard can provide a high-level view of the agent platform.

For example:

+------------------------------------------------+
| AI Agent Operations                            |
+------------------------------------------------+
| Requests       | Error Rate | p95 Latency      |
| 125K           | 1.2%       | 1.8 sec          |
+------------------------------------------------+
| Tool Failures  | Retries    | Active Workflows  |
| 842            | 2.4%       | 1,240            |
+------------------------------------------------+
|                Agent Latency                  |
|      Requests / Latency over time             |
+------------------------------------------------+
|                Tool Performance               |
| Search | Database | API | Email               |
+------------------------------------------------+

The dashboard should focus on operational questions rather than showing every available metric.

Alerts

Monitoring becomes useful when important changes trigger action.

Examples include:

Agent error rate > threshold
Tool latency > threshold
Model timeout rate > threshold
Database errors > threshold
Agent retry rate > threshold

For example:

IF
AgentErrorRate > 5%

THEN
Create operational alert

Thresholds should be based on normal workload behavior.

A threshold that is too sensitive can generate alert fatigue.

Multi-Account Observability

Large AWS environments often separate workloads across multiple accounts.

For example:

Production Account
      |
      +---- Agent API
      +---- Database

AI Account
      |
      +---- Model Infrastructure

Data Account
      |
      +---- Analytics

Security Account
      |
      +---- Monitoring

Centralized observability can help teams inspect telemetry across these environments without requiring engineers to manually switch between every account.

This becomes particularly useful when an agent spans multiple AWS services and accounts.

For example:

Agent API
   |
   v
Account A
   |
   +----> Account B: Model
   |
   +----> Account C: Data
   |
   +----> Account D: External Integration

A cross-account observability strategy can provide a more complete operational picture.

Security and Privacy

AI telemetry can contain sensitive information.

A careless logging strategy might capture:

User Prompt
Customer Data
Access Token
API Key
Database Result

This creates a significant security risk.

Instead, logs should contain operational metadata where possible:

{
  "requestId": "req-4821",
  "agent": "OrderAssistant",
  "tool": "OrderSearch",
  "status": "success",
  "durationMs": 182
}

Avoid logging full prompts or tool responses unless there is a specific, controlled requirement.

When sensitive information must be captured for troubleshooting, apply appropriate redaction, access controls, retention policies, and encryption.

Cost Observability

Observability should also include cost-related signals.

For an AI agent, cost can increase because of:

• More model calls

• Larger prompts

• Longer outputs

• Excessive retries

• Inefficient tool workflows

• Increased infrastructure usage

Consider tracking:

Requests per workflow
Model calls per request
Tokens per request
Retries per request
Tool calls per request

A sudden increase in average model calls may indicate an agent workflow regression.

For example:

Before:
1.8 model calls / request

After:
4.7 model calls / request

That change deserves investigation even if the application still appears functional.

A Practical .NET Observability Pattern

A reusable telemetry service can keep instrumentation consistent.

public interface IAgentTelemetry
{
    void AgentStarted(string requestId, string agent);

    void ToolStarted(
        string requestId,
        string tool);

    void ToolCompleted(
        string requestId,
        string tool,
        long durationMs);

    void AgentCompleted(
        string requestId,
        long durationMs);

    void AgentFailed(
        string requestId,
        Exception exception);
}

The agent service can then use it:

var stopwatch = Stopwatch.StartNew();

telemetry.AgentStarted(
    requestId,
    "OrderAssistant");

try
{
    await ExecuteWorkflowAsync(
        request,
        cancellationToken);

    telemetry.AgentCompleted(
        requestId,
        stopwatch.ElapsedMilliseconds);
}
catch (Exception ex)
{
    telemetry.AgentFailed(
        requestId,
        ex);

    throw;
}

This keeps observability concerns from spreading throughout every component.

Common Mistakes

Logging the Entire Prompt

This can expose sensitive information and increase log volume.

Log metadata instead unless the actual prompt is specifically required for controlled debugging.

Monitoring Only Infrastructure

CPU and memory can look healthy while the agent is producing slow or incorrect workflows.

Monitor agent behavior as well as infrastructure.

Ignoring Tool Latency

A fast model does not help if the agent spends several seconds waiting for an external API.

Measure each tool separately.

Creating Too Many Metrics

Every metric should have a purpose.

Avoid creating hundreds of metrics that nobody uses.

Start with signals that support operational decisions.

No Correlation Between Signals

A metric saying that latency increased is useful.

A trace and correlated log explaining why it increased are much more useful.

Alerting on Everything

Not every error deserves an immediate page.

Separate informational events from actionable operational failures.

Best Practices

When monitoring AI agents with CloudWatch and related AWS observability capabilities:

1. Instrument the entire agent workflow.

2. Track model, tool, application, and infrastructure latency separately.

3. Use correlation IDs or trace context consistently.

4. Prefer structured logs.

5. Monitor agent-specific metrics in addition to infrastructure metrics.

6. Track retries and repeated model calls.

7. Monitor tool-level failure rates.

8. Measure p50, p95, and p99 latency.

9. Create dashboards around operational questions.

10. Keep sensitive prompt and tool data out of logs where possible.

11. Use centralized observability for multi-account environments.

12. Create alerts around actionable thresholds.

13. Monitor token and workflow behavior for cost anomalies.

14. Use distributed tracing to connect application and infrastructure behavior.

Conclusion

AI agents make cloud observability more complex because one user request can trigger many model calls, tools, APIs, databases, and infrastructure components.

Amazon CloudWatch can provide an important observability layer for AWS-based AI applications by bringing together logs, metrics, dashboards, alarms, and related telemetry. Cross-account observability can be particularly useful when production systems are distributed across multiple AWS accounts.

The key is to monitor the agent as a workflow rather than treating it as a single API request.

A useful mental model is:

User Request
     |
     v
Agent Workflow
     |
     +---- Model
     |
     +---- Tool
     |
     +---- Database
     |
     +---- API
     |
     v
Final Response

Every important stage should be observable.

When logs, metrics, and traces are connected through consistent request or trace identifiers, developers can move from a high-level symptom such as increased latency to the specific model call, tool, database query, or infrastructure component responsible for the problem.

Summary

Monitoring AI agents requires more than traditional cloud metrics. Teams need visibility into agent workflows, model calls, tool execution, retries, latency, errors, and resource usage.

CloudWatch can serve as a central observability layer for AWS workloads, while structured application telemetry connects agent behavior with the underlying infrastructure. A well-designed monitoring strategy should provide enough information to answer three questions quickly:

What happened?
How often is it happening?
Where is the problem occurring?

Once those questions can be answered reliably, AI agent systems become significantly easier to operate, troubleshoot, secure, and optimize in production.