Tracking Tool Calls Made by AI Agents in Enterprise Applications

AI agents often work by calling external tools. An agent might query a database, read application logs, create a support ticket, call an API, or retrieve information from a cloud service.

In a small application, tracking these operations may seem straightforward. However, enterprise applications can have multiple agents, dozens of tools, and several connected services. A single user request may trigger many tool calls before the agent produces a response.

If something goes wrong, developers need to know which tool was called, what input was provided, how long it took, and whether it succeeded.

Without this information, troubleshooting becomes difficult. A slow response could be caused by the AI model, an external API, a database query, or repeated tool calls.

This article explains how to track AI agent tool calls in an ASP.NET Core application using C#, structured logging, and a consistent execution model.

Why Tool Call Tracking Matters

Consider an AI assistant used by an enterprise support team.

A user asks:

"Find the latest payment failure for customer 1052 and create a support ticket."

To complete this request, the agent might:

1. Call a customer lookup tool.
2. Query the payment service.
3. Retrieve the latest failed transaction.
4. Call a support ticket API.
5. Return the ticket reference to the user.

The entire process may look like one operation to the user, but several independent tool calls happen behind the scenes.

If the ticket is not created, developers need to identify the exact failure.

Possible problems include:

• The customer lookup API returned an error.
• The payment service timed out.
• The agent selected an incorrect transaction.
• The ticket API rejected the request.
• The same tool was called repeatedly.
• An external service returned an unexpected response.

Tool call tracking provides the information needed to investigate these problems.

It also helps teams measure execution time, identify expensive integrations, monitor failure rates, and detect unexpected agent behavior.

Understand What Should Be Tracked

A useful tracking system should capture more than the tool name and execution status.

Each tool call should have a unique identifier and enough context to connect it with the original user request.

The following fields provide a practical starting point:

For example, a payment lookup and a ticket creation call should have separate tool call IDs but share the same conversation and correlation IDs.

This makes it possible to investigate the complete execution flow without confusing individual operations.

Do not automatically store every tool input and output. These values may contain customer information, credentials, personal data, or confidential business records. Store only the information needed for troubleshooting, and apply appropriate redaction and access controls.

Create a Tool Call Data Model

Let's implement a tracking system using ASP.NET Core and Entity Framework Core.

Start by defining the execution status:

public enum ToolCallStatus
{
    Started = 0,
    Completed = 1,
    Failed = 2,
    TimedOut = 3,
    Cancelled = 4
}

Next, create an entity to store tool execution details:

public class AgentToolCall
{
    public Guid Id { get; set; }

    public string AgentId { get; set; } = string.Empty;

    public string ConversationId { get; set; } = string.Empty;

    public string CorrelationId { get; set; } = string.Empty;

    public string ToolName { get; set; } = string.Empty;

    public ToolCallStatus Status { get; set; }

    public DateTime StartedAtUtc { get; set; }

    public DateTime? CompletedAtUtc { get; set; }

    public long? DurationMs { get; set; }

    public int RetryCount { get; set; }

    public string? ErrorType { get; set; }

    public string? ErrorMessage { get; set; }
}

The Id property identifies an individual tool call. The CorrelationId connects the call to other operations within the same request.

For example, if a user request triggers three different tools, each tool call gets a separate ID while sharing the same correlation ID.

Register the entity in the database context:

public class AppDbContext : DbContext
{
    public AppDbContext(
        DbContextOptions<AppDbContext> options)
        : base(options)
    {
    }

    public DbSet<AgentToolCall> AgentToolCalls =>
        Set<AgentToolCall>();

    protected override void OnModelCreating(
        ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<AgentToolCall>()
            .HasIndex(x => x.CorrelationId);

        modelBuilder.Entity<AgentToolCall>()
            .HasIndex(x => new
            {
                x.AgentId,
                x.StartedAtUtc
            });
    }
}

These indexes help when retrieving execution history for a conversation or reviewing activity from a particular agent.

For a high-volume application, consider table partitioning or a dedicated observability store rather than keeping every execution record in the primary application database indefinitely.

Build a Tool Call Tracking Service

Instead of writing tracking logic separately for every tool, create a reusable service.

The service should record when execution starts and update the record when the operation completes or fails.

public class ToolCallTracker
{
    private readonly AppDbContext _context;

    public ToolCallTracker(AppDbContext context)
    {
        _context = context;
    }

    public async Task<Guid> StartAsync(
        string agentId,
        string conversationId,
        string correlationId,
        string toolName,
        int retryCount,
        CancellationToken cancellationToken)
    {
        var toolCall = new AgentToolCall
        {
            Id = Guid.NewGuid(),
            AgentId = agentId,
            ConversationId = conversationId,
            CorrelationId = correlationId,
            ToolName = toolName,
            Status = ToolCallStatus.Started,
            StartedAtUtc = DateTime.UtcNow,
            RetryCount = retryCount
        };

        _context.AgentToolCalls.Add(toolCall);

        await _context.SaveChangesAsync(
            cancellationToken);

        return toolCall.Id;
    }

    public async Task CompleteAsync(
        Guid toolCallId,
        ToolCallStatus status,
        string? errorType,
        string? errorMessage,
        CancellationToken cancellationToken)
    {
        var toolCall = await _context.AgentToolCalls
            .FirstOrDefaultAsync(
                x => x.Id == toolCallId,
                cancellationToken);

        if (toolCall == null)
        {
            return;
        }

        var completedAt = DateTime.UtcNow;

        toolCall.Status = status;
        toolCall.CompletedAtUtc = completedAt;
        toolCall.DurationMs = (long)(
            completedAt - toolCall.StartedAtUtc
        ).TotalMilliseconds;

        toolCall.ErrorType = errorType;
        toolCall.ErrorMessage = errorMessage;

        await _context.SaveChangesAsync(
            cancellationToken);
    }
}

The service creates a record before executing the tool and updates it after execution.

Using a separate tracking service keeps monitoring concerns out of individual tool implementations.

In a production system, avoid creating a new database context concurrently with another operation on the same context. Also consider what should happen if the tracking database is unavailable. A durable telemetry queue or independent observability pipeline can prevent monitoring failures from silently interrupting important business operations.

Track Tool Execution Automatically

Now create a common interface for tools.

public interface IAgentTool
{
    string Name { get; }

    Task<string> ExecuteAsync(
        string input,
        CancellationToken cancellationToken);
}

For example, implement a payment lookup tool:

public class PaymentLookupTool : IAgentTool
{
    public string Name => "PaymentLookup";

    public async Task<string> ExecuteAsync(
        string input,
        CancellationToken cancellationToken)
    {
        // Call the payment service here.

        await Task.Delay(
            100,
            cancellationToken);

        return "Payment lookup completed.";
    }
}

Instead of adding tracking code inside every tool, use a wrapper that tracks execution around the actual tool call.

public class TrackedAgentTool
{
    private readonly IAgentTool _tool;
    private readonly ToolCallTracker _tracker;
    private readonly ILogger<TrackedAgentTool> _logger;

    public TrackedAgentTool(
        IAgentTool tool,
        ToolCallTracker tracker,
        ILogger<TrackedAgentTool> logger)
    {
        _tool = tool;
        _tracker = tracker;
        _logger = logger;
    }

    public async Task<string> ExecuteAsync(
        string agentId,
        string conversationId,
        string correlationId,
        string input,
        CancellationToken cancellationToken)
    {
        var toolCallId = await _tracker.StartAsync(
            agentId,
            conversationId,
            correlationId,
            _tool.Name,
            retryCount: 0,
            cancellationToken);

        try
        {
            var result = await _tool.ExecuteAsync(
                input,
                cancellationToken);

            await _tracker.CompleteAsync(
                toolCallId,
                ToolCallStatus.Completed,
                null,
                null,
                cancellationToken);

            return result;
        }
        catch (OperationCanceledException)
            when (cancellationToken.IsCancellationRequested)
        {
            await _tracker.CompleteAsync(
                toolCallId,
                ToolCallStatus.Cancelled,
                "OperationCancelled",
                null,
                CancellationToken.None);

            throw;
        }
        catch (TimeoutException ex)
        {
            await _tracker.CompleteAsync(
                toolCallId,
                ToolCallStatus.TimedOut,
                "Timeout",
                "The tool execution timed out.",
                CancellationToken.None);

            _logger.LogWarning(
                ex,
                "Tool call {ToolCallId} timed out.",
                toolCallId);

            throw;
        }
        catch (Exception ex)
        {
            await _tracker.CompleteAsync(
                toolCallId,
                ToolCallStatus.Failed,
                ex.GetType().Name,
                "Tool execution failed.",
                CancellationToken.None);

            _logger.LogError(
                ex,
                "Tool call {ToolCallId} failed.",
                toolCallId);

            throw;
        }
    }
}

This wrapper ensures that tool executions follow the same tracking process.

Notice that the example does not store the original input or the complete exception message. In enterprise applications, these values may expose sensitive information.

The example also treats cancellation requested by the caller as a cancelled operation. A timeout should be classified separately when the application uses an explicit timeout mechanism.

For production use, tracking updates should be protected against duplicate completion attempts and should not accidentally replace the original execution result.

Measure Execution Time and Identify Slow Tools

Tracking individual calls becomes more useful when the data can be analyzed.

For example, a support assistant may use several tools:

• CustomerLookup
• PaymentLookup
• TicketCreation
• NotificationService

After collecting execution records, developers can calculate average execution time and failure rates.

A basic query using Entity Framework Core can retrieve performance information:

var report = await context.AgentToolCalls
    .Where(x => x.StartedAtUtc >=
        DateTime.UtcNow.AddDays(-7))
    .GroupBy(x => x.ToolName)
    .Select(group => new
    {
        ToolName = group.Key,

        TotalCalls = group.Count(),

        FailedCalls = group.Count(
            x => x.Status == ToolCallStatus.Failed),

        AverageDurationMs = group
            .Where(x => x.DurationMs.HasValue)
            .Average(x => (double?)x.DurationMs)
    })
    .ToListAsync();

This report provides a basic overview of tool performance over the previous seven days.

For example:

The numbers are illustrative.

The report suggests that PaymentLookup deserves further investigation because it has the highest average execution time and failure count.

However, averages do not tell the whole story. Production monitoring should also measure percentiles such as p95 and p99 latency. A tool might usually respond quickly but occasionally take several seconds.

Connect Tool Calls With Distributed Tracing

Database records are useful for historical analysis, but they are not always the best way to investigate a request while it is running.

Distributed tracing provides another layer of visibility.

Using OpenTelemetry, developers can create a trace for an agent request and a separate span for each tool call.

A simplified example looks like this:

using System.Diagnostics;

public class ToolExecution
{
    private static readonly ActivitySource ActivitySource =
        new("Enterprise.AgentTools");

    public async Task<string> ExecuteAsync(
        IAgentTool tool,
        string input,
        CancellationToken cancellationToken)
    {
        using var activity = ActivitySource.StartActivity(
            $"AgentTool.{tool.Name}",
            ActivityKind.Internal);

        activity?.SetTag(
            "agent.tool.name",
            tool.Name);

        try
        {
            var result = await tool.ExecuteAsync(
                input,
                cancellationToken);

            activity?.SetStatus(ActivityStatusCode.Ok);

            return result;
        }
        catch (Exception ex)
        {
            activity?.SetStatus(
                ActivityStatusCode.Error);

            activity?.SetTag(
                "error.type",
                ex.GetType().Name);

            throw;
        }
    }
}

Register the activity source with the application's OpenTelemetry tracing configuration and export traces to your observability platform.

A distributed trace can show the relationship between the original HTTP request, the AI agent operation, and the external tool calls.

For example, if a request takes five seconds, the trace may reveal that four seconds were spent waiting for the payment API.

This helps distinguish model latency from tool execution latency.

Best Practices for Enterprise Applications

A tracking system should provide useful operational information without creating additional security or performance problems.

Use a correlation ID for every agent request

Generate or propagate a correlation ID when an agent request begins. Pass it through tool calls and downstream service requests wherever possible.

Keep tracking separate from business execution

A tracking failure should not automatically cause a successful business operation to be repeated. Define a clear policy for telemetry failures and use durable delivery when audit records are mandatory.

Protect sensitive tool data

Do not log API keys, access tokens, passwords, customer payment details, or unrestricted tool payloads. Apply redaction before data reaches logs or monitoring systems.

Track retries separately

Each execution attempt should have its own tool call record or attempt record. This makes it possible to distinguish an original failure from a successful retry.

Set retention policies

Tool call records can grow quickly. Define how long detailed records must be retained and whether older records should be archived.

Monitor unusual agent behavior

Repeated calls to the same tool, unexpected tool selection, unusual execution volume, and repeated failures can indicate configuration problems or unsafe agent behavior.

Use access controls for monitoring data

Only authorized developers, operators, and auditors should be able to inspect sensitive execution details.

Common Mistakes to Avoid

One common mistake is tracking only failed tool calls. Successful calls are also important because they provide the baseline needed to identify performance changes.

Another mistake is storing complete request and response payloads without considering data privacy. This can expose confidential information and increase storage costs.

Developers also sometimes treat every tool call as an independent operation. Without conversation IDs and correlation IDs, it becomes difficult to reconstruct the sequence of actions that produced a result.

Finally, logging an exception without updating the tool call status can leave records permanently marked as started. Use reliable completion handling and a recovery process to identify executions that stopped unexpectedly.

Summary

AI agent tool calls need the same operational visibility as other enterprise application integrations.

A reusable tracking service can record execution details, measure latency, identify failures, and connect related operations through correlation IDs.

For production applications, combine structured tracking records with distributed tracing, secure logging, retry monitoring, and appropriate retention policies.

The goal is not to record everything an AI agent does. It is to capture enough reliable information to understand its decisions, investigate failures, and maintain control over the tools it uses.