AzureFeeds

Introduction

We’ve entered an era where AI agents autonomously invoke tools — reading and writing files, calling APIs, querying databases. Convenient as this is, without a mechanism to control who can call what, and under what conditions, you can’t put it into production. The Agent Governance Toolkit (AGT), open-sourced by Microsoft, is exactly the toolkit for embedding that “gatekeeper” into AI agents. This article walks through getting started with AGT in .NET (C#), based on the following repository:

https://github.com/normalian/MyAGTSamples/tree/main/AGTAuditBlobTelemetryApp01

This sample walks through implementation examples focused on production-oriented audit log collection and telemetry integration. For basic usage of things like Policies, please refer to the following repository:

https://github.com/microsoft/agent-governance-toolkit

In environments where AI agents make important decisions and execute tools, recording an audit trail of “what happened” and “when it happened” is critical. AGT can export governance events to external systems, integrating seamlessly with production infrastructure such as Azure Blob Storage and Application Insights.

The Importance of Auditing and Telemetry in Production

Meeting Regulatory Requirements

In industries like finance and healthcare, an audit trail of an AI system’s decision-making process and execution details is often legally required.

Anomaly Detection and Post-Incident Response

By analyzing audit logs, you can detect signs of anomalous agent behavior and respond quickly.

Performance Analysis

Collecting telemetry data lets you identify latency bottlenecks and failure patterns in agent execution.

Compliance Reporting

Audit trails can serve as evidence for security audits and compliance reviews.

Overview of the AGTAuditBlobTelemetryApp01 Project

This sample project builds on the previous sample (AGTIdentityWithMAFApp02), adding the following:

Project Structure

AGTAuditBlobTelemetryApp01/
│ AGTAuditBlobTelemetryApp01.csproj
│ AppConfiguration.cs
│ BlobAuditSink.cs
│ Program.cs
│ README.md
└───policies
default.yaml

Architecture Overview

Data Flow

1. Microsoft Agent Framework agent execution: a tool call occurs


2. Governance decision: the GovernanceKernel decides allow/deny based on policy


3. Event firing: a GovernanceEvent is generated in memory


4. Audit output (path ①): BlobAuditSink appends the log to Blob in JSON format


5. Telemetry output (path ②): a span is recorded via ActivitySource and exported to Application Insights

Key Components Explained

1. The BlobAuditSink Class

A component that records audit logs to an Append Blob in Azure Blob Storage.

internal sealed class BlobAuditSink : IDisposable
{
private readonly AppendBlobClient _appendBlobClient;
private readonly SemaphoreSlim _gate = new(1, 1);
private bool _initialized;

// TODO: Use Managed Identity in production
public BlobAuditSink(string storageAccountUri, AzureCliCredential credential,
string containerName, string blobName)
{
// Initialize the Blob container client
var serviceClient = new BlobServiceClient(new Uri(storageAccountUri), credential);
var containerClient = serviceClient.GetBlobContainerClient(containerName);
containerClient.CreateIfNotExists();
_appendBlobClient = containerClient.GetAppendBlobClient(blobName);
}

public void Append(GovernanceEvent governanceEvent)
{
// Append the governance event to the Blob in JSON format
var record = new GovernanceAuditRecord(
governanceEvent.EventId,
governanceEvent.Timestamp,
governanceEvent.Type.ToString(),
governanceEvent.AgentId,
governanceEvent.SessionId,
governanceEvent.PolicyName,
new Dictionary(governanceEvent.Data));

var payload = JsonSerializer.Serialize(record) + Environment.NewLine;
var bytes = Encoding.UTF8.GetBytes(payload);

_gate.Wait(); // Thread safety: only one thread may append to the Blob at a time
try
{
EnsureBlobExists();
using var stream = new MemoryStream(bytes, writable: false);
_appendBlobClient.AppendBlock(stream);
}
finally
{
_gate.Release();
}
}
}

Key Points

• SemaphoreSlim: Controls concurrent access from multiple threads, restricting append operations on the Append Blob to a single thread at a time


• AppendBlobClient: Rather than traditional Upload/Download, this always appends to the end — ideal for log files


• JSONL format: one line = one record, which suits streaming processing and ingestion into spreadsheets. This can also be ingested into data platforms such as Microsoft Fabric.


• ⚠️ Security: Tool arguments may end up containing API keys or personal information. See the “Security Considerations: Sanitizing Audit Logs” section below for details.

2. OpenTelemetry Integration

using var meterProvider = Sdk.CreateMeterProviderBuilder()
.SetResourceBuilder(resourceBuilder)
.AddMeter(GovernanceMetrics.MeterName)
.AddAzureMonitorMetricExporter(options =>
{
options.ConnectionString = applicationInsightsConnectionString;
})
.Build();

using var tracerProvider = Sdk.CreateTracerProviderBuilder()
.SetResourceBuilder(resourceBuilder)
.AddSource(GovernanceMetrics.MeterName)
.AddSource(ActivitySource.Name)
.AddAzureMonitorTraceExporter(options =>
{
options.ConnectionString = applicationInsightsConnectionString;
})
.Build();

Key Points

• MeterProvider: sends metrics from GovernanceMetrics.MeterName (= “agent_governance“) to Azure Monitor


• TracerProvider: records spans generated by ActivitySource into Application Insights


• ResourceBuilder: attaches a service name and version, making it possible to distinguish between multiple agents

3. The Governance Event Audit Hook

kernel.OnAllEvents(evt =>
{
// Create a span: recorded to Application Insights as trace information
using var activity = ActivitySource.StartActivity($”GovernanceEvent.{evt.Type}”,
ActivityKind.Internal);
activity?.SetTag(“event.type”, evt.Type.ToString());
activity?.SetTag(“agent.id”, evt.AgentId);
activity?.SetTag(“policy.name”, evt.PolicyName ?? “(none)”);
activity?.SetTag(“tool.name”, evt.Data.GetValueOrDefault(“tool_name”, “(unknown)”));

// Append the audit log to Blob Storage
auditSink.Append(evt);

// Emit a log entry
logger.LogInformation(
“[Audit -> Blob] {EventType} policy={PolicyName} tool={ToolName} agentId={AgentId}”,
evt.Type,
evt.PolicyName ?? “(none)”,
evt.Data.GetValueOrDefault(“tool_name”, “(unknown)”),
evt.AgentId);
});

Key Points

• OnAllEvents callback: captures every governance event (PolicyCheck, PolicyViolation, ToolCallBlocked, etc.)


• Dual output: written simultaneously to Blob Storage (audit trail) and Application Insights (analytics telemetry)


• Span tags: allow later filtering and grouping within Application Insights

4. Direct Governance Evaluation

A demo that judges allow/deny purely based on policy, without actually executing the tool call.

// Allow example
var allowed = kernel.EvaluateToolCall(
agent.Did,
“GetLocation”,
new Dictionary
{
[“city”] = “London”
});

Console.WriteLine($”GetLocation call allowed? {allowed.Allowed}”);
Console.WriteLine($”GetLocation call reason: {allowed.Reason}”);

// Deny example
var blocked = kernel.EvaluateToolCall(
agent.Did,
“execute_shell”,
new Dictionary
{
[“cmd”] = “rm -rf /”
});

Console.WriteLine($”Blocked call allowed? {blocked.Allowed}”);
Console.WriteLine($”Blocked call reason: {blocked.Reason}”);

Key Points

• Pre-execution decision: can be evaluated before the tool is actually run — efficient for things like log processing


• Detailed reasoning: allowed.Reason lets you see exactly which policy rule produced the decision

Governance Policy Explained

apiVersion: governance.toolkit/v1
version: “1.0”
name: audit-blob-telemetry-policy
default_action: deny

rules:
– name: allow-weather
condition: “tool_name == ‘GetWeather'”
action: allow
priority: 10

– name: allow-time
condition: “tool_name == ‘GetTime'”
action: allow
priority: 10

– name: allow-location
condition: “tool_name == ‘GetLocation'”
action: allow
priority: 10

– name: block-shell
condition: “tool_name == ‘execute_shell'”
action: deny
priority: 100

Policy Strategy

• Default action: deny: any tool not explicitly allowed is denied (a whitelist approach)


• Priority-based rules: higher priority is evaluated first. block-shell has priority 100, giving it an explicit deny

Run the Sample

Prerequisites

Set the following environment variables:

$env:AZURE_OPENAI_ENDPOINT = “https://your-resource.openai.azure.com/”
$env:AZURE_OPENAI_DEPLOYMENT = “gpt-5.4-mini”
$env:APPLICATIONINSIGHTS_CONNECTION_STRING = “InstrumentationKey=…;…”
$env:AZURE_STORAGE_ACCOUNT_NAME = “yourstorageaccount”
$env:AUDIT_STORAGE_CONTAINER = “agt-audit”

Execution Result

==================================================
AGT Audit to Blob + Telemetry to Application Insights
==================================================

Agent DID: did:mesh:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Agent Status: Active
Telemetry: Application Insights via AzureMonitorExporterOptions
Audit: Azure Blob Storage container ‘agt-audit’

✓ OpenTelemetry providers initialized
– Metrics: agent_governance
– Traces: AgentGovernance.Examples.AuditBlobTelemetry
– Logs: AgentGovernance.Examples

=== Agent Run 1 ===
[Audit -> Blob] PolicyCheck policy= tool=GetWeather agentId=did:mesh:…
Role: assistant
FunctionCall: GetWeather({“city”:”Seattle”})
Role: tool
FunctionResult: call_xxx = Seattle is sunny, 22°C
Role: assistant
Text: The weather in Seattle is sunny with a temperature of 22°C.

=== Agent Run 2 ===
[Audit -> Blob] PolicyCheck policy= tool=GetTime agentId=did:mesh:…
Role: assistant
FunctionCall: GetTime({“timezone”:”Tokyo”})
Role: tool
FunctionResult: call_yyy = Current time in Tokyo: 09:30:00 UTC
Role: assistant
Text: The current time in Tokyo is 09:30:00 UTC.

=== Direct Governance Demo – Allow Example ===
GetLocation call allowed? True
GetLocation call reason: Matches allow-location rule

=== Direct Governance Demo – Deny Example ===
Blocked call allowed? False
Blocked call reason: Matches block-shell rule (deny)

=== Flushing Telemetry to Application Insights ===

Flushing metrics…
Flushing traces…
Flushing logs…
Waiting 5 seconds for data transmission…
✓ Telemetry flush complete.

Security Considerations: Sanitizing Audit Logs

In production, tool arguments may end up containing API keys, passwords, or personally identifiable information (PII). Because of this, it’s important to apply masking before writing to Blob.

Masking Strategy:

 Checking Blob Storage Audit Logs

Open the Blob Storage account in the Azure Portal and check the `governance-audit-YYYYMMDD.jsonl` file in the `agt-audit` container.

Example content:

{“EventId”:”evt-42a6f9d598c44d56b15152699ac8eafa”,”Timestamp”:”2026-07-02T01:47:32.0371686+00:00″,”Type”:”PolicyViolation”,”AgentId”:”did:agentmesh:weatheragent”,”SessionId”:”maf-e2ed4a320f944bbf”,”PolicyName”:null,”Data”:{“message”:”What is the weather in Seattle?”,”allowed”:false,”action”:”deny”,”reason”:”No matching rules; default action is deny.”,”evaluation_ms”:37.2383}}
{“EventId”:”evt-7ff65a1c8bd84195a26722adc20ce2bf”,”Timestamp”:”2026-07-02T01:47:41.1239754+00:00″,”Type”:”PolicyViolation”,”AgentId”:”did:agentmesh:weatheragent”,”SessionId”:”maf-9f949b3838054324″,”PolicyName”:null,”Data”:{“message”:”What time is it in Tokyo?”,”allowed”:false,”action”:”deny”,”reason”:”No matching rules; default action is deny.”,”evaluation_ms”:0.1392}}
{“EventId”:”evt-baf03404cd8a477eaaa5e0af93413a74″,”Timestamp”:”2026-07-02T01:47:48.501694+00:00″,”Type”:”PolicyViolation”,”AgentId”:”did:agentmesh:weatheragent”,”SessionId”:”maf-e88d7382ed3b49a0″,”PolicyName”:null,”Data”:{“message”:”Where is Paris located?”,”allowed”:false,”action”:”deny”,”reason”:”No matching rules; default action is deny.”,”evaluation_ms”:0.0919}}
{“EventId”:”evt-107e6c8e18e14fdcb8272844e57c45b1″,”Timestamp”:”2026-07-02T01:47:53.9755476+00:00″,”Type”:”PolicyCheck”,”AgentId”:”did:mesh:4c3ca93fec8b9d78fd05839694ed7df8″,”SessionId”:”session-775002743f0f4d28″,”PolicyName”:”allow-location”,”Data”:{“tool_name”:”GetLocation”,”allowed”:true,”action”:”allow”,”reason”:”Matched rule u0027allow-locationu0027 with action u0027Allowu0027.”,”evaluation_ms”:4.3211,”arguments”:{“city”:”London”}}}
{“EventId”:”evt-6bc7e6b8632a41cb885fc5e87097c6ac”,”Timestamp”:”2026-07-02T01:48:02.4498334+00:00″,”Type”:”ToolCallBlocked”,”AgentId”:”did:mesh:4c3ca93fec8b9d78fd05839694ed7df8″,”SessionId”:”session-bfda81e9ce30429a”,”PolicyName”:”block-shell”,”Data”:{“tool_name”:”execute_shell”,”allowed”:false,”action”:”deny”,”reason”:”Matched rule u0027block-shellu0027 with action u0027Denyu0027.”,”evaluation_ms”:0.065,”arguments”:{“cmd”:”rm -rf /”}}}

How to Use It

KQL Query — Kusto Query Language** — analyze logs like this:

// Aggregate allow/deny counts for metrics under agent_governance.
AppMetrics
| where Name startswith “agent_governance.”
| where TimeGenerated > ago(24h)
| extend Decision = tostring(Properties.decision)
| summarize Count = sum(Sum) by Name, Decision
| order by Name asc, Decision asc

Checking in Application Insights

In the Azure Portal, go to Application Insights → Search → Traces tab to check the agent’s execution performance.

Real-World Usage Scenarios

Scenario 1: Detecting Anomalous Access Patterns

// Raise an alert if a given agent receives many tool denials in a short time window
// Note: since PolicyName can be null, combine a Type-based check with null-safe checks
var recentDenials = auditLogs
.Where(l => l.Timestamp > DateTime.UtcNow.AddMinutes(-5))
.Where(l => l.Type is “PolicyViolation” or “ToolCallBlocked”
|| (l.PolicyName?.Contains(“deny”) ?? false)
|| (l.PolicyName?.Contains(“block”) ?? false))
.GroupBy(l => l.AgentId)
.Where(g => g.Count() > 10);

foreach (var agentGroup in recentDenials)
{
NotifySecurityTeam($”Agent {agentGroup.Key} attempted ” +
$”{agentGroup.Count()} forbidden operations in the last 5 minutes”);
}

Notes:

• Type-based checks: as the JSONL sample shows, default-deny PolicyViolation entries have a PolicyName of null. Making the Type-based check the primary criterion avoids exceptions from missing null checks.


• Null-safe operators: combining PolicyName?.Contains() with ?? false avoids null-reference exceptions while still allowing policy-name-based filtering.

Scenario 2: Agent Behavior Monitoring Dashboard

Build a custom Application Insights dashboard that shows, in real time:

• Agent count: number of currently active agents


• Denial rate: percentage of policy denials over the past hour


• Average response time: average latency of tool calls


• Error rate: percentage of tool calls that raised an exception

Considerations for Production

1. Scalability

At large agent-fleet scale, writes to Blob Storage may become a bottleneck.

Mitigations:

• Hierarchical partitioning with Azure Data Lake Storage (ADLS)


• Batch processing via Event Hubs


• Load-balancing writes across multiple Append Blobs

2. Cost Optimization

Storing large volumes of logs in Blob Storage increases storage costs.

Mitigations:

• Automatic deletion of old logs (lifecycle policies)


• Staged tiering from Hot → Cool → Archive


• Storing only the information you actually need (filtering)

3. Performance

Emitting telemetry to Application Insights may add latency.

Mitigations:

• Use batch processing mode (the default)


• Limit the number of traces (sampling)


• Asynchronous output

Summary

Integrating AGT’s auditing and telemetry gives you:

1. A complete audit trail: every policy decision and execution result is recorded


2. Real-time monitoring: agent behavior visualized through Application Insights


3. Anomaly detection: early warning through pattern recognition


4. Production readiness: a scalable, robust architecture

In production environments where AI agents play a critical role, transparency and traceability are essential. By leveraging AGT’s auditing and telemetry features, you can achieve safe, reliable agent operations.

References

• Agent Governance Toolkit – GitHub https://github.com/microsoft/agent-governance-toolkit  


• Azure Blob Storage – Append Blob https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-append  


• Azure Monitor OpenTelemetry Exporter https://learn.microsoft.com/en-us/azure/azure-monitor/app/opentelemetry-enable  


• Application Insights – KQL Query Reference https://learn.microsoft.com/en-us/azure/azure-monitor/logs/kql-quick-reference  


• Sample code repository https://github.com/normalian/MyAGTSamples 

I hope this article helps with implementing audit and telemetry capabilities for AI agents in production. If you have any questions or feedback, please let me know in the comments!