AzureFeeds

Why cloud-native and multi-agent are converging

The shift from “a chatbot” to “a fleet of workers”

A single LLM call is a stateless function: text in, text out. A multi-agent system is something else entirely — a set of long-running processes that plan, call tools, spawn helpers, talk to each other, and keep working for minutes or hours without a human in the loop. The moment you give an agent real tools, you have also given it real credentials and a real network. That is the whole story of why this problem is hard.

In practice, a production agent is:

• Long-running. It is not a request/response endpoint; it holds a session, retries, and makes many downstream calls over its lifetime.


• Autonomous. It decides which tool to call and when — you don’t hand-code the control flow.


• Tool-using and network-connected. It reads files, calls APIs, fetches web pages, and increasingly calls other agents.


• Multiplied. One “task” is rarely one agent. It is a pipeline — a researcher hands to a writer, a writer hands to a reviewer.

That profile — long-running, autonomous, networked, and multiplied — is exactly the profile of a microservice fleet. Which is why the operational answer keeps landing on the same place: Kubernetes. Enterprises already know how to give microservices identity, network policy, secrets, quotas, observability, and GitOps. The natural move is to run agents with the same operational discipline as the rest of your services rather than inventing a parallel, unsupervised runtime.

The enterprise deployment problem: blast radius

Here is the uncomfortable truth about deploying multi-agent systems: a single prompt-injected agent can reach everything the agent process can reach. If the agent holds an Azure key, a prompt injection can exfiltrate it. If the agent has open network egress, a poisoned document can turn it into a data pump. If the agent can spawn sub-agents with the same privileges, one compromise becomes many.

Security people have a name for the worst-case pattern — the “lethal trifecta”:

1. access to private data,


2. exposure to untrusted content (a web page, an email, a document the agent was asked to summarize), and


3. an outbound channel to exfiltrate.

A general-purpose agent framework tends to have all three by default. A real, publicly discussed instance of this — a file-exfiltration attack against an agent “cowork” workflow in January 2026 — is exactly the kind of event this whole discipline exists to prevent. (Kars ships a reproduction of it as a demo; more on that in Part 3.)

So the enterprise question is not “can the agent do the task?” It is: “when this agent is compromised — not if — what is the blast radius, and who can prove it was contained?”

The specific risk of long-running third-party frameworks (OpenClaw, Hermes, and friends)

Most teams do not write their agent from scratch. They adopt a framework — OpenClaw, Hermes, the OpenAI Agents SDK, Microsoft Agent Framework, LangGraph, and so on. These frameworks are wonderful for velocity, but they change the security calculus in three ways:

• You did not write the code that runs autonomously. A third-party harness makes tool-calling decisions inside a loop you don’t control. Your review of your prompt does not cover its tool dispatch.


• They are built to run long. A channels-first harness like Hermes is designed to sit on a Telegram or Slack channel and react to whatever arrives — indefinitely. “Long-running” plus “reacts to untrusted input” is the lethal trifecta’s natural habitat.


• They pull dependencies and plugins. Every plugin, every MCP (Model Context Protocol) server, every tool is new supply-chain surface and new egress surface.

The wrong reaction is to fork and patch the framework — you inherit a maintenance burden and you fall behind upstream security fixes. The right reaction is to treat the framework as an untrusted tenant and put the enforcement outside it: no credentials inside the agent, no network of its own, every external call brokered and audited. That is precisely the design stance kars takes, and it is why kars runs OpenClaw and Hermes without modifying, patching, or vendoring their source — any upstream release is drop-in.

Token usage is a first-class production concern

Two things make token cost a governance problem, not just a billing footnote:

• Autonomy amplifies spend. An agent that loops, retries, and spawns sub-agents can burn tokens non-linearly. A runaway or adversarially-driven loop is both a cost incident and an availability incident (a “cascading failure”).


• Multi-tenancy demands fairness. If ten teams share a cluster, one team’s runaway agent must not starve the others.

The production requirement, therefore, is enforced token budgets — per-request caps and per-tenant daily/monthly ceilings — plus request-rate limits, applied before the call leaves the pod, with a hard HTTP 429 on overrun. Budgeting after the fact (reading a bill next month) is not a control; it is an autopsy.

“How do I test it before I ship it?” — the go-live validation problem

Before an agent goes live (“上架” — onto the platform, into production), a security team needs to answer, with evidence, questions like:

• Does the agent actually have zero standing credentials?


• Is egress actually restricted to the hosts I allow-listed?


• Does content safety actually fire on a jailbreak?


• If I feed it a poisoned document, does the isolation actually hold?


• Are the controls I wrote in YAML actually the controls the runtime is enforcing right now?

The only trustworthy answer is a reproducible test that runs the same controls that production runs — ideally the same pod shape, same policies, same audit format on a laptop as in the cloud, plus a signed attack corpus you can replay, and a tamper-evident attestation that proves what a live sandbox is actually enforcing. “It worked in the demo” is not go-live evidence; “here is the replayable proof that all N layers held” is.

And then: how do I actually deploy this to the cloud?

Finally, the integration question. A production agent platform is not one container — it needs a registry, a cluster with real workload identity, a model backend with content safety, an inter-agent message relay, and a public ingress for cross-org calls. The friction most teams hit is that the dev loop and the prod loop are two different systems, so what you tested locally is not what ships. The winning pattern is a single mental model from laptop to cloud where graduation is a one-line change, not a port.

The next shows how one open-source stack answers all six of these concerns with the same design.

Kars: Microsoft’s open-source Agent Reference Stack for Kubernetes

What it is, in one paragraph

kars (Agent Reference Stack for Kubernetes) is an open-source stack from Microsoft’s Azure Cloud Native team for running AI agents safely on Kubernetes. Its one-line thesis is: one hardened sandbox per agent, zero credentials in the agent, every external call governed. Every byte that leaves an agent leaves through a per-pod Rust inference router that enforces identity, content safety, token budgets, tool policy, egress rules, and a tamper-evident audit log. Agents on different frameworks talk to each other over an end-to-end encrypted mesh. And you drive the whole thing with one CLI, from a local Kubernetes cluster on your laptop to AKS, using the same resources.

The core idea: the trust boundary is the pod, not the cluster

This is the single most important design decision in kars, and it is what makes it different from a cluster-edge gateway.

Read the picture carefully, because every security property falls out of it:

• The agent container has no network of its own. It can only reach localhost. Every external call — the model, a tool, another agent — must go through the router.


• The router runs as a different process under a different UID (1001 vs the agent’s 1000). It holds the credentials; the agent never sees an Azure key.


• An init container, egress-guard, installs iptables rules so the agent’s UID can only reach the router locally, and a Kubernetes NetworkPolicy contains lateral movement. These two are safety nets — the router is the policy point; the nets catch anything that tries to bypass it.

The payoff, stated as a guarantee: compromise of the agent does not compromise the cloud account, the model, the audit log, or the peer mesh. Even a perfect prompt-injection payload that reads every byte the agent process can read cannot exfiltrate an Azure key — because there are none in the agent.

Why this is not “just an API gateway.” A cluster-edge gateway governs north-south traffic at the boundary. The kars router is an in-pod enforcement point sitting on localhost between the agent and everything else, so the agent has no network path that bypasses it — and it can do per-agent identity, content-safety, budget, and audit that a shared edge cannot do per-sandbox. They are complementary: a cluster-edge gateway can front kars, and the per-pod router still does the per-agent work.

The zero-trust core: what the inference router enforces

The per-pod router is where most of the security model lives. Everything it enforces maps directly back to the Part 1 concerns:

Multi-runtime: your framework, unmodified, in a hardened box

kars is a host for agent runtimes. The runtime is whatever framework your agent code is written against, plus a small adapter that wires it to the sandbox. Switching runtime is a one-field change in KarsSandbox.spec.runtime.kind — the same router, the same governance profile, the same audit chain, the same NetworkPolicy apply to all of them.

Runtimes that ship today include:

• OpenClaw (the default) — plus two multi-agent helpers on top of the mesh: sub-agent inheritance (a spawned child inherits the parent’s provider, model, endpoint, and credential wiring) and a peer roster (every spawn takes a role like “data analyst” or “technical writer”, so agents address each other by role — critical for analyst → visualizer → writer pipelines).


• Hermes — a channels-first agent harness with native MCP support, ideal when you want a Telegram- or Slack-driven agent without writing the integration. Hermes joins the same encrypted mesh as OpenClaw, and kars_mesh_send works in either direction between OpenClaw and Hermes peers (this interop is exercised end-to-end on every push).


• OpenAI Agents SDK, Microsoft Agent Framework (Python), LangGraph (Python & TypeScript), Anthropic Claude Agent SDK, Pydantic-AI, and BYO (bring any container under a small contract).

The adapters do three unglamorous but critical things: pin the model base URL to the router (http://127.0.0.1:8443) so the SDK physically cannot reach the public model endpoint directly, replace the API key with a sentinel (ROUTED-VIA-KARS) so no real credential is in the agent, and wire federated identity + OpenTelemetry + mesh registration. This is how a third-party framework runs unmodified yet governed.

The API is YAML, so security teams review YAML — not Python

Everything an operator configures is a Kubernetes Custom Resource. That is a deliberate move: approval gates, rate limits, tool allowlists, content-safety floors, token budgets, and trust topology become declarative resources you commit to a repo, reconcile with Argo/Flux, and audit with git log.

The ten workload CRDs you author:

Policy content can be pinned by immutable OCI digest and cosign-signed; the controller verifies the signature and re-canonicalises the bytes before the router loads them, and the CRD only goes Ready when the router echoes back the exact same digest it is enforcing. In other words: what’s in the YAML = what the controller verified = what the runtime is actually running, or the resource is not Ready.

The nine security layers (defence in depth)

kars is a layered control plane. Each layer bounds blast radius on its own; together they are why “one compromise” does not become “game over.” A live validation captured all nine on a real AKS cluster:

How this maps back to Last Part 

Planning a financial short-video production pipeline on Kars

Now let’s make it concrete. Below is a worked scenario — a multi-agent system that produces short financial-news videos (“财经短视频”) — mapped onto real kars primitives. It is illustrative (kars ships general examples, not this exact one), but every resource and command below is real kars API. This scenario is a perfect fit for kars because it has all the hard properties at once: untrusted inbound content (market news), regulated output (financial content needs compliance), expensive generation (long scripts + media), and a natural multi-agent handoff chain.

 

The pipeline as a fleet of agents

A financial short-video “factory” is naturally a pipeline of specialists:

Each box is one KarsSandbox. The handoffs happen over the encrypted mesh using the OpenClaw peer roster (agents address each other by role — “send the approved script to the voiceover agent”). This is exactly the analyst → writer → reviewer shape kars’ sub-agent inheritance and roster were built for.

Map each agent to a runtime and a risk profile:

Why Hermes for the Publisher? Because Hermes is channels-first (Telegram/Slack/publishing surfaces) and its interop with the OpenClaw pipeline over the encrypted mesh is a supported, tested path — the Scriptwriter (OpenClaw) can hand the finished package to the Publisher (Hermes) with kars_mesh_send and neither you, nor Microsoft, nor the relay can read the payload in transit.

Token planning — govern spend where it happens

The Scriptwriter is your token hot-spot, so give it an explicit budget. Every sandbox references an InferencePolicy; that policy is where model routing, the content-safety floor, and token budgets live.

apiVersion: kars.azure.com/v1alpha1
kind: InferencePolicy
metadata:
name: scriptwriter-inference
namespace: finvideo
spec:
appliesTo:
sandboxName: scriptwriter
modelPreference:
primary:
provider: azure-openai
deployment: gpt-4.1
inference:
contentSafety: true # Foundry content filter + Prompt Shields
contentSafetyMinimum: Medium # cannot be set below the cluster floor (admission-rejected)
# token budgets are enforced in-router: per-request cap + per-tenant daily/monthly ceilings,
# HTTP 429 on overrun — the runaway-loop / cost-incident guard from Part 1.4

Design guidance:

• Give each agent its own budget. The Research agent needs little; the Scriptwriter needs a lot. Per-agent InferencePolicy means one runaway agent hits its own ceiling, not the shared bill.


• Budgets are a safety control, not just cost control. A 429 on a runaway loop is how you stop a cascading-failure/DoS, not only how you save money.


• The rate limiter is separate and always on (global and per-agent request rates) — budgets cap volume of tokens, rate limits cap frequency of calls.

Security planning — least privilege per agent

Because the API is YAML, “least privilege” is something you write down and a reviewer can read.

Lock the Research agent’s egress so a poisoned article can’t turn it into an exfil pump. Baseline egress is signed and pinned; anything extra is a time-boxed EgressApproval:

apiVersion: kars.azure.com/v1alpha1
kind: EgressApproval
metadata:
name: research-newswire-window
namespace: finvideo
spec:
sandboxName: research-agent
hosts:
– api.approved-newswire.example
ttl: 4h # auto-revoked; no standing broad egress
reason: “Q3 earnings-week coverage”

Scope the Market-Data agent to exactly one tool surface via an MCP server with OAuth and a per-tool allowlist — the agent can fetch a quote but cannot call anything else the MCP server happens to expose:

apiVersion: kars.azure.com/v1alpha1
kind: McpServer
metadata:
name: market-quotes
namespace: finvideo
spec:
# OAuth to the upstream MCP server; only the listed tools are callable
allowedTools:
– get_quote
– get_daily_ohlc

Gate the Compliance approval with a ToolPolicy (approval + a high trust threshold) so the “publish-approved” action is a governed decision, and the hash-chained audit log becomes your regulatory record of which agent approved which script, when. In a regulated domain, that tamper-evident chain is not a nice-to-have — it is the artefact an auditor asks for.

Content safety is on by default for Foundry-provider requests (jailbreak / indirect-attack / hate / violence / self-harm / sexual), and the operator sets a floor that individual policies cannot go below — enforced at admission time, so a developer literally cannot ship a sandbox that is less safe than the cluster minimum.

Identity — every agent is its own principal

Turn on per-sandbox identity so each agent authenticates to Azure as itself:

 

With –mesh-trust=entra, the controller mints a per-sandbox Microsoft Entra Agent ID (a typed microsoft.graph.agentIdentity service principal), assigns Foundry RBAC scoped to that SP, wires a federated credential, and configures the mesh relay to verify peer JWTs against Entra’s JWKS. Foundry then sees the Publisher agent or the Scriptwriter agent as the calling principal — not one shared cluster identity. That means least-privilege RBAC and clean audit per agent. (The default –mesh-trust=anonymous skips Entra and shares the cluster’s Workload Identity — fine for demos, single-tenant.)

Crucially, in every mode the router brokers tokens via federated OIDC/IMDS and the agent never holds a long-lived key.

Azure integration — what one command provisions

When you are ready for the cloud, a single command stands up the whole platform in your subscription:

kars up –name finvideo-prod –region swedencentral –release –mesh-trust=entra

kars up does this, in order (and it’s idempotent — re-run to resume after a quota or IAM hiccup):

1. Preflight — checks subscription RBAC, resource providers, the Entra Agent ID directory role, preview features.


2. Resource group kars-finvideo-prod-rg.


3. ACR (your private registry) and an AKS cluster with Workload Identity + OIDC issuer enabled.


4. Azure AI Foundry project, a Content Safety binding, and a model deployment.


5. Images into ACR — –release imports the public, cosign-signed ghcr.io/azure/* images (no local build, no Rust toolchain).


6. Helm chart — controller + AgentMesh relay/registry + A2A gateway + CRDs.


7. First sandbox, waited until Ready.

You can bring your own AKS / Foundry / ACR if you already run them. Providers are pluggable: GitHub Copilot (device-code login, no Azure account — easiest to start), Azure AI Foundry / Azure OpenAI (full feature set: Memory Store, agents, Content Safety), or GitHub Models (free, PAT-only). Switching backend is a one-field CRD change.

Testing before go-live — evidence, not vibes

This is the “上架期间如何测试” answer, and it has four parts.

1. Test in the production pod shape, locally.The recommended dev loop runs your agent in a localkind cluster using the same Helm chart, NetworkPolicies, UID split, and router code path as AKS:

kars dev –release –target local-k8s
kars connect scriptwriter

Because the local pod shape mirrors AKS (it differs only in auth source and cloud infra), what you test locally is what ships. Graduation to the cloud is a one-line change (kars up), not a rewrite.

2. Replay a signed attack corpus withKarsEval.Author a KarsEval that runs your sandbox spec against a reproducible, signed evaluation corpus — so “does content safety fire, does egress hold, does isolation contain a poisoned document” becomes a repeatable, versioned test, not a one-off demo.


3. Reproduce the real attack.kars ships alethal-trifecta-demo that reproduces the January-2026 file-exfiltration attack against a vanilla OpenClaw versus a kars-managed agent — and shows six independent layers, each of which alone catches it. Running this against your own pipeline is the most honest go-live test you can do: point a known attack at the Research agent and watch the layers hold. The demo-clawshield example does the multi-tenant version (a poisoned document, two victim tenants, an isolation proof).


4. Attest what’s actually enforcing.kars attest surfaces tamper-evident evidence for a sandbox without cluster-admin: a spec hash, the SSA field-owner map, observed-generation lineage (drift detection), and per-policy version hashes. You can also verify the loop directly — the controller’s compiled policy digest must equal the router’s loaded digest, or the CRD is not Ready. That closes the gap between “what the YAML says” and “what the runtime is doing” with a check a reviewer can run.

CI backs all of this: cargo audit / npm audit for dependencies, fuzz and property tests on the security-critical paths (handoff blobs, blocklist parsing, policy engine, Double-Ratchet), and a sandbox-hardening test suite that asserts the UID split, read-only rootfs, dropped capabilities, and the seccomp profile.

The end-to-end mental model for the pipeline

Putting it together, here is the lifecycle of one financial short-video job under kars:

1. Research agent fetches approved newswire content (tight egress; content safety scans the inbound text; no credentials to steal).


2. It hands to Market-Data over the E2E mesh; Market-Data pulls quotes through a scoped MCP tool only.


3. Scriptwriter drafts the script under an explicit token budget (a runaway draft hits 429, not your monthly bill).


4. Compliance reviews under a ToolPolicy approval gate; its approval is written to the hash-chained audit log — your regulatory record.


5. Voiceover and Video-Assembly call scoped media tools; heavy/risky steps can run in confidential (Kata) isolation.


6. Publisher (Hermes) ships to the platform through the strictest egress + a TTL EgressApproval — the one agent allowed to talk to the outside world, and the one most tightly watched.

Every step: agent has no credentials, no direct network, and every call is brokered, budgeted, safety-checked, and audited by the per-pod router. Every control is YAML in a repo. And you validated all of it locally in the production pod shape before kars up ever ran.

Closing — what to take away

• Multi-agent is a cloud-native problem. Long-running, autonomous, networked, multiplied agents have the operational profile of a microservice fleet — so give them the same discipline: identity, network policy, quotas, GitOps, and audit.


• The framework is not the trust boundary — the pod is. Run OpenClaw, Hermes, or any framework unmodified, but put the enforcement outside it: zero credentials in the agent, no agent network, and a per-pod router that brokers and audits every call.


• Budgets and egress are safety controls, not just cost/ops hygiene. Enforce token budgets and per-host egress before the call leaves the pod.


• Go-live testing means reproducible evidence. Test in the production pod shape locally, replay a signed attack corpus, reproduce the real exfiltration attack, and attest what the runtime is actually enforcing.


• The cloud integration is one command with a one-line graduation. kars up provisions AKS + ACR + Foundry + mesh + gateway; –mesh-trust=entra gives every agent its own Entra identity and scoped Foundry RBAC.

kars is a reference stack — the point is the architecture, not the product badge. If you take one idea from this article, take this: treat every agent as an untrusted tenant, make the pod the trust boundary, and make every control a signed piece of YAML you can review, replay, and attest.

Reference map

Learn Kars  :  https://github.com/Azure/kars 

Source Code : https://github.com/kinfey/Multi-AI-Agents-Cloud-Native/tree/main/code/kars_openclaw_arch