---
title: "5 Ways to Stop Azure AI Agents from Blowing Your Budget"
description: "Cap Azure AI agent spend with an API Management token limit, Application Insights telemetry, semantic caching, loop caps, and budget alerts."
canonical: "https://adamtheautomator.com/azure-ai-agent-budget-guardrails/"
---
# 5 Ways to Stop Azure AI Agents from Blowing Your Budget
> Cap Azure AI agent spend with an API Management token limit, Application Insights telemetry, semantic caching, loop caps, and budget alerts.
Source: https://adamtheautomator.com/azure-ai-agent-budget-guardrails/
---
ATA Learning
Tap to hide
[
ATA Learning
](/)
* [Home](/)
* [Tutorials](/tutorials/)
* [Instructors](/author/)
* [Advertising](/advertising/)
* [Recommended Resources](/resources/)
* [About Adam](/about-adam/)
Search for:
* [](https://twitter.com/adbertram)
* [](https://github.com/Adam-the-Automator)
* [](https://www.linkedin.com/company/adam-the-automator-llc)
* [](/feed/)
# 5 Ways to Stop Azure AI Agents from Blowing Your Budget
[Adam Bertram](https://adamtheautomator.com/author/adam-bertram/)4 November 202610 min. read
Categories: [AI Agents](/category/ai-agents/)
Tags:[Azure](/tag/azure/)[AI](/tag/ai/)[Monitoring](/tag/monitoring/)[API](/tag/api/)
Table of Contents
* [Prerequisites for Guarding an Agent](#prerequisites-for-guarding-an-agent)
* [Way 1: Cap Tokens in the Request Path with the llm-token-limit Policy](#way-1-cap-tokens-in-the-request-path-with-the-llm-token-limit-policy)
* [What the Cap Cannot Stop](#what-the-cap-cannot-stop)
* [Way 2: Prove the Cap Worked with Application Insights Token Telemetry](#way-2-prove-the-cap-worked-with-application-insights-token-telemetry)
* [Query Token Totals by Agent with KQL](#query-token-totals-by-agent-with-kql)
* [Confirm the Cap Fired](#confirm-the-cap-fired)
* [Turn the Query into Dollars](#turn-the-query-into-dollars)
* [From Invoice Line to the Failing Prompt](#from-invoice-line-to-the-failing-prompt)
* [Way 3: Cut Token Volume with Semantic Caching](#way-3-cut-token-volume-with-semantic-caching)
* [Tune the Threshold and Prove the Hit Rate](#tune-the-threshold-and-prove-the-hit-rate)
* [Way 4: Bound Autonomy in Code with Step Caps and a Circuit Breaker](#way-4-bound-autonomy-in-code-with-step-caps-and-a-circuit-breaker)
* [Cap Steps and Retries in the Loop](#cap-steps-and-retries-in-the-loop)
* [Put Each Tool Behind a Breaker](#put-each-tool-behind-a-breaker)
* [Way 5: Attribute Spend Per Agent and Wire the Escalation](#way-5-attribute-spend-per-agent-and-wire-the-escalation)
* [Budgets and Anomaly Alerts That Fire Before Month End](#budgets-and-anomaly-alerts-that-fire-before-month-end)
* [Which Guardrail to Turn On First](#which-guardrail-to-turn-on-first)
* [FAQ](#faq)
* [Do I Need API Management in Front of Every Model?](#do-i-need-api-management-in-front-of-every-model)
* [Will a Quota Take My Production Agent Offline Mid-Shift?](#will-a-quota-take-my-production-agent-offline-mid-shift)
* [Does Any of This Work Through a Wrapper Around the Model?](#does-any-of-this-work-through-a-wrapper-around-the-model)
Do not ship another agent until you can name the token ceiling it will stop at.
A system that picks its own step count at runtime makes that ceiling hard to state. This walkthrough sets one for a support-triage agent that checks four internal sources before answering, caps that spend at the gateway, proves the cap fired with request telemetry, and then follows a bad invoice line back to the prompt that caused it.
## Prerequisites for Guarding an Agent
If you want to follow along hands-on, you’ll need:
* A chat model deployment you can route through a gateway, since every policy here attaches to a model API.
* An API Management instance in a [tier that supports `llm-token-limit`](https://learn.microsoft.com/en-us/azure/api-management/llm-token-limit-policy). Consumption tier cannot run this.
* An [Application Insights resource connected to the gateway as a logger](https://learn.microsoft.com/en-us/azure/api-management/api-management-howto-app-insights), because Way 2 needs it.
* Azure CLI for the billing queries, or skip it if you only want the Kusto Query Language (KQL) examples.
The agent runs nine model calls per ticket: one planning call, four retrieval checks, up to four follow-ups after tool timeouts. The [AI gateway](https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities) governs what each call costs; your code governs how many calls happen.
## Way 1: Cap Tokens in the Request Path with the llm-token-limit Policy
A ceiling inside your agent protects the agent you wrote. A ceiling at the gateway protects the model deployment, whichever client calls it: your agent, a notebook someone left running, a logic app that grew a retry loop. The [`llm-token-limit`](https://learn.microsoft.com/en-us/azure/api-management/llm-token-limit-policy) policy runs inbound, so an over-budget request never reaches the backend.
It enforces two numbers that fail differently. Microsoft Learn states: “When a specified token rate limit is exceeded, the caller receives a `429 Too Many Requests` response status code. When a specified quota is exceeded, the caller receives a `403 Forbidden` response status code.” A 429 clears when the minute rolls over. A 403 holds until the period resets, so retry-on-error logic hammers a closed door.
The inbound policy below emits the Way 2 metric and enforces the cap on the same request:
```xml
```
The request path below shows both policies on one call.
`counter-key` gives each value its own counter, so `triage-prod` owns its 4 million daily tokens. `tokens-per-minute` caps the burst near ten model calls a minute, and `token-quota` caps the day near 208 runs. `estimate-prompt-tokens` is required, and `true` prices the prompt before forwarding.
### What the Cap Cannot Stop
Without prompt estimation, “Prompts may be sent to the backend even when the limit is exceeded; this is detected from the response, after which subsequent requests are blocked until the limit resets.” Counters also stay local: the same reference notes the policy “doesn’t aggregate token counts across the entire instance,” so a second region gets a second budget. And `X-Agent-Id` is a header your caller sets, so key the counter on its `appid` claim instead.
* * *
_**Reality Check: A 403 from a quota is not a retry problem, and most SDKs treat it as fatal. Good at the ceiling, miserable if the quota sits below a normal Tuesday.**_
* * *
## Way 2: Prove the Cap Worked with Application Insights Token Telemetry
Six weeks in, finance asks which agent spent an extra $1,200 while your logs show request counts. The `llm-emit-token-metric` policy reads the `usage` object from each response and writes token counts to Application Insights, the same service where you [catch failures before your customers do](https://adamtheautomator.com/application-insights-catch-failures/).
The metric type decides whether the data is usable for AI FinOps. The [Application Insights metrics model](https://learn.microsoft.com/en-us/azure/azure-monitor/app/metrics-overview) differs from [platform metrics](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-custom-overview), which carry a fixed dimension set. Only a custom metric keeps the `Agent-Id` you attached at the gateway.
### Query Token Totals by Agent with KQL
The policy accepts up to five custom dimensions, and the result lands in `customMetrics`. This query sums the model’s total-token metric per agent in 15-minute slices. Prompt and completion counts sit in the same telemetry under their own names when a model reports them:
```text
let Lookback = 24h;
customMetrics
| where timestamp > ago(Lookback)
| where name == "Total Tokens"
| extend AgentId = tostring(customDimensions["Agent-Id"]),
SubscriptionId = tostring(customDimensions["Subscription ID"])
| summarize TotalTokens = sum(valueSum), Samples = sum(valueCount)
by AgentId, SubscriptionId, bin(timestamp, 15m)
| order by TotalTokens desc
```
`name == "Total Tokens"` matches the total-token row only, so a model that also emits prompt and completion names cannot double the sum; `has "Tokens"` matches all three and reports roughly twice the spend. `valueSum` carries the count and `valueCount` shows how many requests fed it, which separates many small calls from a few enormous ones. If your agents stream, set `include_usage` or the counts arrive empty, per the [policy reference](https://learn.microsoft.com/en-us/azure/api-management/llm-emit-token-metric-policy).
### Confirm the Cap Fired
Evidence that the ceiling works:
```text
requests
| where timestamp > ago(24h)
| where resultCode in ("429", "403")
| summarize Blocked = count() by name, resultCode
| order by Blocked desc
```
Hundreds of `429` rows and no `403` mean the rate cap works and the quota has headroom. A wall of `403` before lunch means the quota is too tight and the fix is sizing.
### Turn the Query into Dollars
The [Azure OpenAI pricing page](https://azure.microsoft.com/en-us/pricing/details/azure-openai/) lists $2.50 per 1 million input tokens and $10.00 per 1 million output tokens for the GPT-4o family at Global Standard, which is the rate a Global deployment bills, or $0.0025 and $0.010 per 1,000. The regional US Standard meters run about 10% higher, at $2.75 and $11.00 per 1 million. Check those rates before you size a quota, because they change.
| Run profile (illustrative) | Input tokens | Output tokens | Cost per run | Per 1,000 runs | 20,000 runs monthly |
| --- | --- | --- | --- | --- | --- |
| Triage run, nine model calls | 18,000 | 1,200 | $0.057 | $57.00 | $1,140 |
| Same agent after a loop bug | 145,000 | 6,000 | $0.42 | $422.50 | $8,450 |
| Nine calls, 30% answered from cache | 12,600 | 840 | $0.040 | $39.90 | $798 |
Row one multiplies 18 thousand input tokens by $0.0025 and 1.2 thousand output tokens by $0.010. Row two is the same agent with a timing-out tool and a loop that ran 40 steps instead of nine. Nothing in that month looks unusual in a request count.
* * *
_**Key Insight: At $0.057 a run, twenty thousand runs is a rounding error in a department budget. One loop bug multiplies that line by seven without changing a single page view.**_
* * *
## From Invoice Line to the Failing Prompt
Cost data arrives after the fact, so use it to find the day the line left its baseline. The steps below narrow that window to a single prompt, and the funnel below traces the same path.
1. Open **Cost analysis**, filter to the model resource, and group by day. You want the day the line leaves its seven-day baseline. A shared resource needs [cost allocation](https://learn.microsoft.com/en-us/azure/cost-management-billing/costs/allocate-costs) first.
2. Run the token query for that date and resource. One agent usually owns most of the delta; if several do, sort by `valueCount` to separate many small calls from a few huge ones.
3. Drop to a 15-minute window inside that agent’s peak and group by operation and result code. `500` responses followed by successes on the same operation are a retry loop.
4. Read the prompt. With body logging enabled through a diagnostic setting or trace policy, the first request in the window carries the conversation that started the run.
5. Re-run step 2 after the cap ships. That delta is the number for the incident review.
## Way 3: Cut Token Volume with Semantic Caching
The guardrails so far limit spending. Caching changes how much there is to spend, and support tickets repeat. Semantic caching returns a stored response for a new prompt that means the same thing, removing the model call instead of discounting it.
Semantic caching needs an [Azure Managed Redis cache with RediSearch](https://learn.microsoft.com/en-us/azure/redis/tutorial-semantic-cache), an embeddings deployment, and two policies on the API.
```xml
@(context.Subscription.Id)
```
`vary-by` is the partition key, so entries never cross a subscription boundary. The `rate-limit` after the lookup matters: when the cache is unreachable every request falls through at full price. `duration` sets the cache lifetime in seconds, so 300 means entries expire after five minutes. `ignore-system-messages` drops the system prompt from the comparison, and `max-message-count` stops caching once a conversation runs past that many remaining messages. The [lookup](https://learn.microsoft.com/en-us/azure/api-management/llm-semantic-cache-lookup-policy) and [store](https://learn.microsoft.com/en-us/azure/api-management/llm-semantic-cache-store-policy) references cover the rest.
### Tune the Threshold and Prove the Hit Rate
`score-threshold` reads backwards from the usual assumption. Microsoft Learn states: “A lower score threshold means that prompts must have higher semantic similarity to return cached responses.” A threshold of 0.15 is strict. Raise it and loose matches return, which is embarrassing for a policy bot and costly for a billing agent.
Prove the hit rate before defending the savings. The lookup policy writes dependency telemetry for every check, so divide hits by checks over a week. Thirty percent is realistic for a triage queue, and it lands on row three: $798 a month instead of $1,140. Provision the cache early, since RediSearch can only be enabled at creation.
## Way 4: Bound Autonomy in Code with Step Caps and a Circuit Breaker
The gateway sees the calls the agent makes. It cannot see the loop that decides to make them, and a loop that keeps a failing tool will spend until a quota cuts it off.
A retry costs more than the call it repeats. Each failed attempt is appended to the conversation, so the next call sends a longer prompt and the model pays to read its own error. Three retries against a dead API can cost more than a successful run of the agent. The [Circuit Breaker pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker) covers this shape, and the [AI SDK loop control docs](https://ai-sdk.dev/docs/agents/loop-control) cover the stopping conditions.
### Cap Steps and Retries in the Loop
The constants at the top are the part to copy; `agent` and `budget` are your own objects on Python 3.10 or later.
```python
import time
from dataclasses import dataclass
MAX_STEPS = 12
MAX_RETRIES_PER_TOOL = 2
FAILURE_THRESHOLD = 3
COOLDOWN_SECONDS = 60
@dataclass
class Breaker:
failures: int = 0
opened_at: float | None = None
def allows_call(self) -> bool:
if self.opened_at is None:
return True
if time.monotonic() - self.opened_at >= COOLDOWN_SECONDS:
self.opened_at = None # half-open: one probe gets through
return True
return False
def record(self, ok: bool) -> None:
if ok:
self.failures = 0
return
self.failures += 1
if self.failures >= FAILURE_THRESHOLD:
self.opened_at = time.monotonic()
def run(agent, ticket, breakers, budget):
retries = {}
for step in range(MAX_STEPS):
if budget.exhausted:
return {"stopped": "token_budget", "steps": step}
action = agent.next_action(ticket)
if not action.is_tool_call:
return {"stopped": "finished", "steps": step + 1}
breaker = breakers.setdefault(action.tool, Breaker())
if not breaker.allows_call():
agent.note(f"{action.tool} is unavailable. Use another tool or answer without it.")
continue
try:
action.invoke()
breaker.record(True)
except Exception as exc:
retries[action.tool] = retries.get(action.tool, 0) + 1
if retries[action.tool] > MAX_RETRIES_PER_TOOL:
return {"stopped": f"retry_cap:{action.tool}", "steps": step + 1}
breaker.record(False)
agent.note(f"{action.tool} failed with {exc}. Do not retry it.")
return {"stopped": "step_cap", "steps": MAX_STEPS}
```
### Put Each Tool Behind a Breaker
The breaker returns without invoking the tool, so a dead API costs nothing after the third failure. The `agent.note` call tells the model the tool is gone, or it plans around a capability it lacks. The retry ceiling stops the tool that fails slowly and succeeds on the fourth attempt.
Size the threshold per tool: a fast API tolerates three failures, a scraper that times out at 30 seconds should open on the first. Share breaker state across runs, or every parallel request opens its own circuit.
## Way 5: Attribute Spend Per Agent and Wire the Escalation
This control acts on AI costs directly, the only layer that survives a resource whose tag someone forgot to set.
A [Foundry project](https://learn.microsoft.com/en-us/azure/foundry/concepts/manage-costs) carries usage, and [project tags propagate into cost analysis](https://adamtheautomator.com/fix-azure-cost-reporting-finops-tagging-strategy-3/), so tagging by owner and environment gives you a filter that outlives staff changes. Pair that with the `Agent-Id` dimension: tags say which unit owns the spend, dimensions say which agent produced it.
### Budgets and Anomaly Alerts That Fire Before Month End
A [Cost Management budget](https://learn.microsoft.com/en-us/azure/cost-management-billing/costs/tutorial-acm-create-budgets) thresholds actual or forecast spend at a scope or filter. Set alerts at 50, 80, and 100 percent and attach an action group to the highest one. That action group can start an [Azure Logic Apps](https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-overview) workflow or runbook, which is where a kill switch lives: disable the deployment, scale to zero, revoke the credential.
[Anomaly alerts](https://learn.microsoft.com/en-us/azure/cost-management-billing/understand/analyze-unexpected-charges) catch the spike a budget cannot, a bad week inside a normal month. Rules can only be created at subscription scope, so a per-agent rule needs its own subscription, and alerts arrive by email, which makes a mailbox-watching Logic App the bridge.
For chargeback, multiply each agent’s token total from Way 2 by its rate card and join it to the tag-derived owner map. Per-request pricing bills a 400-token classification the same as a 40,000-token research run.
## Which Guardrail to Turn On First
The five controls overlap, and the gaps are where money escapes.
| Guardrail | Failure it catches | What it cannot see |
| --- | --- | --- |
| `llm-token-limit` rate cap | One agent taking the whole tokens-per-minute (TPM) share | Which prompt drove the spike |
| `llm-token-limit` quota | A patient loop that never trips the rate cap | Loops that straddle the quota reset |
| Token metric plus alert | Burn rate rising before the invoice | Why the agent loops |
| Semantic cache | Repeat questions paying full price | Novel prompts, usually the costly ones |
| Step and retry caps | The infinite loop itself | A slow tool inside a normal step count |
| Budget and anomaly alerts | What got past the request path | The agent, unless tags line up |
Emit the token metric first, because a limit without attribution turns a budget incident into a support incident where nobody can name the team that hit the ceiling. Add the limit second, sized from the 95th percentile of two weeks of traffic and priced against the [rate card](https://azure.microsoft.com/en-us/pricing/details/azure-openai/). Add the cache third, once you know which prompts repeat. Bound the loop fourth, the only control that stops spend inside one request.
* * *
_**Warning: Enable the quota before the metric and the first 403 arrives with no telemetry to explain it. You will spend a day proving which agent hit a ceiling you could have seen in a KQL chart.**_
* * *
## FAQ
Whether you need a gateway at all depends on how many clients can reach the deployment, so that question comes first.
### Do I Need API Management in Front of Every Model?
No. Telemetry, the code-level caps, and budgets work without a gateway, and Foundry exposes [per-deployment token limits](https://learn.microsoft.com/en-us/azure/foundry/openai/quotas-limits). What you lose is enforcement: each app owns its ceiling, and an unmanaged caller can drain the deployment.
### Will a Quota Take My Production Agent Offline Mid-Shift?
It can, and the risk lives in sizing. Set the quota from measured daily consumption plus headroom for the busiest day last quarter, then have the agent [read the remaining-quota header](https://learn.microsoft.com/en-us/azure/api-management/llm-token-limit-policy) and degrade first: shorter answers, fewer retrieval passes.
### Does Any of This Work Through a Wrapper Around the Model?
Partly. The [token limit policy](https://learn.microsoft.com/en-us/azure/api-management/llm-token-limit-policy) works on APIs conforming to the OpenAI Chat Completions or Responses schema, the Anthropic Messages API (API Management v2 tiers only), or Google Vertex AI. Normalize a bespoke wrapper to one of those before you trust the cap.
Share this article
[Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fazure-ai-agent-budget-guardrails%2F&text=5%20Ways%20to%20Stop%20Azure%20AI%20Agents%20from%20Blowing%20Your%20Budget)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fazure-ai-agent-budget-guardrails%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fazure-ai-agent-budget-guardrails%2F)
## Related Posts
### [Notion Workers: Stop Agents Guessing Customer Lookups](/notion-worker-customer-lookup/)
Build a read-only lookupCustomer tool in a Notion Worker, then attach it to a Custom Agent so it answers with a plan and status instead of a guess.
### [Notion Automation: Pick the Right Tier Every Time](/notion-automation-agents-vs-workers/)
Pick the right Notion automation tier: compare database automations, Custom Agents, and Workers, and split ticket routing to cut credit costs.
### [Stop Runaway Notion Agent Costs with the Agent API](/notion-agent-api-cost-tracking/)
Use the Notion Agent API to read credits and runs per agent, compute cost per run, and gate runaway spending with status and credit-limit controls.
## Categories
* [IT Ops](/category/it-ops/)
* [Cloud](/category/cloud/)
* [DevOps](/category/devops/)
* [Home Ops](/category/home-ops/)
* [Information Security](/category/infosec/)
* [Software Development](/category/software-development/)
## Site
* [Home](/)
* [Tutorials](/tutorials/)
* [Instructors](/author/)
* [Advertising](/advertising/)
* [Recommended Resources](/resources/)
* [About Adam](/about-adam/)
Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)