--- title: "How to Stop AI Sprawl in Azure with a Governance Hub" description: "Build an Azure governance hub that inventories your AI estate, routes every model call through an AI gateway, and enforces identity, DLP, and policy." canonical: "https://adamtheautomator.com/stop-ai-sprawl-in-azure-governance-hub/" --- # How to Stop AI Sprawl in Azure with a Governance Hub > Build an Azure governance hub that inventories your AI estate, routes every model call through an AI gateway, and enforces identity, DLP, and policy. Source: https://adamtheautomator.com/stop-ai-sprawl-in-azure-governance-hub/ --- ATA Learning Tap to hide [ ATA Learning ](/) * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Search for: * [](https://twitter.com/adbertram) * [](https://github.com/Adam-the-Automator) * [](https://www.linkedin.com/company/adam-the-automator-llc) * [](/feed/) ![How to Stop AI Sprawl in Azure with a Governance Hub](https://adamtheautomator.com/wp-content/uploads/publisher/2e05d9c85b2b81dcbcf5eda389061979/e56f758e2e845005bdef1f99eb4feb9315fc53a0423f968bd418c5cf15e5e4ee.webp) # How to Stop AI Sprawl in Azure with a Governance Hub [![](https://secure.gravatar.com/avatar/d0b9d42e21e5622713f8b693aa5c0f9244d5f7dd200ed29b8398f52dee5de337?s=192&d=mm&r=g)Adam Bertram](https://adamtheautomator.com/author/adam-bertram/)1 October 202619 min. read Categories: [Information Security](/category/infosec/) Tags:[Azure](/tag/azure/)[AI](/tag/ai/)[Security](/tag/security/)[Governance](/tag/governance/) Table of Contents * [What AI Sprawl Actually Costs You](#what-ai-sprawl-actually-costs-you) * [The Numbers Behind Shadow AI](#the-numbers-behind-shadow-ai) * [Why an Enterprise License Is Not a Control](#why-an-enterprise-license-is-not-a-control) * [Prerequisites for the Build](#prerequisites-for-the-build) * [Why the Build Order Matters](#why-the-build-order-matters) * [The Control Plane: One Choke Point, Many Subscriptions](#the-control-plane-one-choke-point-many-subscriptions) * [What Belongs in the Hub](#what-belongs-in-the-hub) * [What Stays in the Spoke](#what-stays-in-the-spoke) * [Matching Each Risk to a Control](#matching-each-risk-to-a-control) * [Step 1: Stand Up the Azure AI Governance Hub](#step-1-stand-up-the-azure-ai-governance-hub) * [Create the Central Log Sink](#create-the-central-log-sink) * [Create the Shared AI Account](#create-the-shared-ai-account) * [Create the Gateway](#create-the-gateway) * [Step 2: Give Every Agent Its Own Identity](#step-2-give-every-agent-its-own-identity) * [Grant Data-Plane Access Without Keys](#grant-data-plane-access-without-keys) * [Keep the Inventory Authoritative](#keep-the-inventory-authoritative) * [Step 3: Route Every Model Call Through the AI Gateway](#step-3-route-every-model-call-through-the-ai-gateway) * [Model Aliases Decouple Callers from Providers](#model-aliases-decouple-callers-from-providers) * [Enforce Limits Where the Tokens Are Counted](#enforce-limits-where-the-tokens-are-counted) * [Turn Token Spend into a Billable Number](#turn-token-spend-into-a-billable-number) * [Step 4: Classify the Data Behind Every Prompt](#step-4-classify-the-data-behind-every-prompt) * [Build the Classification Baseline First](#build-the-classification-baseline-first) * [Apply Sensitivity Labels and DLP to AI Interactions](#apply-sensitivity-labels-and-dlp-to-ai-interactions) * [Step 5: Turn On Posture Management and Threat Protection](#step-5-turn-on-posture-management-and-threat-protection) * [Read the AI Bill of Materials](#read-the-ai-bill-of-materials) * [Rank Fixes with Attack Paths](#rank-fixes-with-attack-paths) * [Step 6: Add Runtime Guardrails and Cost Controls](#step-6-add-runtime-guardrails-and-cost-controls) * [Stop the Two Kinds of Injection](#stop-the-two-kinds-of-injection) * [Add the Remaining Runtime APIs Where They Fit](#add-the-remaining-runtime-apis-where-they-fit) * [Cap the Spend Before It Caps Itself](#cap-the-spend-before-it-caps-itself) * [Enforce the Whole Platform with Policy and Bicep](#enforce-the-whole-platform-with-policy-and-bicep) * [Assign Policy at the Right Scope](#assign-policy-at-the-right-scope) * [Keep the Hub Reproducible](#keep-the-hub-reproducible) * [From Sprawl to Governed: A Phased Roadmap](#from-sprawl-to-governed-a-phased-roadmap) * [Phase 1: Discover and Freeze](#phase-1-discover-and-freeze) * [Phase 2: Centralize the Choke Point](#phase-2-centralize-the-choke-point) * [Phase 3: Enforce and Prove It](#phase-3-enforce-and-prove-it) * [Governance KPIs, Incident Response, and Your Next Move](#governance-kpis-incident-response-and-your-next-move) * [Give the SOC an AI Runbook](#give-the-soc-an-ai-runbook) * [Your Next Three Moves](#your-next-three-moves) Can you list every AI workload running in your Azure tenant right now? Not the ones your architecture board reviewed. The agent a business analyst assembled from a Copilot Studio template, the model deployment a developer wired into a Friday demo, and the third-party assistant a regional team bought on a corporate card. Azure AI governance is the practice of answering that question continuously, and this tutorial builds the platform that does it. Answering that question once is the easy half. Keeping the answer accurate is the job this tutorial hands to a platform instead of a spreadsheet. You will inventory the AI estate, then stand up a governance hub that every model request passes through. Identity, data, threat, and runtime controls get attached to that hub, and the whole platform ends up encoded in Azure Policy and Bicep. The practical outcome is a platform that produces the inventory continuously and can show who authorized each model call. ## What AI Sprawl Actually Costs You [AI sprawl](https://adamtheautomator.com/taming-ai-tool-sprawl-powershell-guide-auditing/) describes the accumulation of models, agents, and third-party assistants across an organization that never assigned a central owner or a shared identity model, and that has no single place to watch the whole estate. The Microsoft Cloud Adoption Framework treats that as the default starting condition, and [Govern and secure AI agents across the organization](https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/governance-security-across-organization) prescribes the fix in one sentence: “Establish a centralized and enforceable governance and security baseline for all AI agents that aligns with existing identity, data governance, and security practices.” The same document explains why agents deserve their own baseline: “AI agents are software systems that can access data, make decisions, and take actions across business systems.” An agent that can read a ticketing system, query a database, and post results to a chat channel holds more effective authority than most service accounts your team audits every quarter. ### The Numbers Behind Shadow AI The adoption figures behind that gap come from studies that sample different populations, so the table below reports a range for each finding rather than a single value. | Finding | Reported range | Study | | --- | --- | --- | | Employees using AI at work | 75% to 78% | Microsoft Work Trend Index (2024, 2026) | | AI users bringing their own tools | 71% to 78% | Microsoft Work Trend Index (2025, 2026) | | Employees using unapproved AI tools | 55% to 65% | Salesforce State of IT (2024) | | Employees who shared non-public data with AI tools | 48% to 57% | Cisco (2024), Menlo Security (2025) | | Unapproved AI tools in active enterprise use | 158 or more | Gartner AI Governance Survey (2025) | | Breaches where shadow AI was a contributing factor | 20% | IBM Cost of a Data Breach (2025) | The [shadow AI](https://adamtheautomator.com/stop-shadow-ai-microsoft-purview/) ranges are compiled in [Airia’s shadow AI data roundup](https://airia.com/blog/shadow-ai-statistics-key-data-points-every-ciso-needs-in-2026/), and the two studies this article leads with have their own pages: the [Microsoft Work Trend Index](https://www.microsoft.com/en-us/worklab/work-trend-index) for the adoption figures and the [IBM Cost of a Data Breach report](https://www.ibm.com/reports/data-breach) for the breach attribution. Read the compilation before you quote any single figure upward, because an executive will argue with a suspiciously round number and accept a range. The IBM finding is the one that changes budget conversations: shadow AI contributing to one breach in five makes a controls budget defensible on incident data. ### Why an Enterprise License Is Not a Control Buying licenses for a sanctioned assistant feels like the governance answer because it moves the tool onto a corporate contract. Contracts do not classify data, and they do not decide which employee may paste which document into which prompt. The Cloud Adoption Framework’s [governance guidance for AI platform services](https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai/platform/governance) names the missing piece directly: “Data security boundaries prevent sensitive information from reaching unauthorized AI endpoints.” A procurement contract moves the invoice and leaves the data exactly where it was. The rest of this tutorial builds the data security boundaries as a platform you operate. The first decision is where the boundary lives. ## Prerequisites for the Build If you want to follow along hands-on, you will need: * An Azure subscription where you can create resource groups and assign policy at subscription scope. Every control here is enforced from the hub, so permissions scoped to one resource group will strand you at the first policy assignment. * Azure CLI 2.85.0 and Bicep CLI 0.44.1. Every command and template in this article was checked against those versions; run `az bicep version` to confirm the bundled CLI and `az bicep upgrade` to update it. * The `Microsoft.CognitiveServices` resource provider registered on the subscription. Run `az provider register --namespace Microsoft.CognitiveServices --wait` once; the API Management provider registers during its first deployment. * A decision about Defender CSPM. The AI posture features in Step 5 sit behind the paid plan, which bills per covered resource. Skip that step for a lab and the platform still functions, minus the AI bill of materials and attack path findings. * Purview permissions if you intend to complete Step 4. Purview is configured through its own governance portal rather than the Azure CLI, so that step mixes CLI verification with portal configuration, and you will need a role that can publish classification and Data Loss Prevention policies. ### Why the Build Order Matters Network isolation and a central log sink come first, because [Foundry Tools restrict network access](https://learn.microsoft.com/en-us/azure/ai-services/cognitive-services-virtual-networks) once you turn off public endpoints: “When network rules are configured, only applications that request data over the specified set of networks can access the account.” An AI service with no private endpoint and no log sink is a service you cannot govern after the fact. That order is the part of Azure AI governance that is hardest to change later: a model endpoint already serving production traffic is far harder to move behind a private path than a new deployment is. The diagram below shows the target topology: a hub virtual network that carries the shared services, and spoke networks that reach them privately. ![Governance hub and spokes](https://adamtheautomator.com/wp-content/uploads/publisher/d8b0d8735a6aa791c63a64abf49b2dc78bf0eac78886c9ff49bede661264b617.png) ## The Control Plane: One Choke Point, Many Subscriptions The hub is the control plane for every model request in your tenant, and a governance platform without a choke point is a set of recommendations. Every request that reaches a model, an agent, or a tool should pass through infrastructure you own, because that is the only place where policy gets enforced rather than suggested. Microsoft’s [Azure Virtual WAN hub-spoke architecture](https://learn.microsoft.com/en-us/azure/architecture/networking/architecture/hub-spoke-virtual-wan-architecture) supplies the topology: a hub that carries shared services and centralized inspection, and spokes that isolate individual workloads while staying private. ### What Belongs in the Hub The hub subscription holds the pieces that must be identical for every business unit: * The AI gateway, which is [API Management acting as the governed runtime boundary](https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities) for models, MCP servers, and agent-to-agent APIs. * The shared Foundry account and its model deployments, so no team provisions its own frontier model endpoint. * Azure Firewall for north-south and east-west inspection, plus the private DNS zones that make private endpoints resolvable. * One Log Analytics workspace that every AI service sends diagnostics to. ### What Stays in the Spoke Business unit workloads keep their own subscriptions and resource groups. Each spoke holds tenant-specific agent logic, private storage, and the application code that calls the gateway. The spoke team never holds a model provider key, because the gateway holds provider credentials and authenticates the caller through Microsoft Entra ID or a runtime access key. ### Matching Each Risk to a Control The table below maps the risks you are actually reducing to the Azure control that reduces them. Use it as your build checklist and as the artifact you hand a risk committee. | Risk | Azure control | Where it runs | | --- | --- | --- | | Ungoverned model endpoints | AI gateway with model aliases | Hub | | Credential sprawl across teams | Managed identities plus gateway-held provider credentials | Hub and spoke | | Sensitive data reaching a model | Purview classification, sensitivity labels, and DLP for AI | Tenant | | Prompt injection and jailbreak | Content Safety Prompt Shields | Hub | | Unpatched AI libraries in containers | Defender for Cloud AI security posture | Subscription | | Runaway token spend | Token rate limits and OpenTelemetry cost metrics at the gateway | Hub | | No audit trail for AI actions | Central diagnostics to Log Analytics, forwarded to Microsoft Sentinel | Hub | A single request meets those controls in a fixed order, and the infographic below traces it. ![Five control layers](https://adamtheautomator.com/wp-content/uploads/publisher/312ab69c1542afe8284784f3086cb0b6599646cd064853dbfd0dcaa8752676b0.png) * * * _**Reality Check: A hub that only routes traffic is a proxy. The controls in this platform live in policy, identity, and data classification, and the hub exists so those controls have one place to run.**_ * * * ## Step 1: Stand Up the Azure AI Governance Hub Start with the subscription that will hold shared services, and create the resource group every hub component lands in. ```bash az group create \ --name rg-ai-gov-hub \ --location eastus2 ``` Choose the region deliberately. Model availability, Private Link support, and the gateway’s regional footprint all vary by location. Moving a governance hub later is a project measured in weeks, and picking the region correctly on day one costs ten seconds. ### Create the Central Log Sink The Log Analytics workspace is where AI service diagnostics, gateway traces, and firewall logs land. Build it before the services, because attaching diagnostics is a per-resource step you would otherwise repeat. ```bash az monitor log-analytics workspace create \ --workspace-name law-ai-gov \ --resource-group rg-ai-gov-hub \ --location eastus2 \ --retention-time 90 ``` The `--retention-time 90` value sets interactive retention to 90 days, and that number is a compliance decision: pick the shortest window that satisfies your longest audit requirement, then archive beyond it. The [workspace CLI reference](https://learn.microsoft.com/en-us/cli/azure/monitor/log-analytics/workspace?view=azure-cli-latest) documents the full property set if you need longer retention or a different access model. ### Create the Shared AI Account The hub owns the model endpoints. Give the account a custom subdomain, because a private endpoint on a Foundry Tools account requires one. ```bash az cognitiveservices account create \ --name aifoundrygovhub \ --resource-group rg-ai-gov-hub \ --location eastus2 \ --kind AIServices \ --sku S0 \ --custom-domain aifoundrygovhub \ --assign-identity \ --yes ``` `--custom-domain` is mandatory for network isolation, and `--assign-identity` gives the account a system-assigned managed identity that you will grant data-plane roles to in the next step instead of distributing keys. To see which models and model versions are available in your region before you ask for one: ```bash az cognitiveservices model list \ --location eastus2 \ --query "[?kind=='OpenAI'].{name:model.name, version:model.version, skus:model.skus[].name}" \ --output table ``` Then close the public door, because the default state of the account is open to every network: ```bash resourceId=$(az cognitiveservices account show \ --resource-group rg-ai-gov-hub \ --name aifoundrygovhub \ --query id --output tsv) az resource update \ --ids $resourceId \ --set properties.networkAcls="{'defaultAction':'Deny'}" ``` Read the [network configuration guidance](https://learn.microsoft.com/en-us/azure/ai-services/cognitive-services-virtual-networks) before you run that second command on an account anything already depends on. The documentation warns that “by default, Foundry Tools resources accept connections from clients on any network,” so denying access blocks every caller until a network rule grants it back. Denying that traffic is the intended outcome, and it will break any running application that has not been moved onto a private endpoint. Everything below covers the governance layer; the hub virtual network, Azure Firewall, private DNS zones, and the private endpoint that makes this account reachable again are the network build, and Phase 2 points at the maintained template for it. ### Create the Gateway The gateway is the component every model request will pass through, so it lands in the hub subscription next. Create it before any workload points at a model endpoint: ```bash az apim create \ --name apim-ai-gov \ --resource-group rg-ai-gov-hub \ --location eastus2 \ --publisher-email security@contoso.com \ --publisher-name "Contoso Security" \ --sku-name Standard \ --no-wait ``` API Management provisioning is slow enough that `--no-wait` keeps your terminal usable. The [AI gateway capabilities](https://learn.microsoft.com/en-us/azure/api-management/genai-gateway-capabilities) apply across the service tiers, so a Standard instance is a valid governance boundary for a first build. Confirm the instance finished before you continue: ```bash az apim show \ --name apim-ai-gov \ --resource-group rg-ai-gov-hub \ --query "provisioningState" ``` ## Step 2: Give Every Agent Its Own Identity The most common governance failure in an AI estate is a shared API key copied between environments until nobody knows which application uses it. Microsoft’s Cloud Adoption Framework states the requirement plainly: “A governance and security baseline sets the minimum requirements that every agent must meet before it is allowed to operate.” An API key cannot express that baseline. It carries no caller identity, so it cannot be scoped per workload, rotated per team, or revoked for one application without breaking the others. Revoking a shared key to contain one incident takes every dependent application down with it, which is the decision nobody wants to make at 2 a.m. A [managed identity carries identity](https://adamtheautomator.com/secure-azure-managed-identities/) into the authorization decision, which makes the decision auditable. ### Grant Data-Plane Access Without Keys Assign the agent’s managed identity a data-plane role on the model account instead of handing it a key. For autonomous agents, [Microsoft Entra Agent ID](https://learn.microsoft.com/en-us/entra/agent-id/what-are-agent-identities) extends the same identity-first pattern with a first-class agent identity and lifecycle. The [built-in roles for AI and machine learning](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/ai-machine-learning) include Cognitive Services OpenAI User, which grants inference access without granting account administration. ```bash az role assignment create \ --assignee-object-id \ --assignee-principal-type ServicePrincipal \ --role "Cognitive Services OpenAI User" \ --scope $(az cognitiveservices account show --resource-group rg-ai-gov-hub --name aifoundrygovhub --query id --output tsv) ``` `--assignee-principal-type ServicePrincipal` prevents a class of silent failure. Without it, Azure RBAC may try to resolve the object ID through Microsoft Entra ID as a user, and the assignment is created incorrectly or rejected outright. ### Keep the Inventory Authoritative Identity gives you the inventory, because an agent that has to authenticate is an agent that appears in the sign-in and audit logs. The Cloud Adoption Framework treats that inventory as a precondition for the rest of model governance, listing “Create and maintain an AI agent inventory” as the first step of model governance. An inventory assembled from logs differs from one assembled from a spreadsheet. It updates when an agent is created, and it reports the last time each agent authenticated. ## Step 3: Route Every Model Call Through the AI Gateway The gateway turns the architecture from a diagram into a control. It terminates the caller’s request, applies policy, and forwards the request to whichever backend actually serves the model, so every capability in this section depends on the gateway seeing the traffic first. Azure AI governance starts at the gateway, because every request has to pass through it before a model sees anything. Microsoft describes the service this way: “The AI gateway in Azure API Management is a set of capabilities that help you manage your AI backends effectively.” ### Model Aliases Decouple Callers from Providers Publish a stable alias, `chat-standard` in this build, and point it at a backend deployment. Applications call the alias. When you move from one model version to another, or from one provider to another, you change the alias target while every caller keeps using the same name. Without aliases, a model swap becomes a coordinated release across every team that hard-coded an endpoint name. The unified model API goes further: “It exposes multiple backends through a single OpenAI-compatible endpoint, handles format translation automatically, and lets you apply governance policies once across all models.” Format translation is what makes a multi-vendor estate governable, because one set of policies now covers providers whose native APIs disagree with each other. ### Enforce Limits Where the Tokens Are Counted The gateway enforces policy on the request path, so a developer cannot bypass a control by skipping an application library, and a control that lives in a library only protects the applications that imported it. The controls worth configuring on day one: * Token rate limits and request rate limits per subscription key or identity, so one runaway loop cannot exhaust a capacity reservation. * IP filtering on the gateway so calls arrive only from spoke networks or the corporate egress range. * Semantic caching for repeated prompts, which cuts latency and token cost on workloads that ask the same question repeatedly. * Model and content policies attached to the alias, so the same guardrails apply regardless of which caller arrives. ### Turn Token Spend into a Billable Number The gateway emits OpenTelemetry traces, token counts, and estimated cost metrics through its [observability integration](https://learn.microsoft.com/en-us/azure/api-management/ai-gateway-overview). Send those to the hub workspace and you can attribute consumption to a subscription, a business unit, or a single agent. Attributing measured consumption to a business unit changes the politics of AI adoption. A team asking for more capacity can be answered with its own measured consumption, and the platform team charges the cost back instead of absorbing it in a central budget line. * * * _**Key Insight: The gateway is also the cheapest place to survive a model deprecation. If every caller uses an alias, retiring a model version becomes a routing change you make once instead of an application migration across a dozen teams.**_ * * * ## Step 4: Classify the Data Behind Every Prompt Identity and routing control who may call a model. They say nothing about what the prompt contains, and the prompt is where regulated data leaves your boundary. [Microsoft Purview supplies that layer](https://adamtheautomator.com/microsoft-purview-dlp-policies/), and its [data security and compliance protections for generative AI apps](https://learn.microsoft.com/en-us/purview/ai-microsoft-purview) cover Copilot experiences, custom Foundry applications, and third-party AI tools in one policy surface. ### Build the Classification Baseline First Enforcement without classification produces a stream of false positives that a business unit will eventually ask you to switch off. Start by discovering sensitive information in the systems that feed your agents, then apply the system sensitive information types and trainable classifiers that match your data. [Purview’s data governance overview](https://learn.microsoft.com/en-us/purview/data-governance-overview) covers the catalog and lineage side that tells you which data stores matter. Two classifier behaviors decide whether the rollout succeeds or stalls: * Credit card patterns and national identifiers are mechanical and produce few surprises. * Free-text categories, contracts and source code among them, are statistical, and they need tuning against examples from your own environment before you attach them to a blocking policy. ### Apply Sensitivity Labels and DLP to AI Interactions Once classification exists, the controls compound: * Sensitivity labels travel with the content, so a document that reaches a Copilot prompt still carries its protection. * Data Loss Prevention policies evaluate prompts and responses against the same sensitive information types you already defined. * Retention and Data Lifecycle Management policies keep or delete prompts and AI responses on a schedule, which is the first thing most audits ask about. Prompts and responses are records. If your retention policy deletes chat history after 30 days while your regulator expects seven years for a decision record, that gap is a finding waiting to be written. ## Step 5: Turn On Posture Management and Threat Protection Configuration drifts after the build, and AI configuration drifts faster than most because the features are new and the defaults change. Microsoft Defender for Cloud provides the continuous assessment that catches the drift, and its [AI security posture management](https://learn.microsoft.com/en-us/azure/defender-for-cloud/ai-security-posture) covers environments that span Azure, AWS, and GCP. Enable the paid posture plan first, since the AI inventory and attack path findings live there: ```bash az security pricing create \ --name CloudPosture \ --tier Standard ``` Run that in each subscription that hosts AI workloads, or at a management group through a policy assignment if you have several. The plan distinction is documented in the [CSPM concept article](https://learn.microsoft.com/en-us/azure/defender-for-cloud/concept-cloud-security-posture-management): the free plan provides a benchmark score and basic recommendations, while the paid plan adds attack path analysis and risk prioritization. ### Read the AI Bill of Materials Defender for Cloud builds an inventory of your AI estate instead of asking you to maintain one. The documentation states that “Defender for Cloud automatically and continuously discovers deployed AI workloads across the following services,” and the same discovery pass covers library dependencies: “Defender for Cloud can also discover vulnerabilities within generative AI library dependencies … by scanning source code for Infrastructure as Code (IaC) misconfigurations and container images for vulnerabilities.” ### Rank Fixes with Attack Paths A list of findings becomes useful when it is sorted by what an attacker could chain together. Attack path analysis connects an externally reachable AI endpoint, an over-scoped managed identity, and a data store into one story. A severity column satisfies a report; a chain tells you what to fix this sprint. The AI bill of materials this pass produces is the most accurate inventory your organization is likely to get, so treat it as the deliverable and the attack paths as the order of work. ## Step 6: Add Runtime Guardrails and Cost Controls Posture management watches configuration. Something else has to watch the traffic, because a [prompt injection arrives as a legitimate request](https://adamtheautomator.com/stop-github-copilot-leaking-enterprise-data-2/) from an authorized identity. Azure AI Content Safety provides that runtime layer. Microsoft describes the service this way: “Azure AI Content Safety is an AI service that detects harmful user-generated and AI-generated content in applications and services.” For AI governance, the prompt protection APIs matter more than the harm categories. ### Stop the Two Kinds of Injection [Prompt Shields](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection) handles both directions of this attack class. The API analyzes the user prompt and up to five documents, and the response reports an `attackDetected` flag for each. Direct attacks arrive from the person typing. An indirect attack arrives inside the grounding data, which is the harder case, because the instructions hide in a document the agent fetched and trusted. Microsoft describes these document attacks as “hidden instructions in third-party content … that attempt to take control of the model session.” ### Add the Remaining Runtime APIs Where They Fit The other prompt protection APIs answer different questions: * Groundedness detection scores whether a response is supported by the grounding sources, catching the hallucination a customer or an auditor will notice. * Protected material text detection flags output that reproduces copyrighted content, including lyrics and licensed source code. * Task adherence detects a tool call that misfires in the conversation, either arriving too early or targeting the wrong thing. * Content analysis returns harm categories with severity levels, which you translate into thresholds that express your acceptable use policy. ### Cap the Spend Before It Caps Itself Runtime guardrails cost money, and so does the traffic they inspect. The gateway’s token rate limits and cost telemetry from Step 3 are the enforcement point, and the budget alerts on the hub subscription are the backstop. Set both. A token limit tells an application to slow down; a budget alert tells you that a team found a way around the limit. * * * _**Pro Tip: Run Prompt Shields in annotate mode for a week before you switch it to block. The annotate mode logs what would have been filtered, which tells you whether your custom categories match your real traffic or your imagination.**_ * * * ## Enforce the Whole Platform with Policy and Bicep Everything built so far is a configuration that someone can undo, and the undo almost never happens in a subscription you are watching. [Azure Policy turns each decision into a guardrail](https://adamtheautomator.com/automate-soc-2-compliance-powershell-azure/) that new resources inherit. The Cloud Adoption Framework frames the tool this way: “Azure Policy offers predefined policy definitions for common governance needs in AI services. These policies enforce security settings, cost controls, and compliance requirements without custom development.” ### Assign Policy at the Right Scope Assign a policy that restricts which models may be deployed, using the [built-in policies for approved model deployments](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/built-in-policy-model-deployment), and assign it above the subscriptions you intend to cover. Assignment at a management group is what makes a subscription created next quarter inherit the same rules: ```bash az policy assignment create \ --name deny-unapproved-models \ --policy \ --scope /providers/Microsoft.Management/managementGroups/ \ --enforcement-mode Default ``` The `--enforcement-mode Default` value makes the policy active. Use `DoNotEnforce` the first time you assign a broad deny policy so you can read the compliance report before anything fails to deploy, then switch it. ### Keep the Hub Reproducible Policy enforces the rules; Bicep rebuilds the platform. This template creates the hub workspace, the shared AI account with public access disabled, and the gateway in a single deployment: ```text param location string = resourceGroup().location param workspaceName string = 'law-ai-gov' param aiAccountName string = 'aifoundrygovhub' param apimName string = 'apim-ai-gov' resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2023-09-01' = { name: workspaceName location: location properties: { retentionInDays: 90 features: { enableLogAccessUsingOnlyResourcePermissions: true } } } resource aiAccount 'Microsoft.CognitiveServices/accounts@2024-10-01' = { name: aiAccountName location: location sku: { name: 'S0' } kind: 'AIServices' identity: { type: 'SystemAssigned' } properties: { customSubDomainName: aiAccountName publicNetworkAccess: 'Disabled' networkAcls: { defaultAction: 'Deny' } } } resource apim 'Microsoft.ApiManagement/service@2024-05-01' = { name: apimName location: location sku: { name: 'Standard', capacity: 1 } identity: { type: 'SystemAssigned' } properties: { publisherEmail: 'security@contoso.com' publisherName: 'Contoso Security' } } ``` Deploy it and let Bicep do the waiting: ```bash az deployment group create \ --resource-group rg-ai-gov-hub \ --template-file main.bicep \ --parameters workspaceName=law-ai-gov aiAccountName=aifoundrygovhub apimName=apim-ai-gov ``` Two properties carry most of the governance value. `publicNetworkAccess: 'Disabled'` is the default you want on every AI account, and `enableLogAccessUsingOnlyResourcePermissions: true` removes the shared-key path to the log workspace so only identities with a role can read AI telemetry. ## From Sprawl to Governed: A Phased Roadmap Nobody turns this platform on in one change window. The migration works as three phases, and each phase produces an artifact that stands on its own. ### Phase 1: Discover and Freeze Run the Defender CSPM deployment across every subscription that might host AI resources, then export the AI inventory it produces. In parallel, freeze new AI subscriptions by assigning the policy that requires workloads to land in spoke subscriptions. Freezing before you can inventory produces shadow deployments, and inventorying before you freeze produces a moving target. Do both in the same sprint, applying the freeze to new resources only so nobody breaks a running service. ### Phase 2: Centralize the Choke Point Stand up the hub from the [Azure AI Landing Zones reference implementation](https://github.com/Azure/AI-Landing-Zones) if you would rather start from a maintained template than the minimal Bicep above, then migrate one workload rather than all of them. Pick the workload with the most obvious credential sprawl, because it demonstrates value fastest and gives you a real token cost baseline. The other teams keep running while you prove the pattern, and the instruction to each of them stays short: point your client at the alias and delete your provider key. ### Phase 3: Enforce and Prove It The final phase is where policy stops being advice. Attach the deny policies, move the content safety configuration from annotate to block, and start producing the evidence your auditors ask for. Expect pushback in this phase from a team whose application violates a content filter. The annotate-mode logs from Step 6 turn that argument into a data discussion about thresholds. ## Governance KPIs, Incident Response, and Your Next Move A platform that cannot report on itself gets cut at the next budget review. The table below lists the metrics worth tracking, along with how each one is produced. | KPI | What it measures | Source | | --- | --- | --- | | AI workloads under the inventory | Coverage of the AI bill of materials against known projects | Defender for Cloud AI inventory | | Requests routed through the gateway | Share of model traffic the platform actually sees | Gateway OpenTelemetry metrics | | Unapproved model deployments blocked | Guardrail coverage across subscriptions | Azure Policy compliance report | | Sensitive data incidents in AI prompts | DLP and classifier hits in prompts and responses | Purview activity explorer | | Token cost per business unit | Attribution accuracy for chargeback | Gateway cost metrics | | Mean time to revoke a rogue agent | Response capability, measured against a test revocation | Identity audit log and Azure Automation runbook | ### Give the SOC an AI Runbook Two scenarios belong in your incident response plan before you need them. An indirect prompt injection that reaches production is an investigation into which grounding document carried the instruction and which data the agent could reach, so the runbook needs document lineage and the agent’s role assignments in one place. Credential or key compromise on a model endpoint calls for an API key rotation plus a review of the gateway’s IP filtering rules. An agent you cannot revoke within an hour is an agent you do not control, so time both runbooks against a test revocation and write down the number. Both runbooks depend on the same evidence, so route the hub workspace data into [Microsoft Sentinel](https://learn.microsoft.com/en-us/azure/sentinel/overview), which Microsoft describes as “a cloud-native SIEM solution that delivers scalable, cost-efficient security across multicloud and multiplatform environments.” One dataset then serves both the alert correlation and the audit query. ### Your Next Three Moves Deploy the Log Analytics workspace and the Foundry account from the Bicep template today, because everything else attaches to them. Measure two numbers over the first thirty days: the gateway’s share of model traffic, and the token cost attributed to each business unit. Take those two numbers to the next funding cycle: a gateway share that accounts for your known estate and an attributed cost that lands where each team expected it is the evidence that gets the rollout funded. Share this article [Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fstop-ai-sprawl-in-azure-governance-hub%2F&text=How%20to%20Stop%20AI%20Sprawl%20in%20Azure%20with%20a%20Governance%20Hub)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fstop-ai-sprawl-in-azure-governance-hub%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fstop-ai-sprawl-in-azure-governance-hub%2F) ## Related Posts ![](https://adamtheautomator.com/wp-content/uploads/2026/08/featured_image.webp) ### [Entra PIM vs. Delinea vs. CyberArk: Don’t Buy the Wrong PAM](/entra-pim-vs-delinea-cyberark/) Entra PIM grants roles; Delinea and CyberArk vault credentials. Compare scope, session recording, audit evidence, and three-year cost. ![](https://adamtheautomator.com/wp-content/uploads/2026/06/featured_image-10.png) ### [Automate Your SOC: A Guide to Sentinel Playbook Generation](/automate-soc-guide-sentinel-playbook-generation/) Generate Microsoft Sentinel Python playbooks with AI, Integration Profiles, real-alert testing, and safe SOC automation rollout steps. ![](https://adamtheautomator.com/wp-content/uploads/2026/05/featured_image-5.webp) ### [Protect Sensitive Data with Microsoft Purview DLP Policies](/microsoft-purview-dlp-policies/) Configure Microsoft Purview DLP in M365 to protect sensitive data across Teams, Exchange, and SharePoint with sensitive information types and policy templates. ## Categories * [IT Ops](/category/it-ops/) * [Cloud](/category/cloud/) * [DevOps](/category/devops/) * [Home Ops](/category/home-ops/) * [Information Security](/category/infosec/) * [Software Development](/category/software-development/) ## Site * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)