Taming AI Tool Sprawl: A PowerShell Guide to Auditing and Governing Unauthorized AI Applications

Published:24 June 2026 - 8 min. read

Adam Bertram Image

Adam Bertram

Read more tutorials by Adam Bertram!

Audit Active Directory for stale users, weak passwords, and other security risks with Specops Password Auditor.

LinkedIn

AI tools show up the same way every other shadow IT tool shows up: one helpful employee tries one app, another grants consent to a browser plugin, and suddenly security is chasing a dozen “just testing it” services.

The good news? If your organization uses Microsoft Entra ID, Microsoft Graph, and Microsoft Defender for Cloud Apps, you already have useful evidence. You can audit enterprise applications, delegated consent grants, application permissions, and recent sign-ins with PowerShell before you decide what to approve, restrict, or block.

In this tutorial, you will build a repeatable PowerShell audit that finds AI-related applications in Microsoft Entra ID, scores the risky ones, and produces a governance report your security or IT team can act on.

Prerequisites

This tutorial is a hands-on demonstration. To follow along, you will need:

  • PowerShell 7 or Windows PowerShell 5.1.

  • The Microsoft Graph PowerShell SDK installed.

  • A Microsoft Entra account with permissions to read applications and audit logs.

  • Microsoft Graph consent for the scopes used in this tutorial.

  • Optional: Microsoft Defender for Cloud Apps if you want network-level cloud discovery and block scripts.

The examples in this tutorial are read-only. They export CSV reports and do not remove applications, revoke consent, or publish firewall rules.

Understanding What PowerShell Can and Cannot See

Start with the right mental model. Microsoft Graph can show you AI tools that interact with Microsoft Entra ID: enterprise applications, OAuth consent grants, application permissions, and sign-in activity. The List servicePrincipals Microsoft Graph API returns service principal objects from the tenant by using GET /servicePrincipals.

Graph can also list delegated permissions through GET /oauth2PermissionGrants, and application permissions granted to a service principal through GET /servicePrincipals/{id}/appRoleAssignments.

But Graph does not magically see every visit to every AI website. If someone uses a personal account at an AI site and never authenticates through your tenant, you need network, endpoint, or CASB telemetry. Microsoft documents that Defender for Cloud Apps Cloud Discovery provides a dashboard for cloud app usage, app risk levels, top users, source IPs, and discovered apps.

Think of this as two lanes:

  • Identity lane: What apps have identity objects, consent, permissions, or sign-ins in Entra ID?

  • Network lane: What AI destinations are users accessing, even when Entra ID is not involved?

You will build the identity lane first.

Installing and Connecting to Microsoft Graph

Open PowerShell as an account that can install modules, and install the SDK if needed.

Install-Module Microsoft.Graph -Scope CurrentUser

Next, connect to Microsoft Graph with the least read permissions required for this audit.

$Scopes = @(
    'Application.Read.All',
    'Directory.Read.All',
    'AuditLog.Read.All'
)

Connect-MgGraph -Scopes $Scopes

The Graph documentation lists Application.Read.All as a least-privileged permission for listing service principals. It lists Directory.Read.All as a least-privileged permission for listing delegated permission grants, and AuditLog.Read.All for listing sign-ins. You might need an Entra role such as Global Reader, Directory Readers, Reports Reader, Security Reader, or another supported role depending on which API you call and how your tenant is configured.

Run this quick check to confirm the connection context.

Get-MgContext | Select-Object Account, TenantId, Scopes

If the scopes are missing, disconnect and reconnect with the scope list above.

Creating an AI App Keyword List

No Microsoft API classifies every service principal as “AI.” You need a starting watchlist. Keep the list visible and editable so your governance process can improve over time.

Create a folder for the report and define AI-related keywords.

$ReportPath = Join-Path $HOME 'AI-App-Audit'
New-Item -Path $ReportPath -ItemType Directory -Force | Out-Null

$AiKeywords = @(
    'openai',
    'chatgpt',
    'copilot',
    'claude',
    'anthropic',
    'gemini',
    'bard',
    'perplexity',
    'midjourney',
    'stability',
    'jasper',
    'notion ai',
    'grammarly',
    'otter',
    'fireflies',
    'descript',
    'synthesia'
)

This list intentionally includes broad product and vendor names. Expect some false positives. A governance report is better when a reviewer can approve or reject findings instead of trusting a hidden matching rule.

Finding AI-Related Enterprise Applications

Now query service principals and filter them locally against the keyword list. This approach avoids guessing unsupported API filters and works well for a scheduled report.

$ServicePrincipals = Get-MgServicePrincipal -All -Property @(
    'id',
    'appId',
    'displayName',
    'appOwnerOrganizationId',
    'accountEnabled',
    'createdDateTime',
    'publisherName',
    'servicePrincipalType',
    'tags'
)

$AiServicePrincipals = foreach ($Sp in $ServicePrincipals) {
    $SearchText = @(
        $Sp.DisplayName
        $Sp.PublisherName
        $Sp.AppId
        ($Sp.Tags -join ' ')
    ) -join ' '

    $Matches = $AiKeywords | Where-Object {
        $SearchText -match [regex]::Escape($_)
    }

    if ($Matches) {
        [pscustomobject]@{
            DisplayName            = $Sp.DisplayName
            PublisherName          = $Sp.PublisherName
            AppId                  = $Sp.AppId
            ObjectId               = $Sp.Id
            AccountEnabled         = $Sp.AccountEnabled
            CreatedDateTime        = $Sp.CreatedDateTime
            ServicePrincipalType   = $Sp.ServicePrincipalType
            MatchedKeywords        = ($Matches -join ', ')
        }
    }
}

$AiServicePrincipals |
    Sort-Object DisplayName |
    Export-Csv -Path (Join-Path $ReportPath 'ai-service-principals.csv') -NoTypeInformation

$AiServicePrincipals | Format-Table DisplayName, PublisherName, AccountEnabled, MatchedKeywords -AutoSize

At this point, you have the first practical report: AI-flavored applications registered in or added to your tenant. This list catches enterprise apps that were added through SSO, OAuth consent, or admin configuration.

Auditing Delegated OAuth Consent

Application presence is only part of the story. An app with no consent and no sign-ins is less urgent than an app that can read user data.

Microsoft Graph exposes delegated permission grants as oAuth2PermissionGrant objects. The documentation describes these grants as delegated permissions granted for client applications to access APIs on behalf of signed-in users.

Run the following script to map delegated grants back to the AI service principals you found.

$OAuthGrants = Get-MgOauth2PermissionGrant -All -Property @(
    'id',
    'clientId',
    'consentType',
    'principalId',
    'resourceId',
    'scope'
)

$AiGrantReport = foreach ($App in $AiServicePrincipals) {
    $AppGrants = $OAuthGrants | Where-Object { $_.ClientId -eq $App.ObjectId }

    foreach ($Grant in $AppGrants) {
        [pscustomobject]@{
            DisplayName     = $App.DisplayName
            AppId           = $App.AppId
            ObjectId        = $App.ObjectId
            ConsentType     = $Grant.ConsentType
            PrincipalId     = $Grant.PrincipalId
            ResourceId      = $Grant.ResourceId
            Scope           = $Grant.Scope
        }
    }
}

$AiGrantReport |
    Export-Csv -Path (Join-Path $ReportPath 'ai-delegated-consent.csv') -NoTypeInformation

$AiGrantReport |
    Sort-Object DisplayName, ConsentType |
    Format-Table DisplayName, ConsentType, Scope -Wrap

Pay special attention to ConsentType. AllPrincipals means consent applies tenant-wide. Principal means consent applies to one user. Both can matter, but tenant-wide consent deserves faster review.

Checking Application Permissions

Delegated permissions run as a signed-in user. Application permissions run as the application itself and can be more sensitive in automation-heavy environments.

Graph lists app role assignments granted to a service principal through the app role assignment endpoint. The docs also note that app roles assigned to service principals are known as application permissions.

Use Invoke-MgGraphRequest so you can call the documented endpoint directly.

$AiAppRoleReport = foreach ($App in $AiServicePrincipals) {
    $Uri = "/servicePrincipals/$($App.ObjectId)/appRoleAssignments"

    try {
        $Assignments = Invoke-MgGraphRequest -Method GET -Uri $Uri
    }
    catch {
        Write-Warning "Could not read app role assignments for $($App.DisplayName): $($_.Exception.Message)"
        continue
    }

    foreach ($Assignment in $Assignments.value) {
        [pscustomobject]@{
            DisplayName   = $App.DisplayName
            AppId         = $App.AppId
            ObjectId      = $App.ObjectId
            ResourceId    = $Assignment.resourceId
            ResourceName  = $Assignment.resourceDisplayName
            AppRoleId     = $Assignment.appRoleId
            CreatedDate   = $Assignment.createdDateTime
        }
    }
}

$AiAppRoleReport |
    Export-Csv -Path (Join-Path $ReportPath 'ai-application-permissions.csv') -NoTypeInformation

$AiAppRoleReport | Format-Table DisplayName, ResourceName, AppRoleId -AutoSize

The AppRoleId value is not friendly by itself. For a deeper report, resolve the resource service principal and map AppRoleId to its appRoles collection. But even this report is useful because it shows which AI-related apps have application permission assignments at all.

Reviewing Recent Sign-Ins

An enterprise app that exists but has no recent activity might be a cleanup candidate. An unsanctioned app with fresh sign-ins needs a governance conversation.

The List signIns API uses GET /auditLogs/signIns. Microsoft recommends using a $filter time range to avoid request timeouts.

Search the last 14 days of sign-ins for your AI app display names.

$StartTime = (Get-Date).ToUniversalTime().AddDays(-14).ToString('yyyy-MM-ddTHH:mm:ssZ')
$EndTime   = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
$Filter    = "createdDateTime ge $StartTime and createdDateTime le $EndTime"

$RecentSignIns = Get-MgAuditLogSignIn -All -Filter $Filter -Property @(
    'createdDateTime',
    'userDisplayName',
    'userPrincipalName',
    'appDisplayName',
    'appId',
    'ipAddress',
    'clientAppUsed',
    'conditionalAccessStatus',
    'status'
)

$AiSignIns = foreach ($SignIn in $RecentSignIns) {
    $Matches = $AiKeywords | Where-Object {
        $SignIn.AppDisplayName -match [regex]::Escape($_)
    }

    if ($Matches) {
        [pscustomobject]@{
            CreatedDateTime          = $SignIn.CreatedDateTime
            UserPrincipalName        = $SignIn.UserPrincipalName
            UserDisplayName          = $SignIn.UserDisplayName
            AppDisplayName           = $SignIn.AppDisplayName
            AppId                    = $SignIn.AppId
            IpAddress                = $SignIn.IpAddress
            ClientAppUsed            = $SignIn.ClientAppUsed
            ConditionalAccessStatus  = $SignIn.ConditionalAccessStatus
            ErrorCode                = $SignIn.Status.ErrorCode
            FailureReason            = $SignIn.Status.FailureReason
            MatchedKeywords          = ($Matches -join ', ')
        }
    }
}

$AiSignIns |
    Export-Csv -Path (Join-Path $ReportPath 'ai-signins-last-14-days.csv') -NoTypeInformation

$AiSignIns |
    Group-Object AppDisplayName |
    Sort-Object Count -Descending |
    Select-Object Count, Name

This output helps you separate idle registrations from active usage. If an app is active and unauthorized, do not start by deleting it. First identify the business owner, user population, permissions, and data exposure.

Building a Governance Score

Security teams need prioritization, not three disconnected CSV files. The following script creates a simple score based on tenant-wide consent, application permissions, recent sign-ins, and whether the app is enabled.

$GovernanceReport = foreach ($App in $AiServicePrincipals) {
    $Delegated = @($AiGrantReport | Where-Object { $_.ObjectId -eq $App.ObjectId })
    $AppRoles  = @($AiAppRoleReport | Where-Object { $_.ObjectId -eq $App.ObjectId })
    $SignIns   = @($AiSignIns | Where-Object { $_.AppId -eq $App.AppId -or $_.AppDisplayName -eq $App.DisplayName })

    $Score = 0
    $Reasons = New-Object System.Collections.Generic.List[string]

    if ($App.AccountEnabled) {
        $Score += 1
        $Reasons.Add('Enterprise app is enabled')
    }

    if ($Delegated | Where-Object { $_.ConsentType -eq 'AllPrincipals' }) {
        $Score += 3
        $Reasons.Add('Tenant-wide delegated consent exists')
    }

    if ($AppRoles.Count -gt 0) {
        $Score += 3
        $Reasons.Add('Application permissions exist')
    }

    if ($SignIns.Count -gt 0) {
        $Score += 2
        $Reasons.Add("Recent sign-ins found: $($SignIns.Count)")
    }

    $Recommendation = switch ($Score) {
        { $_ -ge 6 } { 'Review immediately' ; break }
        { $_ -ge 3 } { 'Review in next governance cycle' ; break }
        default      { 'Inventory and monitor' }
    }

    [pscustomobject]@{
        DisplayName       = $App.DisplayName
        PublisherName     = $App.PublisherName
        AppId             = $App.AppId
        AccountEnabled    = $App.AccountEnabled
        DelegatedGrants   = $Delegated.Count
        AppPermissions    = $AppRoles.Count
        RecentSignIns     = $SignIns.Count
        Score             = $Score
        Recommendation    = $Recommendation
        Reasons           = ($Reasons -join '; ')
        MatchedKeywords   = $App.MatchedKeywords
    }
}

$GovernanceReport |
    Sort-Object Score -Descending, DisplayName |
    Export-Csv -Path (Join-Path $ReportPath 'ai-governance-summary.csv') -NoTypeInformation

$GovernanceReport |
    Sort-Object Score -Descending, DisplayName |
    Format-Table DisplayName, Score, Recommendation, RecentSignIns, DelegatedGrants, AppPermissions -AutoSize

Now you have a single governance summary that an IT manager, security engineer, or compliance reviewer can read without opening every raw export.

Adding Defender for Cloud Apps to Catch Web-Only Usage

Identity data is necessary, but it is not enough. If your users visit AI services without Entra authentication, use Defender for Cloud Apps Cloud Discovery.

Microsoft says Cloud Discovery APIs can automate log uploads, list and interact with discovered apps, and generate block scripts for firewall or Secure Web Gateway enforcement. The upload flow uses three consecutive API calls: initiate file upload, perform file upload, and finalize file upload. Microsoft also documents a Generate block script endpoint at GET /api/discovery_block_scripts/.

Use this PowerShell pattern to call the documented block script API. Replace the tenant host and token values with your Defender for Cloud Apps details.

$PortalHost = 'contoso.us.portal.cloudappsecurity.com'
$ApiToken = Read-Host 'Enter Defender for Cloud Apps API token' -AsSecureString
$TokenText = [Runtime.InteropServices.Marshal]::PtrToStringAuto(
    [Runtime.InteropServices.Marshal]::SecureStringToBSTR($ApiToken)
)

$Headers = @{
    Authorization = "Token $TokenText"
}

$Uri = "https://$PortalHost/api/discovery_block_scripts/?format=102&type=banned"
Invoke-RestMethod -Method GET -Uri $Uri -Headers $Headers -OutFile (Join-Path $ReportPath 'defender-cloud-apps-block-script.txt')

Do not import the generated block script blindly. Treat it as a change artifact. Review the apps marked unsanctioned, test in a pilot group, and coordinate with the network team before enforcement.

Turning the Report into a Governance Process

A report is only useful if it changes behavior. Use the CSVs you generated to drive a lightweight review workflow:

  1. Approve known tools. Add approved AI applications to a sanctioned catalog with owner, data classification, and allowed use cases.

  2. Challenge risky consent. Review tenant-wide delegated consent and application permissions before focusing on low-risk sign-ins.

  3. Limit who can approve. Microsoft Entra supports admin consent workflow, which lets users request admin review when they cannot consent directly.

  4. Document exceptions. If a team needs a nonstandard AI tool, require a business owner and a renewal date.

  5. Repeat the audit. Schedule the PowerShell report weekly and compare new findings against your approved catalog.

If you need to grant tenant-wide consent after review, use the documented Microsoft Entra admin consent process. Avoid granting broad consent directly from a vendor prompt during a rushed support call.

Wrapping Up

AI tool sprawl is not a single product problem. It is a visibility and governance problem. PowerShell gives you a practical way to start with evidence: enterprise applications, delegated consent, application permissions, and sign-in activity from Microsoft Graph.

You also learned where Graph stops. For browser-only or personal-account usage, bring in Defender for Cloud Apps Cloud Discovery and your network telemetry. Combine both lanes and you can replace guesswork with a repeatable process: discover, score, review, approve, restrict, and monitor.

Run the audit on a schedule, tune the keyword list, and keep the output tied to real decisions. That is how you tame AI tool sprawl without turning every new productivity experiment into a security fire drill.

Hate ads? Want to support the writer? Get many of our tutorials packaged as an ATA Guidebook.

Explore ATA Guidebooks

More from ATA Learning and Partners

Looks like you're offline!