The Complete Guide to FIPS 140-2 on Azure Application Gateway v2

Published:9 February 2026 - 12 min. read

Adam Bertram Image

Adam Bertram

Read more tutorials by Adam Bertram!

Audit your Active Directory for weak passwords and risky accounts. Run your free Specops scan now!

LinkedIn

You flipped the switch. The Application Gateway configuration page refreshed, showing “FIPS mode: Enabled” in clean sans-serif type. You sent the screenshot to your compliance team and closed the ticket.

Three weeks later, the audit report landed: “Non-compliant cryptographic implementation.” Your gateway was technically in FIPS mode, but your backend servers weren’t speaking the same language. Every request was dying at the handshake, and the monitoring dashboards had been showing 502 errors you never noticed because nobody told you what FIPS mode actually does.

FIPS 140-2 compliance on Azure Application Gateway v2 isn’t a checkbox—it’s an architectural decision that ripples through your entire infrastructure. The Federal Information Processing Standard (FIPS) 140-2 defines the security requirements for cryptographic modules used by U.S. federal agencies and regulated industries. When you enable FIPS mode on Application Gateway v2, you’re not just turning on a feature. You’re enforcing that every encryption, decryption, and signing operation uses only government-validated cryptographic modules and approved algorithms.

Here’s what actually happens, how to configure it without breaking your production environment, and the troubleshooting patterns that separate a successful FIPS deployment from one that fails spectacularly at 2 AM.

What FIPS Mode Actually Enforces

When you enable FIPS mode on Azure Application Gateway v2, Microsoft restricts the gateway to use only FIPS 140-2 validated cryptographic modules for all security operations. This isn’t a performance toggle or a logging preference—it’s a fundamental change to which code paths the gateway uses for TLS termination.

The gateway shifts to calling only the validated crypto libraries that NIST has certified. Those libraries support a specific subset of algorithms: AES-GCM, AES-CBC with ECDHE key exchange, and SHA-256 or SHA-384 hashing. Everything else—RC4, 3DES, MD5, even some perfectly functional modern ciphers—gets blocked at the platform level.

You can’t accidentally use a weak cipher suite when FIPS mode is active. If you try to configure TLS_RSA_WITH_3DES_EDE_CBC_SHA in a custom TLS policy, the gateway will either reject the configuration or silently ignore that cipher during handshakes. The Azure Portal won’t even show it as an option once FIPS mode is enabled.

Reality Check: Microsoft certifies the underlying Windows cryptographic modules with NIST for each OS release. When you enable FIPS mode, you’re trusting that certification chain—you can’t audit the module source code yourself.

The SKU Requirement Nobody Mentions Upfront

FIPS mode is natively available on Application Gateway v2 SKUs: Standard_v2 and WAF_v2 (Web Application Firewall). If you’re running a v1 gateway, FIPS mode technically exists but requires manual feature registration (AllowApplicationGatewayEnableFIPS) and PowerShell or ARM configuration–there’s no portal toggle. Given that v1 is deprecated for new deployments, the practical path forward is v2.

This means FIPS compliance often leads to an infrastructure upgrade anyway. You could enable FIPS on your existing v1 gateway through feature registration, but with v1 retirement scheduled for April 2026, you’ll need to migrate to v2 regardless. You need to provision a v2 gateway, migrate your listeners and backend pools, and deal with the fact that v2 uses a fundamentally different autoscaling model than v1’s fixed instance count.

The migration path isn’t terrible—Azure provides side-by-side deployment guidance—but it’s not a ten-minute change, either. Plan for testing. Your legacy clients that worked fine against the v1 gateway might not negotiate TLS successfully with the v2 FIPS-restricted cipher suite.

Regional Defaults: Azure Government vs. Public Cloud

In Azure Government (Fairfax) regions, FIPS mode is enabled by default. Every Application Gateway v2 you provision in those regions boots up with enableFips: true in the configuration. You can technically disable it, but doing so in a Federal Risk and Authorization Management Program (FedRAMP) environment is asking for a failed audit.

In commercial Azure (public cloud), FIPS mode is disabled by default. You need to explicitly enable it. This creates a trap: developers build and test in public Azure with FIPS off, then deploy to Azure Government where it’s on by default, and suddenly their application can’t connect to backends because the cipher negotiation fails.

Test with FIPS mode enabled in your development environment if your production workload runs in Azure Government. The cryptographic behavior is fundamentally different.

How to Enable FIPS Mode Without Breaking Everything

Enabling FIPS mode is a three-step process: verify your environment supports it, apply the configuration change, and validate the result. Here’s each step with the exact commands you need.

Prerequisites Check

Before you touch the FIPS toggle, verify your environment can actually support it. You need TLS 1.2 or TLS 1.3 support across your entire stack: Application Gateway, backend servers, health probes, and client applications.

Run this query against your Application Insights or Log Analytics workspace to identify clients still using TLS 1.0 or 1.1:

requests
| where timestamp > ago(30d)
| where toint(customDimensions.tlsVersion) < 12
| summarize count() by client_IP, customDimensions.tlsVersion

Any result here is a client that will break when you enable FIPS mode. Fix those integrations first—upgrading ancient Android devices or convincing a vendor to patch their API client is easier before you’ve broken production.

Enabling FIPS Mode via Azure CLI

The cleanest way to enable FIPS mode is through the Azure CLI, especially if you’re automating deployments or tracking infrastructure as code.

az network application-gateway update \
  --name your-gateway-name \
  --resource-group your-resource-group \
  --enable-fips true

This triggers a rolling update across all gateway instances. The process takes between 15 and 60 minutes depending on your instance count and current load. During this window, the gateway remains available—Azure applies the update to one update domain at a time, draining connections gracefully before restarting each instance.

Do not attempt other configuration changes while the FIPS update is running. Adding a listener, modifying a rule, or changing the Web Application Firewall (WAF) policy during this window can push the gateway into a “Failed” provisioning state that requires manual recovery.

Verify the setting applied:

az network application-gateway show \
  --name your-gateway-name \
  --resource-group your-resource-group \
  --query "properties.enableFips"

You should see true in the response. If you see null or false, the update didn’t apply—check the Activity Log for provisioning errors.

The ARM Template Approach

For repeatable deployments, define enableFips in your Azure Resource Manager (ARM) template or Bicep file:

{
  "type": "Microsoft.Network/applicationGateways",
  "apiVersion": "2023-02-01",
  "properties": {
    "enableFips": true,
    "sslPolicy": {
      "policyType": "Predefined",
      "policyName": "AppGwSslPolicy20220101"
    }
  }
}

The TLS policy specification here is critical. When FIPS mode is enabled, the gateway defaults to AppGwSslPolicy20220101, which supports only TLS 1.2 and TLS 1.3 with FIPS-compliant ciphers. If you try to specify a custom policy that includes non-FIPS algorithms, the deployment will either fail validation or the gateway will ignore those ciphers at runtime.

The TLS Policy Restriction You Need to Understand

Enabling FIPS mode doesn’t just restrict which cryptographic modules the gateway calls—it also locks down which TLS policies you can use. The AppGwSslPolicy20220101 policy becomes the default, and it’s strict.

Supported Cipher Suites in FIPS Mode

Cipher Suite Key Exchange Encryption Hash
TLSAES128GCMSHA256 N/A (TLS 1.3) AES-128-GCM SHA-256
TLSAES256GCMSHA384 N/A (TLS 1.3) AES-256-GCM SHA-384
TLSECDHERSAWITHAES128GCM_SHA256 ECDHE-RSA AES-128-GCM SHA-256
TLSECDHERSAWITHAES256GCM_SHA384 ECDHE-RSA AES-256-GCM SHA-384
TLSECDHERSAWITHAES128CBC_SHA256 ECDHE-RSA AES-128-CBC SHA-256
TLSECDHERSAWITHAES256CBC_SHA384 ECDHE-RSA AES-256-CBC SHA-384
TLSECDHEECDSAWITHAES128GCM_SHA256 ECDHE-ECDSA AES-128-GCM SHA-256
TLSECDHEECDSAWITHAES256GCM_SHA384 ECDHE-ECDSA AES-256-GCM SHA-384
TLSECDHEECDSAWITHAES128CBC_SHA256 ECDHE-ECDSA AES-128-CBC SHA-256
TLSECDHEECDSAWITHAES256CBC_SHA384 ECDHE-ECDSA AES-256-CBC SHA-384

Notice what’s missing: no RSA key exchange (non-ephemeral), no CBC mode with SHA-1, no DHE (Diffie-Hellman Ephemeral) ciphers despite what some older documentation suggests. The policy does include CBC suites, but only with SHA-256 or SHA-384–the older SHA-1 variants are excluded. If your backend server only supports TLS_RSA_WITH_AES_128_CBC_SHA, the Application Gateway can’t connect to it in FIPS mode. The cipher negotiation will fail, you’ll get 502 Bad Gateway errors, and the backend health probe will show “Unhealthy.”

Your backend servers need to support ECDHE key exchange with either AES-GCM or AES-CBC (SHA-256/SHA-384) encryption. On Windows Server, this means enabling TLS 1.2 cipher suites via registry or Group Policy. On Linux with OpenSSL, you need OpenSSL 1.0.2 or later with the GCM ciphers enabled in your ssl_ciphers directive.

Pro Tip: Test your backend TLS configuration before enabling FIPS mode. Use openssl s_client -connect your-backend:443 -cipher ECDHE-RSA-AES256-GCM-SHA384 to verify the backend can negotiate a FIPS-compliant cipher. If that handshake fails, enabling FIPS mode on the gateway will break your application.

The End-to-End TLS Trap

If you’re using Application Gateway for end-to-end TLS, enabling FIPS mode affects both the client-to-gateway and gateway-to-backend connections.

Here’s the architecture: the client connects to the Application Gateway’s public IP, the gateway terminates the TLS session using its certificate, inspects the HTTP request, then re-encrypts the traffic and forwards it to a backend server. In FIPS mode, both TLS sessions—client-to-gateway AND gateway-to-backend—must use FIPS-compliant ciphers.

This means your backend pool servers need to:

  1. Support TLS 1.2 or higher

  2. Support ECDHE key exchange

  3. Support AES-GCM or AES-CBC encryption

  4. Have a valid certificate the gateway can verify (or you’ve configured the gateway to trust a specific certificate chain)

If your backend is an older IIS 7.5 server defaulting to TLS 1.0, the gateway can’t connect. If your backend only offers TLS_RSA_WITH_AES_128_CBC_SHA, the handshake fails. You’ll see 502 errors, the backend health probe will report “Unhealthy,” and your application is down.

The fix: update your backend TLS configuration to match the FIPS-compliant cipher suite list. On Windows Server 2016 or later, enable the ECDHE ciphers and disable TLS 1.0/1.1 via IIS Crypto or manual registry edits. On Linux, update your NGINX or Apache ssl_protocols and ssl_ciphers directives to prefer TLSv1.2 TLSv1.3 and ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256.

Azure Key Vault Integration: The FIPS-Compliant Certificate Store

For true end-to-end FIPS compliance, you can’t just enable FIPS mode on the Application Gateway—you also need to store your TLS certificates in a FIPS 140-2 validated environment. Azure Key Vault Premium SKU provides hardware security module (HSM)-backed certificate storage. New keys are now protected by HSMs validated at FIPS 140-3 Level 3–an upgrade from the earlier FIPS 140-2 Level 2 hardware that legacy keys may still use.

Configuring Key Vault with Managed Identity

Instead of uploading a .pfx file directly to Application Gateway, store the certificate in Key Vault and grant the gateway’s managed identity permission to retrieve it.

Create a user-assigned managed identity:

az identity create \
  --name appgw-keyvault-identity \
  --resource-group your-resource-group

Assign the identity to your Application Gateway:

az network application-gateway identity assign \
  --gateway-name your-gateway-name \
  --resource-group your-resource-group \
  --identity appgw-keyvault-identity

Grant the identity access to Key Vault secrets:

IDENTITY_PRINCIPAL_ID=$(az identity show \
  --name appgw-keyvault-identity \
  --resource-group your-resource-group \
  --query principalId -o tsv)

az keyvault set-policy \
  --name your-keyvault-name \
  --object-id $IDENTITY_PRINCIPAL_ID \
  --secret-permissions get list

Now reference the certificate in your Application Gateway listener configuration using the Key Vault secret ID instead of uploading a .pfx file. The gateway will retrieve the certificate at runtime, perform the cryptographic operations using its FIPS-validated module, and you’ve maintained compliance from storage through execution.

Validation: Proving FIPS Mode Is Actually Active

Enabling the toggle isn’t enough—you need to verify the gateway is actually enforcing FIPS-compliant cryptography. Since Application Gateway is a managed platform-as-a-service (PaaS), you can’t SSH into the instance and check sysctl crypto.fips_enabled. Validation happens externally by testing the TLS handshake.

Testing with OpenSSL

From a Linux machine or Windows Subsystem for Linux (WSL) environment, use openssl s_client to test that the gateway accepts a FIPS-compliant cipher:

openssl s_client -connect your-gateway-fqdn:443 \
  -cipher ECDHE-RSA-AES256-GCM-SHA384

You should see a successful handshake with output like:

“`plain text
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384

Now test that the gateway rejects a non-FIPS cipher:

openssl s_client -connect your-gateway-fqdn:443 \
-cipher DES-CBC3-SHA

This should fail with:

```plain text
alert handshake failure

If the non-FIPS cipher test succeeds, FIPS mode isn’t actually enforced. Check that the enableFips property is true and that the rolling update completed successfully.

Using Nmap for Cipher Suite Enumeration

For a comprehensive audit, scan the gateway with Nmap’s ssl-enum-ciphers script:

nmap --script ssl-enum-ciphers -p 443 your-gateway-fqdn

The output should show only TLS 1.2 and TLS 1.3 protocols with AES-GCM and ECDHE ciphers. If you see TLS 1.0, TLS 1.1, or ciphers like TLS_RSA_WITH_RC4_128_SHA, FIPS mode isn’t active.

Troubleshooting the “Failed” Provisioning State

One of the most frustrating FIPS deployment issues is the gateway entering a “Failed” provisioning state. This happens when Azure can’t apply the configuration change—often because of a conflict between FIPS requirements and existing settings.

Common causes:

  1. Certificate configuration mismatch: You referenced a Key Vault secret, but the managed identity doesn’t have Get permissions. The gateway can’t retrieve the certificate, so the provisioning fails.

  2. Concurrent configuration changes: You enabled FIPS mode while simultaneously adding a listener or modifying a backend pool. The rolling update process can’t handle both changes, and the provisioning fails.

  3. Non-FIPS TLS policy conflict: You specified a custom TLS policy that includes non-FIPS ciphers. The gateway tries to apply both the FIPS restriction and your policy, can’t reconcile them, and fails.

Recovery Steps

If your gateway shows “Failed” in the Azure Portal:

Check the Activity Log for the specific error:

az monitor activity-log list \
  --resource-group your-resource-group \
  --resource-id "/subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.Network/applicationGateways/{name}" \
  --max-events 20

Look for error messages like “Key vault access denied” or “Invalid TLS policy.” Fix the underlying issue—grant Key Vault permissions, remove the conflicting TLS policy—then force a configuration refresh by updating a benign property like a tag:

az network application-gateway update \
  --name your-gateway-name \
  --resource-group your-resource-group \
  --set tags.refreshState="$(date +%s)"

Sometimes just triggering a new provisioning cycle clears the “Failed” state if the underlying issue is already resolved.

The 502 Bad Gateway Error Pattern

After enabling FIPS mode, the most common runtime issue is 502 errors. The Application Gateway successfully terminates the client TLS connection but can’t establish a secure connection to the backend.

Diagnosis

Check the backend health:

az network application-gateway show-backend-health \
  --name your-gateway-name \
  --resource-group your-resource-group

If the backend shows “Unhealthy” with a reason like “The server returned an SSL handshake failure,” the backend doesn’t support FIPS-compliant ciphers.

Enable Application Gateway diagnostics and send logs to Log Analytics:

az monitor diagnostic-settings create \
  --name appgw-diagnostics \
  --resource "/subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.Network/applicationGateways/{name}" \
  --workspace your-log-analytics-workspace-id \
  --logs '[{"category":"ApplicationGatewayAccessLog","enabled":true},{"category":"ApplicationGatewayPerformanceLog","enabled":true}]'

Query for SSL handshake failures:

AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS"
| where OperationName == "ApplicationGatewayAccess"
| where httpStatus_d == 502
| project TimeGenerated, backendServerList_s, errorInfo_s

If errorInfo_s contains “SSL handshake failed,” the backend’s TLS configuration is incompatible. Update the backend to support TLS 1.2 and ECDHE-AES-GCM ciphers.

FedRAMP and Federal Compliance Requirements

If you’re deploying Application Gateway to meet FedRAMP authorization requirements, FIPS mode is mandatory. FedRAMP Moderate and High baselines both require cryptographic modules to operate in a FIPS 140-2 approved mode.

The NIST Special Publication 800-53 control SC-13 (Cryptographic Protection) specifically requires FIPS-validated cryptography for federal information systems. Enabling FIPS mode on Application Gateway v2 helps satisfy this control for your application’s ingress traffic.

However, FIPS mode on the gateway alone isn’t sufficient for FedRAMP. You also need:

  • FIPS-compliant backend servers: Every system in the data path must use validated cryptographic modules.

  • FIPS-compliant Key Vault: Store certificates in Key Vault Premium (HSM-backed) or Managed HSM.

  • Audit logging: Enable Application Gateway diagnostics and retain logs online for at least 90 days (required across all FedRAMP baselines per AU-11), with additional offline retention per NARA guidelines.

FIPS mode handles the cryptographic enforcement at the application layer. You still need to address data at rest encryption, key rotation policies, and the dozens of other FedRAMP controls that aren’t directly related to TLS termination.

Warning: Enabling FIPS mode doesn’t automatically make your application FedRAMP compliant. It satisfies specific cryptographic controls, but FedRAMP authorization requires a complete security posture assessment, continuous monitoring, and third-party attestation.

Monitoring FIPS-Enabled Gateways

Remember that 2 AM scenario from the intro? This is how you avoid it. Once FIPS mode is active, monitor for TLS handshake failures that indicate client compatibility issues before they become outages.

Create a Log Analytics alert for SSL errors:

AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS"
| where errorInfo_s contains "SSL"
| summarize count() by bin(TimeGenerated, 5m), errorInfo_s
| where count_ > 10

This query identifies patterns of SSL errors—if you see a spike after enabling FIPS mode, you likely have clients or backends that don’t support the restricted cipher suite.

Track backend health status:

AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS"
| where OperationName == "ApplicationGatewayBackendHealth"
| where backendServerHealth_s != "Healthy"
| project TimeGenerated, backendServerList_s, backendServerHealth_s

Unhealthy backends after enabling FIPS mode usually indicate TLS configuration mismatches.

The FIPS Deployment Checklist

FIPS 140-2 compliance on Azure Application Gateway v2 is straightforward to enable but has architectural implications you need to plan for. The enableFips toggle restricts the gateway to using only government-validated cryptographic modules and FIPS-compliant cipher suites. This breaks compatibility with older TLS versions, weak ciphers, and backends that don’t support ECDHE key exchange with AES-GCM encryption.

Before enabling FIPS mode, audit your client and backend TLS capabilities. Verify that all systems support TLS 1.2 or higher and can negotiate the restricted cipher suite. Test the configuration in a non-production environment first—enabling FIPS mode triggers a rolling update that takes up to an hour, and recovering from a misconfigured backend during that window is painful.

Store your certificates in Azure Key Vault Premium for end-to-end FIPS compliance, grant the Application Gateway’s managed identity access to retrieve them, and validate the configuration by testing TLS handshakes with OpenSSL. Monitor for 502 errors and backend health issues after the change.

FIPS mode isn’t optional for FedRAMP or most federal contracts. Plan the migration, test thoroughly, and don’t assume that clicking “Enable” is the same as being compliant. The toggle enforces the cryptography—you still need to verify every component in your application stack supports it.

Hate ads? Want to support the writer? Get many of our tutorials packaged as an ATA Guidebook.

Explore ATA Guidebooks

More from ATA Learning and Partners

Looks like you're offline!