--- title: Automate Terraform with Azure DevOps description: Automate Terraform with Azure DevOps Pipelines: structure a multi-environment repo, store remote state in Azure Storage with locking, and deploy through approval-gated pipelines with drift detection. canonical: https://adamtheautomator.com/automate-terraform-azure-devops/ --- # Automate Terraform with Azure DevOps > Automate Terraform with Azure DevOps Pipelines: structure a multi-environment repo, store remote state in Azure Storage with locking, and deploy through approval-gated pipelines with drift detection. Source: https://adamtheautomator.com/automate-terraform-azure-devops/ --- - Automate Terraform with Azure DevOps Tap to hide Home - Tutorials - Guidebooks - Instructors - Get Paid to Write - Advertising - Recommended Resources - About Adam Search for: - - - - # Automate Terraform with Azure DevOps Published:31 August 2026 - 12 min. read - Automation - Azure - Azure DevOps - Azure Pipelines [](https://adamtheautomator.com/author/adam-bertram/) [Adam Bertram](https://adamtheautomator.com/author/adam-bertram/) Read [more tutorials](https://adamtheautomator.com/author/adam-bertram/) by Adam Bertram! - - [](https://specopssoft.com/product/specops-password-auditor/?utm_source=adamtheautomator&utm_medium=referral&utm_campaign=adamtheautomator_referral_na&utm_content=display) Audit Active Directory for stale users, weak passwords, and other security risks with [Specops Password Auditor](https://specopssoft.com/product/specops-password-auditor/?utm_source=adamtheautomator&utm_medium=referral&utm_campaign=adamtheautomator_referral_na&utm_content=text). Table of Contents - Prerequisites - Structuring Your Terraform Repository - Storing Terraform State in Azure StorageLocking and Securing the State File - Creating a Service Connection with a Service Principal - Building the Pipeline: Validate and PlanPublishing the Plan as an Artifact - Applying Terraform with Approval Gates - Managing Multiple Environments - Securing the Pipeline with Workload Identity Federation - Detecting Drift with a Scheduled Pipeline - Troubleshooting Common Pipeline Failures - Wrapping Up XFacebookLinkedIn You're the one running terraform apply from your laptop and calling it a deployment strategy. This post is for you. Maybe you're past that phase and the Azure portal is your real problem: someone clicks through a few blades, changes a network security group rule, and the "who did this" question starts a group chat that never ends. Both roads hit the same wall: hand-run infrastructure doesn't scale past one person, and unversioned cloud changes are undocumented outages waiting to happen. The answer is to treat your infrastructure like your application code. Infrastructure as Code (IaC) means defining your cloud resources in declarative configuration files, and [Terraform](https://learn.microsoft.com/en-us/azure/developer/terraform/overview) turns those files into real Azure resources. Pair it with [Azure DevOps Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/get-started/what-is-azure-pipelines) and every change flows through the same version-controlled, peer-reviewed pipeline your code does, with approvals gating each step, and if Azure DevOps is new to you, [Udemy's Azure DevOps Fundamentals for Beginners course](https://trk.udemy.com/c/1454808/3281541/39854?u=https%3A%2F%2Fwww.udemy.com%2Fcourse%2Fazure-devops-for-beginners%2F) covers the basics. In this tutorial, you'll build that path end to end. First, you'll structure a Terraform repository for multiple environments and store remote state in [Azure Storage](https://learn.microsoft.com/en-us/azure/storage/common/storage-introduction) with locking. Then you'll wire a service connection for authentication and a two-stage pipeline that validates and plans every change, publishes the plan as an artifact, and applies it only after a human approves. You'll finish by hardening the pipeline with workload identity federation and a scheduled drift check, so that deploying infrastructure runs through the same review-and-approve cycle as a code change, and if you want to take your Azure DevOps skills further, [Pluralsight's Azure DevOps Solutions (AZ-400) path](https://www.anrdoezrs.net/links/7627660/type/dlg/sid/ata-ps-azure-devops-solutions-az-400-path/https://www.pluralsight.com/search?q=Azure%20DevOps%20Solutions%20(AZ-400)%20Path) is the natural next step. ## Prerequisites To follow along hands-on, you'll need: - An Azure subscription with permission to create resources and service principals. If you don't have one, a free account works; just skip the production-hardening steps and keep everything in one resource group. - An Azure DevOps organization and project with a repository. Any project-level access works for the first pipeline; creating environments and approval gates needs Project Administrator. - The Azure CLI installed and logged in with az login. Every setup step here uses the CLI so you can re-run it; nothing requires clicking through the portal. - Terraform installed locally (optional). The pipeline installs its own pinned version, but having it locally helps you validate code before pushing. ## Structuring Your Terraform Repository Decide where your Terraform code lives before any pipeline exists. The most common failure I see in teams starting out is one giant folder with a single state file for everything, which guarantees a dev experiment and a production change fight over the same resources. Separate code by environment from day one with a layout like this: terraform/ ├── environments/ │ ├── dev/ │ │ └── terraform.tfvars │ └── prod/ │ └── terraform.tfvars ├── modules/ │ └── networking/ ├── main.tf ├── variables.tf ├── outputs.tf └── backend.tf azure-pipelines.yml The root main.tf, variables.tf, and outputs.tf hold the infrastructure definition. The environments/ folders hold only environment-specific values (resource names, SKUs, sizes), passed in with the [-var-file flag](https://developer.hashicorp.com/terraform/language/values/variables#variables-on-the-command-line) at plan time. modules/ holds reusable building blocks so dev and prod call the same code with different inputs. Keep azure-pipelines.yml in the same repository as the Terraform code. When the pipeline definition itself is versioned, a change to your deployment process needs the same peer review as a change to your infrastructure. ## Storing Terraform State in Azure Storage Terraform keeps a state file (terraform.tfstate) that maps your configuration to the real resources in Azure. If you've only run Terraform locally, that file lives on your laptop. That works fine until the laptop dies, a teammate runs the same config, or a pipeline agent spins up, plans, and vanishes with the state on its ephemeral disk. All three scenarios corrupt or lose the source of truth for what's actually deployed. The fix is a remote backend. For Azure, that means pointing Terraform at a blob in an Azure Storage account, configured in backend.tf: terraform { backend "azurerm" { resource_group_name = "rg-tfstate" storage_account_name = "saterraformstate" container_name = "tfstate" key = "prod/terraform.tfstate" } } Create the storage account and container once, before the pipeline ever runs: az group create --name rg-tfstate --location eastus az storage account create --name saterraformstate --resource-group rg-tfstate --sku Standard_LRS --allow-blob-public-access false az storage container create --name tfstate --account-name saterraformstate The backend block's parameters control where each environment's state lands: Parameter What it controls resource_group_name The resource group holding the storage account storage_account_name The (globally unique) storage account name container_name The blob container, e.g. tfstate key The state file blob name, e.g. prod/terraform.tfstate ### Locking and Securing the State File Two behaviors of the azurerm backend save you from the worst Terraform accidents. First, state locking: when a pipeline runs terraform plan or terraform apply, Terraform takes a lease on the state blob, so a second run that starts mid-deploy fails with a "state is locked" error instead of two processes writing the same file. Unlike AWS, where locking needs a separate DynamoDB table, [the azurerm backend](https://developer.hashicorp.com/terraform/language/settings/backends/azurerm) does this natively, so you don't need extra infrastructure. Second, treat the storage account as a high-security boundary. The state file stores plaintext JSON, and that JSON contains connection strings, initial passwords, and your entire network map. Assign the [Storage Blob Data Contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/storage#storage-blob-data-contributor) role only to the pipeline's identity, enable blob versioning and soft delete so a corrupted file can be rolled back, and disable public network access on the account. Warning: If you ever see a terraform.tfstate committed to a git repository, assume those secrets are compromised and rotate them. State belongs in the backend, never in source control. ## Creating a Service Connection with a Service Principal Your pipeline needs an identity in Azure, and Azure DevOps needs a way to hand that identity to the pipeline. The traditional approach is a [service connection backed by a service principal](https://learn.microsoft.com/en-us/azure/devops/pipelines/library/connect-to-azure). Create the service principal with the Azure CLI: az ad sp create-for-rbac --name "terraform-ado-sp" --role Contributor --scopes /subscriptions/ The output gives you an appId and a password (the client secret). The --role Contributor --scopes pair grants this identity permission to manage resources in your subscription; scope it tighter (to a resource group or a specific set of resource groups) if your pipeline only deploys to part of the subscription. Now grant that same principal access to the state storage account, separate from its resource-management role: az role assignment create \ --assignee \ --role "Storage Blob Data Contributor" \ --scope /subscriptions//resourceGroups/rg-tfstate Pro Tip: Scope the state-storage role separately from the subscription role. If the pipeline's subscription-level credentials leak, the attacker still can't read your state files. Least privilege is what keeps one compromise from becoming full infrastructure access. Back in Azure DevOps, create an Azure Resource Manager service connection, choose Service principal (manual), and paste in the subscription ID, appId, and secret. Name it azure-terraform-service-connection so you can reference it in YAML; Azure DevOps injects the credentials into the job's environment, and the secret never appears in your repository. If you prefer scripting the setup, the [az devops service-endpoint azurerm create](https://learn.microsoft.com/en-us/cli/azure/devops/service-endpoint/azurerm) command creates the same connection from the command line. ## Building the Pipeline: Validate and Plan With state and authentication in place, you can automate the boring parts. The pipeline below lives in azure-pipelines.yml and runs on every push to main and on every pull request: trigger: branches: include: - main pool: vmImage: ubuntu-latest variables: - group: terraform-common - name: terraformVersion value: "1.9.8" stages: - stage: ValidateAndPlan displayName: "Validate and Plan" jobs: - job: Plan steps: - task: TerraformInstaller@1 displayName: "Install Terraform" inputs: terraformVersion: $(terraformVersion) - task: TerraformTaskV4@4 displayName: "terraform init" inputs: provider: "azurerm" command: "init" backendServiceArm: "azure-terraform-service-connection" backendAzureRmResourceGroupName: "rg-tfstate" backendAzureRmStorageAccountName: "saterraformstate" backendAzureRmContainerName: "tfstate" backendAzureRmKey: "prod/terraform.tfstate" - task: TerraformTaskV4@4 displayName: "terraform validate" inputs: provider: "azurerm" command: "validate" - task: TerraformTaskV4@4 displayName: "terraform plan" inputs: provider: "azurerm" command: "plan" environmentServiceNameAzureRM: "azure-terraform-service-connection" commandOptions: "-out=$(Build.ArtifactStagingDirectory)/tfplan" workingDirectory: "$(System.DefaultWorkingDirectory)/terraform" - task: PublishBuildArtifacts@1 displayName: "Publish plan artifact" inputs: PathtoPublish: "$(Build.ArtifactStagingDirectory)" ArtifactName: "tfplan" The tasks come from the [Microsoft DevLabs Terraform extension](https://marketplace.visualstudio.com/items?itemName=ms-devlabs.custom-terraform-tasks) (install it from the Visual Studio Marketplace before your first run; [HashiCorp's official extension](https://github.com/hashicorp/azure-pipelines-extension-terraform) is the alternative). A few details matter. The plan task produces output like the run shown below, and that output is what your approvers review before anything is applied: Terraform plan output - TerraformInstaller@1 pins the version. It downloads a specific Terraform binary onto the agent's PATH, so your pipeline doesn't silently pick up a breaking release when Microsoft updates the hosted image. Bump terraformVersion deliberately. - terraform init connects the backend. It authenticates to Azure, prepares the state blob, and downloads providers. This is the first task that can fail with a 403 if the service connection's role-based access control (RBAC) is wrong. - terraform validate is your cheap safety net. It runs offline syntax and structure checks, catching errors before anything touches the cloud. - terraform plan is the review artifact. The -out flag saves the exact execution plan as a binary file instead of just printing it. Set TF_IN_AUTOMATION=true as a pipeline variable and Terraform tailors its output for automation; the -auto-approve flag on the saved-plan apply is what skips the prompt, and the environment variable is documented for exactly this automated use case. ### Publishing the Plan as an Artifact The PublishBuildArtifacts@1 step is what makes this design safe. Microsoft-hosted agents are ephemeral: the agent that ran plan is destroyed when the job ends. If the apply stage ran terraform apply again, Terraform would calculate a brand-new plan. Between your review and that recalculation, someone could have changed something in the portal, producing an apply that does things nobody approved. Publishing the .tfplan binary as a pipeline artifact means the apply stage applies the exact reviewed changes. If the cloud state drifts after the plan was saved, Terraform detects the stale plan and fails instead of silently doing something new. ## Applying Terraform with Approval Gates Automatic apply on every push to main is how you get "the pipeline deleted the staging database" stories. Production changes need a human checkpoint, and Azure DevOps gives you that with environments and approval checks. The apply stage targets an environment instead of a plain job: - stage: Apply displayName: "Apply to Production" dependsOn: ValidateAndPlan condition: succeeded() jobs: - deployment: ApplyProd environment: "prod" strategy: runOnce: deploy: steps: - download: current artifact: tfplan - task: TerraformTaskV4@4 displayName: "terraform apply" inputs: provider: "azurerm" command: "apply" environmentServiceNameAzureRM: "azure-terraform-service-connection" commandOptions: "-auto-approve $(Pipeline.Workspace)/tfplan/tfplan" workingDirectory: "$(System.DefaultWorkingDirectory)/terraform" Create the prod environment in Azure DevOps under Pipelines → Environments, then add an Approvals check and name the approvers (usually a senior engineer or a small DevOps group). Environments and approval checks are portal-first, with no CLI equivalent, so this short setup is the one click-through the tutorial asks for. When the pipeline reaches the Apply stage, it pauses and notifies them; the approvers review the plan output from the previous stage (exactly which resources get created, modified, or destroyed) and approve or reject. The [approvals and checks documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals) covers setting a timeout so a forgotten approval doesn't hold the queue hostage. The approval check appears in the environment settings as shown below, listing the gate the pipeline must clear before applying: Configuring prod approvals The -auto-approve flag is safe here only because you're applying a saved plan file. Terraform skips its interactive confirmation prompt and applies exactly what the reviewers approved, rather than recomputing anything on the spot. Reality Check: An apply stage without a saved plan artifact is just a faster way to make unreviewed changes. The approval gate reviews the plan; the artifact guarantees the apply matches it. Remove either one and the gate approves nothing real. ## Managing Multiple Environments Dev, staging, and prod should never share a state file, and ideally never share a service connection. Each environment gets its own key in the backend config, its own .tfvars file, and its own approval posture: terraform { backend "azurerm" { resource_group_name = "rg-tfstate" storage_account_name = "saterraformstate" container_name = "tfstate" key = "dev/terraform.tfstate" } } A dev state file and a prod state file in the same account are fine; many teams go further and use a separate storage account per environment so a misconfigured account can't expose prod's state. What you should not do is rely on [Terraform workspaces](https://developer.hashicorp.com/terraform/language/state/workspaces) to separate environments. Workspaces isolate state, but not credentials; a single service connection with Contributor on the whole subscription lets a dev pipeline touch prod regardless of workspace. Environment-specific values belong in [variable groups](https://learn.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups), not in the repository. Create a terraform-prod variable group, link it to [Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview) if the values are secrets, and reference it in the pipeline: variables: - group: terraform-prod Pass the environment's variable file at plan time with -var-file=environments/prod/terraform.tfvars, and keep the file free of secrets; the variable group supplies those. Also set batch: true on the trigger if your team commits frequently; it collapses rapid pushes into one run and prevents two pipelines from racing for the state lock. ## Securing the Pipeline with Workload Identity Federation The service principal approach works, but it has a recurring weakness: a long-lived client secret that must be rotated before it expires or leaks. Workload identity federation (OIDC) replaces that static secret with short-lived tokens. You configure a federated credential on a service principal (or managed identity) declaring: "trust tokens from this Azure DevOps organization, project, and service connection." When the pipeline runs, Azure DevOps mints a signed token for the service connection, presents it to [Microsoft Entra ID](https://learn.microsoft.com/en-us/entra/identity/), and gets back a short-lived access token Terraform uses for the run. There is no static secret to manage or leak. Setup is mostly a portal exercise (the automatic flow is portal-only), but the provider side is code. When you create the Azure Resource Manager service connection, choose Workload identity federation (automatic) instead of manual service principal, and the federated credential configuration happens for you. In your Terraform provider block, tell the azurerm provider to use OIDC: provider "azurerm" { features {} use_oidc = true } With the DevLabs Terraform tasks, selecting the OIDC-backed service connection populates the environment variables Terraform needs, so you don't wire anything manually. If you run raw terraform commands in a script task instead, you map ARM_USE_AZUREAD=true and ARM_OIDC_TOKEN yourself; [the introduction to OIDC with Terraform](https://devblogs.microsoft.com/devops/introduction-to-azure-devops-workload-identity-federation-oidc-with-terraform/) walks through both paths. ## Detecting Drift with a Scheduled Pipeline Your pipeline prevents unauthorized changes from being deployed, but it can't stop someone from opening the portal and changing a resource directly. That's drift: the live environment diverges from what your code says it should be, and it's how "it works in prod but not in code" starts. Terraform exposes drift through the terraform plan command's -detailed-exitcode flag, which returns a machine-readable exit code instead of just printing output. A scheduled pipeline runs this check daily and fails loudly when drift exists: exit code 0 means no changes needed, 1 means the plan errored, and 2 means drift was detected. Add a schedule and run plan through a script task so you can capture that code: schedules: - cron: "0 6 * * *" displayName: "Daily drift check" branches: include: - main always: true terraform plan -detailed-exitcode -out=tfplan exitCode=$? echo "##vso[task.setvariable variable=terraformExitCode;isOutput=true]$exitCode" Then add a step that fails the pipeline when the exit code is 2, and wire that failure to a [Microsoft Teams](https://learn.microsoft.com/en-us/azure/devops/pipelines/integrations/microsoft-teams) or Slack webhook so drift shows up as a ticket instead of a silent surprise. Reconciliation is then a decision: if the manual change was wrong, run the normal apply pipeline to snap infrastructure back to code; if it was a legitimate hotfix, update the HCL to match reality and merge it as the new baseline. The [plan command documentation](https://developer.hashicorp.com/terraform/cli/commands/plan) covers the flag details. ## Troubleshooting Common Pipeline Failures When the pipeline turns red, the error message is usually one of three things. Knowing what each means saves an afternoon of trial and error. "Error acquiring the state lock": a previous run crashed mid-deploy and its blob lease never released. Check for a running pipeline; if none exists, break the lease: az storage blob lease break \ --account-name saterraformstate \ --container-name tfstate \ --blob-name prod/terraform.tfstate "The client … does not have authorization to perform action" on the storage account: the service connection's identity lacks the data-plane role. The resource-management role (Contributor) does not grant blob access; you need the separate [Storage Blob Data Contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/storage#storage-blob-data-contributor) assignment from earlier. Wait a minute for RBAC propagation after assigning, then re-run. "Error building ARM Config: obtain authorization token": the service principal secret has expired or was mistyped in the service connection, or someone rotated it out from under the pipeline. Regenerate the secret and update the connection, or treat this as your cue to migrate the connection to workload identity federation so the problem stops recurring. ## Wrapping Up You now have the full loop: a structured repository, remote state in Azure Storage with locking, a service connection that authenticates the pipeline, a validate-and-plan stage that publishes the reviewed plan as an artifact, and an apply stage that waits for human approval. Extend it per environment, remove the last long-lived secret with workload identity federation, and let the scheduled drift check tell you when the real world disagrees with your code. Start with one resource group and one environment. Get a plan artifact flowing through an approval gate, then expand to prod and add the hardening. That first green run, where a pull request rather than a portal session changes your infrastructure, is the point where running Terraform by hand stops and trusting the pipeline begins. Hate ads? Want to support the writer? Get many of our tutorials packaged as an ATA Guidebook. [Explore ATA Guidebooks](https://adamtheautomator.com/ata-guidebooks/) ## More from ATA Learning and Partners - ### Recommended Resources! Recommended Resources for Training, Information Security, Automation, and more! - ### Get Paid to Write! ATA Learning is always seeking instructors of all experience levels. Regardless if you’re a junior admin or system architect, you have something to share. Why not write on a platform with an existing audience and share your knowledge with the world? - ### ATA Learning Guidebooks ATA Learning is known for its high-quality written tutorials in the form of blog posts. Support ATA Learning with ATA Guidebook PDF eBooks available offline and with no ads! ## Categories - IT Ops - Cloud - DevOps - Home Ops - Information Security - Software Development ## Site - Home - Tutorials - Guidebooks - Instructors - Get Paid to Write - Advertising - Recommended Resources - About Adam Copyright 2026© ATA Learning | [Privacy Policy](https://adamtheautomator.com/privacy/) Don't be left behind with the ATA Learning Newsletter! Looks like you're offline!