--- title: "Fix Broken Azure DevOps Pipelines: A Systematic Guide" description: "Azure DevOps pipeline failures rarely point to the real problem. Learn systematic troubleshooting techniques to diagnose cryptic errors, network issues, and infrastructure limits." canonical: "https://adamtheautomator.com/azure-devops-pipeline-down-troubleshooting-guide/" --- # Fix Broken Azure DevOps Pipelines: A Systematic Guide > Azure DevOps pipeline failures rarely point to the real problem. Learn systematic troubleshooting techniques to diagnose cryptic errors, network issues, and infrastructure limits. Source: https://adamtheautomator.com/azure-devops-pipeline-down-troubleshooting-guide/ --- ATA Learning Tap to hide [ ATA Learning ](/) * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Search for: * [](https://twitter.com/adbertram) * [](https://github.com/Adam-the-Automator) * [](https://www.linkedin.com/company/adam-the-automator-llc) * [](/feed/) ![Fix Broken Azure DevOps Pipelines: A Systematic Guide](https://adamtheautomator.com/wp-content/uploads/2026/02/featured_image-11.webp) # Fix Broken Azure DevOps Pipelines: A Systematic Guide [![](https://secure.gravatar.com/avatar/d0b9d42e21e5622713f8b693aa5c0f9244d5f7dd200ed29b8398f52dee5de337?s=192&d=mm&r=g)Adam Bertram](https://adamtheautomator.com/author/adam-bertram/)16 February 20267 min. read Categories: [DevOps](/category/devops/) Tags:[Azure DevOps](/tag/azure-devops/)[DevOps](/tag/devops/) Table of Contents * [The Failure Isn’t Where the Error Appears](#the-failure-isnt-where-the-error-appears) * [Enable Verbose Logging](#enable-verbose-logging) * [Check Agent Logs Directly](#check-agent-logs-directly) * [The “No Hosted Parallelism” Block](#the-no-hosted-parallelism-block) * [Immediate Workaround](#immediate-workaround) * [Timeout Failures That Aren’t Negotiable](#timeout-failures-that-arent-negotiable) * [Your Options](#your-options) * [Network Failures You Can’t See](#network-failures-you-cant-see) * [Configure the Proxy Explicitly](#configure-the-proxy-explicitly) * [The SSL Certificate Problem](#the-ssl-certificate-problem) * [Git Checkout Failures That Block Everything](#git-checkout-failures-that-block-everything) * [Submodules Aren’t Checked Out Automatically](#submodules-arent-checked-out-automatically) * [Shallow Fetch Breaks Merge Operations](#shallow-fetch-breaks-merge-operations) * [NuGet Restore Fails With 401/403](#nuget-restore-fails-with-401403) * [Missing NuGet.config](#missing-nugetconfig) * [YAML Syntax Errors That Look Like Infrastructure Failures](#yaml-syntax-errors-that-look-like-infrastructure-failures) * [Path Length Limits on Windows](#path-length-limits-on-windows) * [Use Sysinternals for Process-Level Diagnosis](#use-sysinternals-for-process-level-diagnosis) * [ProcDump: Capture Crash Dumps](#procdump-capture-crash-dumps) * [ProcMon: Record File System Activity](#procmon-record-file-system-activity) * [The Problem Is Rarely Obvious](#the-problem-is-rarely-obvious) Your [Azure DevOps](https://learn.microsoft.com/en-us/azure/devops/) pipeline just failed. Again. Your deployment window closed an hour ago, stakeholders are waiting for an explanation, and the error message makes about as much sense as regex documentation. The timer’s running, your options are narrowing, and somewhere in Azure’s infrastructure, a resource you didn’t know existed is blocking progress you can’t measure. Pipeline failures don’t announce themselves with helpful diagnostic reports. They leave cryptic exit codes, vague timeout messages, and the occasional “something went wrong” summary that tells you nothing. You need a systematic approach to isolate the actual problem from the noise Azure DevOps generates. ## The Failure Isn’t Where the Error Appears Azure DevOps logs show you where the pipeline stopped—not why it stopped. A failed [NuGet](https://learn.microsoft.com/en-us/nuget/what-is-nuget) Restore task might point to a missing package, but the real issue could be an [expired service principal](https://learn.microsoft.com/azure/devops/pipelines/release/azure-rm-endpoint?view=azure-devops#service-principal-or-secret-expired) three layers deep in your [Azure subscription’s RBAC configuration](https://learn.microsoft.com/en-us/azure/role-based-access-control/overview). Start by enabling [verbose logging](https://learn.microsoft.com/en-us/azure/devops/pipelines/troubleshooting/troubleshooting) before you trust any error message. ### Enable Verbose Logging Queue your pipeline manually and check “Enable system diagnostics” before running it. For persistent verbose logging across all runs, define a pipeline variable: ``` variables: system.debug: true ``` This doesn’t just add more log lines. It activates the `Agent.Diagnostic` variable (on self-hosted agents v2.200.0+), which captures additional logs for troubleshooting network issues that standard logs ignore. * * * **_Pro Tip: Verbose logs expose API calls between the agent and Azure DevOps services. If a task hangs without error output, the diagnostic logs will show the last successful API call before the silence._** * * * ### Check Agent Logs Directly If the pipeline fails before producing useful logs, the problem lives at the agent level. Self-hosted agents store internal logs in the `_diag` folder at the agent’s root directory. Two log types matter: | Log Type | Purpose | | --- | --- | | **Agent Logs** | Registration with Azure DevOps, job polling, connectivity status | | **Worker Logs** | Execution details for each job step | Microsoft-hosted agents don’t grant access to these logs. If you’re hitting infrastructure-level failures repeatedly, [spin up a self-hosted agent](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents) where you control the diagnostic environment. ## The “No Hosted Parallelism” Block New Azure DevOps organizations hit this wall immediately: `##[error]No hosted parallelism has been purchased or granted`. Your pipeline won’t run. Not slowly, not partially—it won’t start. Microsoft disabled automatic free-tier parallelism grants for new projects to prevent [cryptomining abuse](https://devblogs.microsoft.com/devops/change-in-azure-pipelines-grant-for-public-projects/). You now request the grant manually via [this form](https://aka.ms/azpipelines-parallelism-request). Approval takes 2-3 business days. ### Immediate Workaround While waiting for Microsoft’s approval, configure a [self-hosted agent](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/windows-agent). Self-hosted agents bypass the parallelism grant requirement entirely. You can run them on a local VM, a cloud instance, or even a container. The setup process: ``` # Download the agent Invoke-WebRequest -Uri https://download.agent.dev.azure.com/agent/3.x/vsts-agent-win-x64-3.x.zip -OutFile agent.zip # Extract and configure Expand-Archive -Path agent.zip -DestinationPath agent cd agent .\config.cmd ``` You’ll need: * Your Azure DevOps organization URL * A [Personal Access Token (PAT) with Agent Pools (read, manage) scope](https://learn.microsoft.com/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops#create-a-pat) * The name of the agent pool (default: “Default”) Once configured, the agent registers with Azure DevOps and starts polling for jobs. ## Timeout Failures That Aren’t Negotiable Microsoft-hosted agents on the free tier enforce a [**60-minute timeout per job**](https://learn.microsoft.com/azure/devops/pipelines/agents/hosted?view=azure-devops#capabilities-and-limitations). There’s no grace period, no warning—the job terminates at 60:00. If you’re running test suites, publishing large artifacts, or deploying to multiple regions, you’ll hit this limit. Setting `timeoutInMinutes` higher than 60 in your YAML changes nothing: ```yaml jobs: - job: Deploy timeoutInMinutes: 120 # Ignored on free tier ``` ### Your Options | Solution | Cost | Timeout Limit | | --- | --- | --- | | Purchase a [Microsoft-hosted parallel job](https://azure.microsoft.com/en-us/pricing/details/devops/azure-devops-services/) | $40/month | 360 minutes (6 hours) per job | | [Self-hosted agent](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents) | Infrastructure cost only | Unlimited (while machine runs) | If your pipeline legitimately requires more than 60 minutes, you’re paying for parallelism or managing your own agents. There’s no third option. ## Network Failures You Can’t See Self-hosted agents behind corporate firewalls fail in ways that produce no useful error messages. The agent connects to Azure DevOps successfully, polls for jobs, starts the build—then a task fails with `ECONNREFUSED` or times out silently. The problem: Task-level network access doesn’t inherit the agent’s proxy configuration automatically. ### Configure the Proxy Explicitly During agent setup, specify your corporate proxy: ``` ./config.sh --proxyurl http://proxy.company.com:8080 --proxyusername proxyuser --proxypassword proxypass ``` This creates a `.proxy` file in the agent directory and exposes proxy settings via environment variables (`VSTS_HTTP_PROXY`, `VSTS_HTTP_PROXY_USERNAME`, `VSTS_HTTP_PROXY_PASSWORD`). But individual tasks—`npm install`, `git fetch`, `dotnet restore`—must be programmed to check those variables. Not all tasks do. * * * **_Reality Check: Your corporate proxy might work for the agent’s Azure DevOps communication but fail for NuGet feeds, npm registries, or Docker Hub. Each task’s network path is independent._** * * * ### The SSL Certificate Problem If your corporate network uses SSL inspection (a man-in-the-middle proxy that re-signs HTTPS traffic), [Node.js](https://nodejs.org/)\-based tasks will reject the proxy’s certificate: `Error: self signed certificate in certificate chain`. Node.js doesn’t use the Windows System Certificate Store. It maintains its own certificate validation and rejects certificates it doesn’t recognize—including your corporate root CA. **The fix:** 1. Export your corporate root CA certificate in Base64 (PEM) format 2. Set the `NODE_EXTRA_CA_CERTS` environment variable on the agent machine: ``` [System.Environment]::SetEnvironmentVariable('NODE_EXTRA_CA_CERTS', 'C:\certs\corporate-root-ca.pem', [System.EnvironmentVariableTarget]::Machine) ``` 1. Restart the agent service Tasks using Node.js will now trust your corporate certificate chain. ## Git Checkout Failures That Block Everything The `Checkout` task is your pipeline’s entry point. If it fails, nothing else runs. Exit code 128, “reference is not a tree,” or silent hangs—Git’s way of telling you absolutely nothing useful. ### Submodules Aren’t Checked Out Automatically If your repository contains [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules), [Azure Pipelines](https://learn.microsoft.com/en-us/azure/devops/pipelines/) won’t clone them unless you explicitly enable the setting: ```yaml steps: - checkout: self submodules: true ``` If the submodules are in private repositories, the pipeline’s automatically generated token might lack permission to access them. You’ll need to grant the build service account access to the submodule repositories or configure [HTTPS authentication with PATs](https://learn.microsoft.com/en-us/azure/devops/pipelines/repos/pipeline-options-for-git). * * * **_Warning: Shallow fetch improves performance but breaks operations that depend on Git history. If your pipeline calculates versions from tags or validates pull requests, you’ll need the full history._** * * * ### Shallow Fetch Breaks Merge Operations New pipelines created after September 2022 have shallow fetch (`fetchDepth: 1`) enabled by default to improve performance. This downloads only the most recent commit, not the full Git history. If your pipeline validates pull requests or calculates version numbers from Git tags, shallow fetch breaks those operations. The commits or tags your scripts reference don’t exist locally. Set `fetchDepth: 0` to clone the full history: ```yaml steps: - checkout: self fetchDepth: 0 ``` Performance cost: Larger repositories with deep history take longer to clone. You’re trading speed for completeness. ## NuGet Restore Fails With 401/403 Package restoration errors usually point to missing packages. In Azure DevOps, they’re more often permission problems. If you’re using [Azure Artifacts](https://learn.microsoft.com/en-us/azure/devops/artifacts/) as a private feed, the build service account needs explicit permission to access it. By default, the collection-scoped identity is used. For new classic pipelines, the job authorization scope is set to current project by default, which prevents the build agent from reaching feeds in other projects. **Resolution options:** * Disable “Limit job authorization scope” in Project Settings → Pipelines → Settings * Grant the “Project Collection Build Service” account Contributor access to the Artifact feed The first option is faster. The second is more secure if you actually want project-level isolation. ### Missing NuGet.config If you’re pulling packages from both public (nuget.org) and private (Azure Artifacts) sources, Azure Pipelines needs a `nuget.config` file to map package IDs to the correct feed. Without it, you’ll see `NU1101: Unable to find package` errors for private packages, even though they exist in your feed and the build service has permission. Create a `nuget.config` in your repository root: ``` ``` Commit it. The `NuGetCommand` task will use it automatically. ## YAML Syntax Errors That Look Like Infrastructure Failures YAML is indentation-sensitive. An extra space, a missing hyphen, or an incorrect list format produces errors that range from “pipeline not found” to silent failures where jobs never run. YAML: where whitespace has opinions. Before committing YAML changes, validate the syntax using the Azure DevOps REST API [Preview Runs endpoint](https://learn.microsoft.com/en-us/rest/api/azure/devops/pipelines/preview/preview?view=azure-devops-rest-7.1). You can call it via `az rest` or the Azure DevOps web editor’s “Validate” button: ```bash az rest --method post \ --uri "https://dev.azure.com/{org}/{project}/_apis/pipelines/{pipelineId}/preview?api-version=7.1-preview.1" \ --body '{"previewRun": true}' \ --resource "499b84ac-1321-427f-aa17-267ca6975798" ``` This catches schema violations before they block your deployment. ### Path Length Limits on Windows Windows enforces a 260-character path length limit by default. Deeply nested `node_modules` directories, multi-level artifact paths, or long branch names push past this limit during checkout or publish steps. The pipeline fails with `The specified path, file name, or both are too long` or file-not-found errors that make no sense. **Enable long path support:** ```powershell New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force ``` Requires a reboot. If you’re on Microsoft-hosted agents, you don’t control the OS configuration—restructure your artifact paths instead. ## Use Sysinternals for Process-Level Diagnosis The [Sysinternals Azure DevOps extension](https://marketplace.visualstudio.com/items?itemName=Sysinternals.SysinternalsADOExtension) integrates [ProcDump and ProcMon](https://learn.microsoft.com/en-us/sysinternals/) directly into pipeline tasks. This addresses scenarios where tests crash intermittently, builds consume excessive memory, or file locks block artifact publishing. Finally, a log that actually tells you what happened instead of what didn’t. ### ProcDump: Capture Crash Dumps ```yaml - task: Sysinternals.ProcDump@1 displayName: 'Capture Crash Dump with ProcDump' inputs: processName: 'dotnet.exe' dumpType: 'Full' delay: 15 artifactName: dotnet_dumps ``` When the target process triggers the configured threshold (delay, CPU, or memory), ProcDump generates a crash dump and uploads it as a pipeline artifact. You download it post-mortem and analyze it with [WinDbg](https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools) or [Visual Studio](https://learn.microsoft.com/en-us/visualstudio/debugger/using-dump-files). Crash dumps tell you why a process died. But sometimes you need to see what it was doing while it was still alive. ### ProcMon: Record File System Activity ```yaml - task: sysinternals.procmon@1 displayName: 'Procmon' inputs: logFile: procmonlog artifactName: procmon_logs ``` ProcMon logs every file, registry, and process operation. If a build fails with “file in use” or “access denied,” the ProcMon log shows exactly which process locked the file and when. ## The Problem Is Rarely Obvious Pipeline failures cascade. An expired service principal blocks artifact publishing, which triggers a timeout, which produces a vague error message that points to the wrong task entirely. You fix the symptom—the timeout—and the next run fails at a different step because the root cause (the service principal) remains broken. Work backward from the failure. Enable verbose logging. Check agent diagnostics. Verify network paths and certificate chains. Confirm permissions at every boundary—project scope, feed access, subscription RBAC, firewall allowlists. Azure DevOps doesn’t hand you the answer. It hands you log fragments, partial error messages, and infrastructure limits disguised as configuration problems. Systematic diagnosis—layer by layer, boundary by boundary—is the only method that scales. Share this article [Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fazure-devops-pipeline-down-troubleshooting-guide%2F&text=Fix%20Broken%20Azure%20DevOps%20Pipelines%3A%20A%20Systematic%20Guide)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fazure-devops-pipeline-down-troubleshooting-guide%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fazure-devops-pipeline-down-troubleshooting-guide%2F) ## Related Posts ![](https://adamtheautomator.com/wp-content/uploads/2026/08/featured_image-5.webp) ### [Azure DevOps Boards: Trace Every Commit to Deployment](/azure-devops-boards-traceability/) Configure Azure DevOps Boards process templates, backlogs, and Kanban WIP limits, then trace every work item from commit to deployment. ![](https://adamtheautomator.com/wp-content/uploads/2026/06/featured_image-23.png) ### [Guide to Transitioning from DevOps to MLOps Engineer](/guide-transitioning-devops-mlops-engineer/) As enterprise AI adoption accelerates, traditional DevOps engineers are uniquely positioned to pivot into high-paying MLOps roles. This guide breaks down the specific skills gap, maps existing CI/CD ![](https://adamtheautomator.com/wp-content/uploads/2026/04/featured_image-17.webp) ### [Migrating Azure DevOps to GitHub Enterprise: The ROI Case](/migrate-azure-devops-github-enterprise/) Quantify the ROI of migrating Azure DevOps to GitHub Enterprise: AI productivity gains, migration costs, and the hybrid Copilot strategy. ## Categories * [IT Ops](/category/it-ops/) * [Cloud](/category/cloud/) * [DevOps](/category/devops/) * [Home Ops](/category/home-ops/) * [Information Security](/category/infosec/) * [Software Development](/category/software-development/) ## Site * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)