Do you need to download files from the web but hate repeatedly clicking links? If your job involves downloading files from the web regularly, you will probably want to automate the task. Why not use PowerShell to download files?
Windows PowerShell and PowerShell comes with file-download capabilities. Using PowerShell to download files is a matter of knowing which cmdlets and .NET classes to use and how to use them.
In this article, you’ll learn the various ways to use PowerShell to download files from the web.
Table of Contents
Since this is a learning-by-doing article, there are some prerequisites to ensure that you can follow the examples. Below are the basic requirements.
- A computer that is running on Windows 10 or higher. This computer is where you will run the scripts/commands featured in this article.
- Windows PowerShell 5.1 or PowerShell 7.1 (recommended).
- Windows 10 already includes Windows PowerShell 5.1.
- A web site that hosts the files to download.
Using PowerShell to Download Files from URLs: Four Ways
There are four methods to use PowerShell to download files that do not depend on third-party tools. These are:
- .NET WebClient Class.
Whichever one of these four methods you use, the logic and components to make them work are the same. There must be a source URL pointing to the file’s location and the destination path to save the downloaded files. If required by the webserver, you need to enter the credentials as well.
The next sections show each of these four methods. In the end, it’s up to you to decide which way you would adapt when using PowerShell to download files.
The first method in PowerShell to download files is by using the
Invoke-WebRequest cmdlet. Perhaps the most used cmdlet in this article,
Invoke-WebRequest, can download HTTP, HTTPS, and FTP links.
Whether the source location requires users to log in, the
Invoke-WebRequest cmdlet can handle requests with credentials as well.
To download a file, the syntax below shows the minimum parameters required to achieve the desired outcome.
Invoke-WebRequest -Uri <source> -OutFile <destination>
For example, the code below downloads a file with the name 10MB.zip from a website. Then it saves the downloaded file to C:\dload\10MB.zip. You may copy the code below and paste it into your PowerShell session to test.
# Source file location $source = 'http://speedtest.tele2.net/10MB.zip' # Destination to save the file $destination = 'c:\dload\10MB.zip' #Download the file Invoke-WebRequest -Uri $source -OutFile $destination
The demonstration below shows the expected result after running the code above in PowerShell. As you can see, the file download was successful.
How about if the source requires authentication before allowing access? For example, the code below downloads a file from a private website where users must log in.
$source = 'https://mirror.lzex.ml/100MB.zip' $destination = 'c:\dload\100MB.zip' Invoke-WebRequest -Uri $source -OutFile $destination
However, the download failed due to unauthorized access.
If authentication is required, you should add a credential to the request using the
-Credential parameter. The first line in the code below prompts you to enter the credential (username and password) and stores it to the
$credential = Get-Credential $source = 'https://mirror.lzex.ml/100MB.zip' $destination = 'c:\dload\100MB.zip' Invoke-WebRequest -Uri $source -OutFile $destination -Credential $credential
The demonstration below shows what you’d expect to see when you run the above code in PowerShell. As you can see, the
Get-Credential cmdlet prompted a PowerShell credential request. This time, using the credential with
Invoke-WebRequest resulted in a successful download.
Looking Out for Parsing Errors when using Invoke-WebRequest
A crucial thing to remember when using
Invoke-WebRequest in Windows PowerShell is that, by default, this cmdlet uses the Internet Explorer engine to parse data. The error below may happen when using
Invoke-WebRequest on computers without the Internet Explorer in it.
You’ll have to re-issue your command, but this time, include the
Invoke-WebRequest -Uri <source> -OutFile <destination> -UseBasicParsing
In Windows PowerShell, you may receive an error message: The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer’s first-launch configuration is not complete. Specify the UseBasicParsing parameter and try again.
Starting with PowerShell Core 6.0, the
Invoke-WebRequest cmdlet uses basic parsing only. As such, the
-UseBasicParsing parameter is no longer necessary.
Invoke-RestMethod cmdlet is more about sending an HTTP or HTTPS request to a RESTful web service. This cmdlet is more suited for requests that interact with REST APIs such as Microsoft Graph API.
When it comes to downloading files straight from the web,
Invoke-RestMethod is an excellent contender. Do not be deceived into thinking otherwise. There is not much difference between using
Invoke-WebRequest when used for downloading files from a direct web link.
Downloading a File using Invoke-RestMethod
To download a file using
Invoke-RestMethod, use the syntax below. You’ll notice that the command uses the same parameters as
Invoke-RestMethod -Uri <source> -OutFile <destination>
In the example code below, the file is downloaded from the URL value in the
$source variable. Then, saved to the path defined in the
$source = 'http://speedtest.tele2.net/10MB.zip' $destination = 'c:\dload\10MB.zip' Invoke-RestMethod -Uri $source -OutFile $destination
If the source requires authentication, you can pass the credentials using the
-Credential parameter. The example below prompts for the credentials and stores it to the
$credential variable. The value of the
$credential variable is then passed to the
Also, since the file link is an HTTP source and not HTTPS, it means that you are sending an unencrypted authentication. Typically, you should avoid using HTTP sources for security. But if you must use an HTTP source, you need to add the
-AllowUnencryptedAuthentication switch to your command.
$credential = Get-Credential $source = 'http://speedtest.tele2.net/10MB.zip' $destination = 'c:\dload\10MB.zip' Invoke-RestMethod -Uri $source -OutFile $destination -Credential $credential -AllowUnencryptedAuthentication
Start-BitsTransfer is designed specifically for transferring files between client and server computers. This PowerShell cmdlet is dependent on the Background Intelligent Transfer Service (BITS) that is native to the Windows operating system.
Start-BitsTransfer requires BITS to work means that this cmdlet is not available on non-Windows computers. On the flipside,
Start-BitsTransfer enjoys the benefits of BITS itself. Some of these benefits are:
- Network bandwidth and usage awareness.
- Interruption handling (resume, auto-resume, pause, etc.)
- Downloading multiple files as background jobs.
- Ability to set download job priorities.
Downloading a File
The fundamental way to use
Start-BitsTransfer in PowerShell to download a file is to specify a source and destination. Using the script below, you only need to change the
$destination values according to your requirements.
$source = 'http://speedtest.tele2.net/100MB.zip' $destination = 'c:\dload\100MB.zip' Start-BitsTransfer -Source $source -Destination $destination
As you can see from the demo below, the file is downloaded to the path c:\dload\100MB.zip.
Suppose the destination is not specified,
Start-BitsTransferdownloads and saves the file to the current working directory. For example, if you run
Start-BitsTransferfrom C:\dload, the file downloads to the same directory.
For downloads that require authentication,
Start-BitsTransfer has a
-Credential parameter that accepts a PSCredential object.
Downloading Multiple Files
To demonstrate downloading multiple files, you’ll need to create a CSV file with two columns. Name the file filelist.txt. The first column should contain the link to the source, while the second column must contain the destination path. The file contents would like the one below.
# source,destination http://speedtest.tele2.net/1MB.zip,c:\dload\1MB.zip http://speedtest.tele2.net/10MB.zip,c:\dload\10MB.zip http://speedtest.tele2.net/100MB.zip,c:\dload\100MB.zip
Once the CSV file is ready, use the command below to begin the file download. The command imports the CSV file using
Import-Csv and passes the contents to
Import-Csv .\filelist.csv | Start-BitsTransfer
Refer to the demo below to see how the code above works. As you can see, the download starts, and you see the download progress. The PowerShell prompt is not available during the download process.
Suppose you want to start the download process as a background job. To do so, you only have to add the
-Asynchronous switch at the end of the
Import-Csv .\filelist.csv | Start-BitsTransfer -Asynchronous
Initially, the state of each job would show connecting. The screenshot below shows each file download’s job id.
Now that you’ve started the download process, you’ll want to check whether the download has been completed. To check the download job status, use the
Get-BitsTransfer cmdlet. As you can see below, the download jobs’ status has changed to Transferred.
Using WebClient Class and HttpClient Class (.NET Framework)
If you want to know more about these two .NET class in more development and technical way, you could start with → When to use WebClient vs. HttpClient vs. HttpWebRequest. In the next section, you will learn how to use WebClient and HttpClient in PowerShell to download files from the web.
Downloading a File using System.Net.WebClient
To use the WebClient class, you need to initiate an object as a
System.Net.WebClient **type. In the example below, the
$webClient is the new
System.Net.WebClient object. Then, using the
DownloadFile() method starts the download of the file from the source.
Please copy the code below and run it in your PowerShell session to test. Note that you will not see any progress or output on the screen unless there’s an error. However, the PowerShell prompt will be locked until the download is complete.
# Define the source link and destination path $source = 'http://speedtest.tele2.net/10MB.zip' $destination = 'c:\dload\10MB.zip' # Create the new WebClient $webClient = [System.Net.WebClient]::new() # Download the file $webClient.DownloadFile($source, $destination)
If the source requires authentication to allow the file download, you can use the code below. The first line prompts for the credential and stores it to the
$credentials variable. The value of
$credential is then included in the file download request.
# Prompt for username and password $credentials = Get-Credential $source = 'http://speedtest.tele2.net/10MB.zip' $destination = 'c:\dload\10MB.zip' # Create the new WebClient $webClient = [System.Net.WebClient]::new() # Add the credential $webClient.Credentials = $credentials # Download the file $webClient.DownloadFile($source, $destination)
It appears that the WebClient class is obsolete, and the new class that Microsoft is endorsing is the HttpClient class. Don’t worry, though. The next section talks about using the HttpClient class in PowerShell to download files from the web.
Downloading a File using System.Net.Http.HttpClient
Like the WebClient class, you need to create first the
System.Net.Http.HttpClient. Using the code below downloads the file from the
$source to the
$destination. Refer to the comments above each line to know what each line of code does.
The code below is live, and you can test it by running it in your PowerShell session.
# Set the source and destination $source = 'http://speedtest.tele2.net/10MB.zip' $destination = 'c:\dload\10MB.zip' # Create the HTTP client download request $httpClient = New-Object System.Net.Http.HttpClient $response = $httpClient.GetAsync($source) $response.Wait() # Create a file stream to pointed to the output file destination $outputFileStream = [System.IO.FileStream]::new($destination, [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write) # Stream the download to the destination file stream $downloadTask = $response.Result.Content.CopyToAsync($outputFileStream) $downloadTask.Wait() # Close the file stream $outputFileStream.Close()
In situations where downloading a file requires authentication, you need to add the credential to the HttpClient object. To include a credential to the file download request, create a new
System.Net.Http.HttpClientHandler object to store the credentials.
You can copy the code below and run it in PowerShell to test. Or you can also run it as a PowerShell script. In this example, the code is saved as download-file.ps1.
# Set the source and destination $source = 'http://speedtest.tele2.net/10MB.zip' $destination = 'c:\dload\10MB.zip' # Prompt for credentials $credentials = Get-Credential # Create the HTTP client download request with credentials $handler = New-Object System.Net.Http.HttpClientHandler $handler.Credentials = $credentials $httpClient = New-Object System.Net.Http.HttpClient($handler) $response = $httpClient.GetAsync($source) $response.Wait() # Create a file stream to pointed to the output file destination $outputFileStream = [System.IO.FileStream]::new($destination, [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write) # Stream the download to the destination file stream $downloadTask = $response.Result.Content.CopyToAsync($outputFileStream) $downloadTask.Wait() # Close the file stream $outputFileStream.Close()
The demo below shows the result when running the PowerShell script to download the file.
At the start, the directory only has the script file in it. There’s a prompt to enter the username and password. Then, the script proceeds to download the file. After downloading the file, you can see that the new file is now inside the destination directory.
Windows PowerShell and PowerShell Core come with built-in capabilities to download files. Whether downloading password protected sources, single or multiple files – a PowerShell way is available to you.
The file download methods covered in this article works on both Windows PowerShell and PowerShell Core. This means that these methods apply to both Windows and Non-Windows systems, with the exclusion of
And since PowerShell is more than a command prompt, you can translate what you learned into scripts. For you, that would mean an opportunity for automation. No more copying URLs, clicking links and waiting for downloads manually.