--- title: "How to Use Azure Data Lake for Storage and Analysis" description: "Storing, managing, and analyzing big data is a challenge but learn how to make it easier with this Azure Data Lake tutorial!" canonical: "https://adamtheautomator.com/azure-data-lake/" --- # How to Use Azure Data Lake for Storage and Analysis > Storing, managing, and analyzing big data is a challenge but learn how to make it easier with this Azure Data Lake tutorial! Source: https://adamtheautomator.com/azure-data-lake/ --- ATA Learning Tap to hide [ ATA Learning ](/) * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Search for: * [](https://twitter.com/adbertram) * [](https://github.com/Adam-the-Automator) * [](https://www.linkedin.com/company/adam-the-automator-llc) * [](/feed/) ![How to Use Azure Data Lake for Storage and Analysis](https://adamtheautomator.com/wp-content/uploads/2022/06/How-to-Use-Azure-Data-Lake-for-Storage-and-Analysis.jpg) # How to Use Azure Data Lake for Storage and Analysis [![](https://secure.gravatar.com/avatar/2788bb1a3f735603f81eca51d68daec56a9d97e805a10268fb2c20afcc76b81b?s=192&d=mm&r=g)Nicholas Xuan Nguyen](https://adamtheautomator.com/author/nicholas-xuan-nguyen/)1 July 20228 min. read Categories: [IT Ops](/category/it-ops/) Tags:[Azure](/tag/azure/) Table of Contents * [Prerequisites](#prerequisites) * [Creating a Data Lake Through Azure Portal](#creating-a-data-lake-through-azure-portal) * [Creating a Data Lake Using the CLI](#creating-a-data-lake-using-the-cli) * [Uploading Data to the Data Lake Storage](#uploading-data-to-the-data-lake-storage) * [Conclusion](#conclusion) You’ll likely need to store data somewhere as an administrator or developer, not just any data, but massive data. Where do you turn? The cloud is your most practical option, and luckily, Azure Data Lake can help for worry-less data storage. Azure Data Lake lets you perform all types of processing and analytics across platforms and languages at blazing speed. And in this tutorial, you’ll learn how to use Azure Data Lake for storage and analysis. Efficiently and securely store your data in one place with Azure Data Lake today! ## Prerequisites This tutorial will be a hands-on demonstration. If you’d like to follow along, be sure you have an Azure account with an active subscription — If you don’t have one yet, a [free trial](https://azure.microsoft.com/en-us/free/) will suffice. Related:[Get Microsoft Azure Free: Your Goto Guide](https://adamtheautomator.com/azure-free/) ## **Creating a Data Lake Through Azure Portal** Azure Data Lake is a cloud-based data storage service optimized for big data analytics and is highly scalable. You can start small and grow as your needs increase. But how do you create the storage? Take a quick tour of the Azure Portal and create an Azure Data Lake account. 1\. Open your favorite web browser, and navigate to the [Azure Portal](https://portal.azure.com/). 2\. Next, provide your credentials, click on the Sign In button, and sign in to your Azure account. After signing in, your browser redirects to the Azure Portal (step three). ![Logging in to Azure Portal](https://adamtheautomator.com/wp-content/uploads/2022/06/image-380.png) Logging in to Azure Portal 3\. On your Azure Portal, click on Create a resource, which opens up the list of Azure resources available. > _You’ll see the page below when you first log in or don’t have any resources deployed in your subscription._ ![Creating a Resource](https://adamtheautomator.com/wp-content/uploads/2022/06/image-381.png) Creating a Resource 4\. Search for ‘storage account’ in the search bar at the top of the page and select Storage account. Doing so redirects your browser to the Storage account resource’s overview page. > _The search bar lets you quickly find resources you like to create instead of scrolling through the featured ones. But for this tutorial, you’re creating an Azure Data Lake account._ Azure Data Lake is built on top of Azure Storage. So, a storage account is the resource type you need to create a new Azure Data Lake account. ![Searching for Storage Account Resource](https://adamtheautomator.com/wp-content/uploads/2022/06/image-382.png) Searching for Storage Account Resource 5\. Now, click on Create, which redirects your browser to the Create a storage account page (step six), where you’ll configure your storage account. ![Initializing Creating a Storage Account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-383.png) Initializing Creating a Storage Account 6\. Configure your storage account starting with the **Project details** as follows: * Select your **Subscription** – If you have multiple subscriptions, ensure you select one where you prefer to create your storage account. This tutorial uses **Azure subscription 1**, as shown below. * Select your **Resource group** – Resource groups are a way to logically group Azure resources. You can think of resource groups as folders where you place related resources. Resource groups let you manage, monitor, and delete resources easier. > _If you don’t have a resource group yet, click on the [Create new](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal) hyperlink instead to create one._ ![Setting Project Details](https://adamtheautomator.com/wp-content/uploads/2022/06/image-384.png) Setting Project Details Related:[How to Rename/Move Azure Resource Groups (GUI and CLI)](https://adamtheautomator.com/rename-azure-resource-group/) 7\. On the same page, configure the instance details with the following: * Provide a unique **Storage account name** – This tutorial’s choice is **ataazurestorage**. The name must be unique within Azure and must be between three and 24 characters long. * Select the **Region** where you want to deploy your storage account – The region is where your storage account will physically reside. Select the region closest to you or your users. For example, if you’re creating a storage account for a web application that users from the US will access, select US East or US West regions. * Keep all other settings on default values and click on Next: Advanced at the bottom of the page. At this point, you’ll have a standard blob storage account. Related:[How to Use Blob Storage via Azure File Storage](https://adamtheautomator.com/blob-storage/) ![Create a Storage Account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-385.png) Create a Storage Account 8\. Under the Advanced tab, tick the Enable hierarchical namespace checkbox in Data Lake Storage Gen2. This option converts your blob storage account to a Data Lake account and enables all functionalities of a Data Lake, including Analytics and Store. Click on the Review + create button (bottom-left) to validate your settings, which may take a few minutes to complete. ![Click on the Review + create button at the bottom to validate your settings.](https://adamtheautomator.com/wp-content/uploads/2022/06/image-386.png) Click on the Review + create button at the bottom to validate your settings. 9\. After validation, click on the Create button to finalize creating the storage account. ![Creating the Storage Account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-387.png) Creating the Storage Account After your storage account is created, you’ll see the deployment in progress, as shown below, which may take a few minutes to complete. ![Viewing Deployment in Progress](https://adamtheautomator.com/wp-content/uploads/2022/06/image-388.png) Viewing Deployment in Progress 10\. Lastly, click on the Go to resource button to open your newly-created storage account after deployment. At this point, you already have an Azure Data Lake account. ![Accessing the Newly-created Storage Account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-389.png) Accessing the Newly-created Storage Account ![Viewing the New Azure Data Lake Account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-390.png) Viewing the New Azure Data Lake Account ## **Creating a Data Lake Using the CLI** You’ve seen that creating an Azure Data Lake account using the Azure Portal works fine. But what If you like to have a repeatable and automated way of creating Data Lake accounts? Azure Portal is not the best option, but Azure CLI. Related:[Getting Started with the Azure CLI](https://adamtheautomator.com/azure-cli/) The Azure CLI is a cross-platform tool that you can use to manage your Azure resources and lets you integrate with your automated CI/CI process. > _Azure CLI is available for Windows, Linux, Azure Cloud Shell, and macOS._ Related:[How to Install the Azure CLI (Windows, Linux, macOS, Azure Shell)](https://adamtheautomator.com/install-azure-cli/) To create an Azure Data Lake account via the Azure CLI: 1\. On the Azure Portal, click on the Cloud Shell button, as shown below, to open the Azure Cloud Shell. ![](https://adamtheautomator.com/wp-content/uploads/2022/06/image-391.png) Opening the Azure Cloud Shell Related:[Getting Started with the Azure Cloud Shell](https://adamtheautomator.com/azure-cloud-shell/) 2\. At the bottom of the Azure Portal, choose either Bash or PowerShell as your shell type, and the shell opens. ![Azure Portal](https://adamtheautomator.com/wp-content/uploads/2022/06/image-392.png) Azure Portal On your Azure Cloud Shell, you can change your shell type at will to either Bash or PowerShell. But for this tutorial, keep the Bash shell active. ![Viewing Azure Cloud Shell](https://adamtheautomator.com/wp-content/uploads/2022/06/image-393.png) Viewing Azure Cloud Shell 3\. Run the below command on your Azure Cloud Shell to verify the –version of Azure CLI installed. Whichever platform you’re using, ensure you have [Azure CLI version 2.6.0 or later](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli), or else you can’t create a Data Lake account. ```bash az --version ``` ![Verifying Azure CLI Version Installed](https://adamtheautomator.com/wp-content/uploads/2022/06/image-394.png) Verifying Azure CLI Version Installed 4\. Now, run the command below to login to Azure with your Azure account. This tutorial uses Azure Active Directory (Azure AD) authorization method. This method is the recommended authorization type as it’s easier and provides more security than using a service principal. ```bash az login ``` You’ll see a code and a URL appear in the terminal window below. Note down the URL as you’ll need it to authenticate Azure using Azure AD in the following step. ![Log in to Azure using your Azure account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-395.png) Log in to Azure using your Azure account 5\. Authenticate using Azure AD with the following: * Navigate to the URL you noted in step four in your browser. * Log in using your Azure account credentials and the code you noted in step four. ![Authenticating Azure using Azure AD](https://adamtheautomator.com/wp-content/uploads/2022/06/image-396.png) Authenticating Azure using Azure AD 6\. Next, click on Continue to complete the authentication process. ![Logging in to Azure](https://adamtheautomator.com/wp-content/uploads/2022/06/image-397.png) Logging in to Azure 7\. Run the below [az account list](https://docs.microsoft.com/en-us/cli/azure/account?view=azure-cli-latest#az-account-list) command to list the subscriptions for the logged-in account. > _If your account is associated with more than one Azure subscription, you might need to select and set the subscription that you want to use for your Data Lake account._ Note the name of the subscription to use for your Azure Data Lake account. For this tutorial, the subscription to use is Azure subscription 1. ```bash az account list ``` ![Getting the Subscription Name to Use for the Azure Data Lake Account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-398.png) Getting the Subscription Name to Use for the Azure Data Lake Account 8\. Now, run the following [az account set](https://docs.microsoft.com/en-us/cli/azure/account?view=azure-cli-latest#az-account-set) command and specify the name of your subscription. This command doesn’t provide an output but sets the subscription to use for your Azure Data Lake account. ```bash az account set --subscription 'Azure subscription 1' ``` 9\. Run the following [az group create](https://docs.microsoft.com/en-us/cli/azure/group?view=azure-cli-latest#az-group-create) command to create a resource group. Choose a unique name for your resource group, but this tutorial’s choice is ataadatalakecli and a –location set to westus. ```bash az group create --location westus --resource-group ataadatalakecli ``` ![Creating a Resource Group](https://adamtheautomator.com/wp-content/uploads/2022/06/image-399.png) Creating a Resource Group 10\. After creating a group resource, run the [az storage account create](https://docs.microsoft.com/en-us/cli/azure/storage/account?view=azure-cli-latest#az-storage-account-create) command below and pass in the values for the following parameters to create a storage account: * `--name` – Your Data Lake account name (`ataaazuredatalakecli`). * `--resource-group` – Your resource group name (`ataadatalakecli`). * `--location` – Your Data Lake account’s location (`westus`). * `--sku` – The storage SKU for your Data Lake account (`Standard_LRS`). * `--kind` – The type of Data Lake account to create (`StorageV2`). * `--enable-hierarchical-namespace true - Enables the hierarchical namespace for your account, which is required to use Data Lake Storage Gen2.` ```bash az storage account create --name ataaazuredatalakecli --resource-group ataadatalakecli --location westus --sku Standard_LRS --kind StorageV2 --enable-hierarchical-namespace true ``` > _Note that StorageV1 has now been deprecated, and you should use StorageV2 for all newly created Data Lake accounts. Existing StorageV1 accounts can still be used for some time but will eventually be migrated to StorageV2. So, migrating your data to StorageV2 accounts is strongly recommended._ ![Creating a Storage Account](https://adamtheautomator.com/wp-content/uploads/2022/06/image-400.png) Creating a Storage Account 11\. Now, [navigate to your resources group](https://portal.azure.com/#blade/HubsExtension/BrowseResourceGroups) in the Azure Portal, and you’ll see your newly-created resources, as shown below. Click on the hyperlink of your resources group to navigate to the resource group’s overview page (step 12). ![Viewing the Resource Groups](https://adamtheautomator.com/wp-content/uploads/2022/06/image-401.png) Viewing the Resource Groups 12\. Finally, click on your storage account from the list to access its overview page. ![Accessing Storage Account Info](https://adamtheautomator.com/wp-content/uploads/2022/06/image-402.png) Accessing Storage Account Info That’s it! You now have an active storage account. ![Viewing the Storage Account’s Overview](https://adamtheautomator.com/wp-content/uploads/2022/06/image-403.png) Viewing the Storage Account’s Overview ## Uploading Data to the Data Lake Storage You’ve just created your Lake Storage Gen2 account, but it’s currently empty. So why not upload your data? You can upload and verify your data using the Azure Portal and Azure CLI, but first, you must create a container. Related:[How to Copy Files to Azure Blob Storage with PowerShell](https://adamtheautomator.com/copy-files-to-azure-blob-storage/) 1\. On your storage account’s dashboard, click on Container under Data Storage (left panel), and click on Container, as shown below, to create a new container. In Azure, a container is a file system for storing your data. ![Creating a Container](https://adamtheautomator.com/wp-content/uploads/2022/06/image-404.png) Creating a Container 2\. Next, configure the new container with the following: * Specify a name for your container, but this tutorial’s choice is **azuredatalakecotainer**. * Select Private (no anonymous access) for the [access level](https://docs.microsoft.com/en-us/azure/storage/blobs/anonymous-read-access-configure?tabs=portal). * Click on **Create** at the bottom to create the container. ![Creating a Container](https://adamtheautomator.com/wp-content/uploads/2022/06/image-405.png) Creating a Container 3\. Click on your container’s name from the list, as shown below, to open it. Note that your container is currently empty. ![Opening the Container](https://adamtheautomator.com/wp-content/uploads/2022/06/image-406.png) Opening the Container 4\. Now, click on the **Upload** button at the top to upload files or folders to your container. ![Initializing Uploading Files or Folders](https://adamtheautomator.com/wp-content/uploads/2022/06/image-407.png) Initializing Uploading Files or Folders 5\. In the **Upload blob** blade, click on the folder upload button, locate your files or folders, and click on **Upload** to upload them. > _You can select multiple files and folders to upload in one go._ ![Locating Files to Upload](https://adamtheautomator.com/wp-content/uploads/2022/06/image-408.png) Locating Files to Upload You’ll see the status of each file/folder upload like the one below. ![Viewing Upload Progress](https://adamtheautomator.com/wp-content/uploads/2022/06/image-409.png) Viewing Upload Progress Once the upload completes, you’ll see the files listed in your container. ![Verifying Uploaded Files in Azure Portal](https://adamtheautomator.com/wp-content/uploads/2022/06/image-410.png) Verifying Uploaded Files in Azure Portal Alternatively, run the [az storage](https://docs.microsoft.com/en-us/cli/azure/storage/fs/file?view=azure-cli-latest#az-storage-fs-file-list) command below to list all the uploaded files in your container. Replace the container name (azuredatalakecotainer) and the account name (ataaazuredatalakecli ) with your own. ```bash az storage fs file list -f azuredatalakecotainer --account-name ataaazuredatalakecli --auth-mode login ``` The output below verifies that the files have been uploaded successfully to your Azure Data Lake Storage Gen2 account and their metadata. ![Listing Uploaded Files in Container via Azure CLI](https://adamtheautomator.com/wp-content/uploads/2022/06/image-411.png) Listing Uploaded Files in Container via Azure CLI 6\. Run the below command to create a new directory named my-data-lake-directory on your container (azuredatalakecotainer). ```bash az storage fs directory create -n my-data-lake-directory -f azuredatalakecotainer --account-name ataaazuredatalakecli --auth-mode login ``` ![](https://s3.us-west-2.amazonaws.com/secure.notion-static.com/24b99301-e246-4aae-badc-66e31945d4ad/new-directory.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIAT73L2G45EIPT3X45%2F20220614%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220614T004506Z&X-Amz-Expires=86400&X-Amz-Signature=ef79830d266b608449432a9249024ac1404630988713aaab675203592d8719a8&X-Amz-SignedHeaders=host&response-content-disposition=filename%20%3D%22new-directory.png%22&x-id=GetObject) 7\. Finally, navigate back to your container in the Azure Portal, and you’ll see the newly-created directory in the list, as shown below. From there, you can upload more new files and folders to the newly-created directory following steps three to five. ![Verifying Newly-created Directory](https://adamtheautomator.com/wp-content/uploads/2022/06/image-412.png) Verifying Newly-created Directory ## Conclusion Apart from being cost-effective as you’ll only pay for storage that you use, Azure Data uses Azure Active Directory for authentication and authorization. Securing your data is a top priority anyway. And in this tutorial, you’ve learned how to create an Azure Data Lake Storage Gen2 account using the Azure Portal and the CLI. With Azure Data Lake, you get to upload files and verify them, without making complicated processing of big data analytics workloads. At this point, you can now securely store all your data in one place, and begin to analyze your data using the tools and services that Azure offers. Why not begin the [Data Lake Analytics](https://azure.microsoft.com/en-us/services/data-lake-analytics/) service and start querying and visualizing your data? Share this article [Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fazure-data-lake%2F&text=How%20to%20Use%20Azure%20Data%20Lake%20for%20Storage%20and%20Analysis)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fazure-data-lake%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fazure-data-lake%2F) ## Related Posts ![](https://adamtheautomator.com/wp-content/uploads/2026/06/featured_image-21.png) ### [Automate SOC 2 Compliance with PowerShell](/automate-soc-2-compliance-powershell/) A comprehensive walkthrough on using PowerShell and Azure Policy as Code to automate evidence collection and enforce SOC 2 compliance controls across enterprise cloud environments. ![](https://adamtheautomator.com/wp-content/uploads/2026/06/ditch-gpos-intune-featured.webp) ### [Ditch the GPOs: Migrate to Microsoft Intune](/ditch-gpos-migrate-microsoft-intune-2/) Use Group Policy Analytics to migrate GPOs to Intune Settings Catalog profiles, handle unsupported Group Policy Preferences, and manage hybrid AD-to-MDM transitions. ![](https://adamtheautomator.com/wp-content/uploads/2023/12/Privileged-Identity-Management.jpg) ### [Using Azure AD Privileged Identity Management (JIT)](/privileged-identity-management/) Master secure access with the Azure AD Privileged Identity Management (JIT) for heightened control! Seamlessly elevate security today! ## Categories * [IT Ops](/category/it-ops/) * [Cloud](/category/cloud/) * [DevOps](/category/devops/) * [Home Ops](/category/home-ops/) * [Information Security](/category/infosec/) * [Software Development](/category/software-development/) ## Site * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)