--- title: "Learning the gsutil Command Through Examples" description: "Discover the many uses of the gsutil command and learn through examples how to utilize this command-line tool to make your management job easier!" canonical: "https://adamtheautomator.com/gsutil/" --- # Learning the gsutil Command Through Examples > Discover the many uses of the gsutil command and learn through examples how to utilize this command-line tool to make your management job easier! Source: https://adamtheautomator.com/gsutil/ --- ATA Learning Tap to hide [ ATA Learning ](/) * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Search for: * [](https://twitter.com/adbertram) * [](https://github.com/Adam-the-Automator) * [](https://www.linkedin.com/company/adam-the-automator-llc) * [](/feed/) ![Learning the gsutil Command Through Examples](https://adamtheautomator.com/wp-content/uploads/2022/12/gsutil.jpg) # Learning the gsutil Command Through Examples [![](https://secure.gravatar.com/avatar/2788bb1a3f735603f81eca51d68daec56a9d97e805a10268fb2c20afcc76b81b?s=192&d=mm&r=g)Nicholas Xuan Nguyen](https://adamtheautomator.com/author/nicholas-xuan-nguyen/)27 December 20227 min. read Categories: [DevOps](/category/devops/) Tags:[Command Line](/tag/command-line/)[Google Cloud](/tag/google-cloud/) Table of Contents * [Prerequisites](#prerequisites) * [Creating a Google Cloud Project](#creating-a-google-cloud-project) * [Creating a Bucket with gsutil](#creating-a-bucket-with-gsutil) * [Uploading an Object to a Bucket](#uploading-an-object-to-a-bucket) * [Rewriting an Object’s Metadata](#rewriting-an-objects-metadata) * [Setting a Custom Metadata to an Object](#setting-a-custom-metadata-to-an-object) * [Copying an Object to Another Bucket](#copying-an-object-to-another-bucket) * [Deleting Objects and Buckets](#deleting-objects-and-buckets) * [Conclusion](#conclusion) Accessing and managing your Cloud Storage buckets can be a complex endeavor. Fortunately, Google Cloud Platform (GCP) provides an incredible tool to make the task easier — gsutil. gsutil is a powerful command line utility that provides you fast access to hundreds of features that GCP’s Cloud Storage service offers. And in this tutorial, you will learn how to use gsutil through a series of examples demonstrating its many capabilities. Read on and confidently manage your Cloud Storage buckets with ease! ## Prerequisites This tutorial will be a hands-on demonstration. To follow along, be sure you have a Google Cloud Platform (GCP) account — a [free trial](https://cloud.google.com/free) will suffice. ## Creating a Google Cloud Project Google Cloud projects allow you to organize resources, including storage buckets. But before creating a project, you must ensure that [Cloud Shell](https://cloud.google.com/shell) is activated. [Cloud](https://adamtheautomator.com/aws-vs-azure-vs-google-cloud/) Shell is a sandboxed command line environment pre-installed with the gsutil command line tool. Related:[Comprehensive Discovery : AWS vs Azure vs Google Cloud](https://adamtheautomator.com/aws-vs-azure-vs-google-cloud/) To create a Google Cloud Project: 1\. Open your favorite web browser, and log in to the [Google Cloud Console](https://console.cloud.google.com/). 2\. Next, click the Activate Cloud Shell icon (top-right) to activate the Cloud Shell. You will see a pane at the bottom of your browser, as shown below, where you can run commands to interact with GCP services. ![Activating the Cloud Shell](https://adamtheautomator.com/wp-content/uploads/2022/12/image-296.png) Activating the Cloud Shell 3\. Once Cloud Shell is activated, run the following [`gcloud projects`](https://cloud.google.com/sdk/gcloud/reference/projects) command in the Cloud Shell to `create` your project. You can name your projects differently. but this tutorial’s choice is `gsutil-project-demo` Related:[How to Perform a gcloud CLI Install and Manage Google Cloud](https://adamtheautomator.com/gcloud-cli-install/) ```bash gcloud projects create gsutil-project-demo ``` 4\. Now, click **AUTHORIZE** when prompted to allow the required permissions. ![Authorizing the Cloud Shell ](https://adamtheautomator.com/wp-content/uploads/2022/12/image-297.png) Authorizing the Cloud Shell If successful, you will see a message like the one below. ![Creating a Google Cloud project](https://adamtheautomator.com/wp-content/uploads/2022/12/image-298.png) Creating a Google Cloud project 5\. Lastly, run the below [`gcloud config set`](https://cloud.google.com/sdk/gcloud/reference/config/set) command to set your newly created project (`gsutil-project-demo`) as the default project. As a result, all subsequent commands you issue in the Cloud Shell are executed against the default project. `gcloud config set project gsutil-project-demo` ```bash gcloud config set project gsutil-project-demo ``` ![Setting the newly-created project as the default project](https://adamtheautomator.com/wp-content/uploads/2022/12/image-299.png) Setting the newly-created project as the default project ## Creating a Bucket with gsutil Your default project is set, and you are ready to work with the gsutil tool. What is the most basic operation that gsutil lets you perform? Create a bucket in the cloud to store and access your data. The traditional storage options, like block storage and file storage, work fine when you have fewer data. But when your data starts to pile up, like Terabytes or Petabytes, you need a more robust and cost-effective storage option. Google Cloud Storage offers that option. Run the `gsutil` command below to make a bucket (`mb`), but ensure you replace `*gs://my-first-bucket54621*` bucket name with your own that is globally unique. > 💡 _You can create as many buckets as you need, depending on the data you wish to store. But note that the bucket’s name must be in lowercase and should not contain any spaces or special characters._ Notice below that the bucket’s name is in URL form (`*gs://my-first-bucket54621*`). This URL helps you locate and access the data stored in your bucket faster. Unlike other storage solutions, you can even use an endpoint to access the bucket’s content. ```bash gsutil mb gs://my-first-bucket54621 ``` As you can see below, the bucket has been successfully created. ![Creating a bucket](https://adamtheautomator.com/wp-content/uploads/2022/12/image-300.png) Creating a bucket But if the command fails, the reason might be the name of your bucket is already taken, and you will see an error message like the one below. ![Getting an error when the bucket already exists](https://adamtheautomator.com/wp-content/uploads/2022/12/image-301.png) Getting an error when the bucket already exists ## Uploading an Object to a Bucket You have created a bucket in the Google Cloud Storage with gsutil, but that bucket is not serving any purpose if it is empty. In this example, you will upload an object to your bucket using the gsutil tool. An object in Cloud Storage is like a file but contains much more extensive metadata. Each metadata is in key-value pairs that describe the object in more detail. To upload an object to your bucket, follow these steps: 1\. Run the `curl` command below to download an image of a cat from its URL and store it in Cloud Shell. Related:[CURL Linux Command : Learning By Example](https://adamtheautomator.com/curl-linux/) ```bash curl https://cloud.google.com/storage/images/kitten.png -o kitten.png ``` ![Downloading an image](https://adamtheautomator.com/wp-content/uploads/2022/12/image-302.png) Downloading an image 2\. Next, run the following command to list (`ls`) the image (`kitten.png`) to ensure the file exists. ```bash ls -la kitten.png ``` ![Ensuring the downloaded file exists](https://adamtheautomator.com/wp-content/uploads/2022/12/image-303.png) Ensuring the downloaded file exists 3\. Now, run the below `gsutil` command to upload the file `(kitten.png`) into your Cloud Storage bucket (`gs://my-first-bucket54621`). ```bash gsutil cp kitten.png gs://my-first-bucket54621 ``` ![Uploading an object to the bucket](https://adamtheautomator.com/wp-content/uploads/2022/12/image-304.png) Uploading an object to the bucket ## Rewriting an Object’s Metadata Google Cloud Storage automatically generates a system of metadata when you upload the object. These default metadata can be the uploaded date, content type, storage class, and so on, which Cloud Storage uses as part of its object management. In this example, you will rewrite existing metadata on the file you uploaded to your bucket. But first, you must view the metadata already set to the file. 1\. Run the `gsutil stat` command below to list the metadata already set to your image file. ```bash gsutil stat gs://my-first-bucket54621/kitten.png ``` As you can see below, many of the default metadata are set, like the **Creation time**, **Update time**, **Storage class**, and more. Pick one metadata you want to set or rewrite, but this tutorial’s choice is the **Storage class:STANDARD**. ![Viewing a file’s metadata](https://adamtheautomator.com/wp-content/uploads/2022/12/image-305.png) Viewing a file’s metadata 2\. Next, run the below `gsutil rewrite` command to set (`-s`) the storage class of the uploaded object (`kitten.png`) from STANDARD to `nearline`. ``` gsutil rewrite -s nearline gs://my-first-bucket54621/kitten.png ``` ![Rewriting the Storage class metadata value](https://adamtheautomator.com/wp-content/uploads/2022/12/image-306.png) Rewriting the Storage class metadata value 3\. Rerun the `gsutil stat` command below to verify the changes to your file’s (`*kitten.png*`) metadata. ```bash gsutil stat gs://my-first-bucket54621/kitten.png ``` Below, you can see that this time the **Storage class** is set to **NEARLINE** as expected. ![Verifying the metadata changes](https://adamtheautomator.com/wp-content/uploads/2022/12/image-307.png) Verifying the metadata changes ## Setting a Custom Metadata to an Object Rewriting an object’s metadata comes in handy for object management. But can you also set your own metadata instead of rewriting the existing ones? Yes! These custom metadata are anything that helps describe or identify your objects. Run the below `gsutil` command to set custom metadata (`setmeta`). Ensure you replace the following with your own: | KEY:VALUE | A metadata key and value pair for your object. This example uses animal\_type as the KEY and cat as the VALUE. | | --- | --- | | gs://OBJECT\_PATH | Your object’s path (URL) in your bucket (i.e., gs://my-first-bucket54621/kitten.png) | > 💡 _Do not start your key with `goog`, as it is reserved for Google Cloud Storage. Additionally, do not use [non-ASCII characters](https://terpconnect.umd.edu/~zben/Web/CharSet/htmlchars.html) in your metadata key and value. Why? You might use an HTTP endpoint to access your objects, and non-ASCII characters will be rejected, which leads to an error._ ```bash gsutil setmeta -h "x-goog-meta-KEY:VALUE" gs://OBJECT_PATH ``` ![Setting a custom metadata](https://adamtheautomator.com/wp-content/uploads/2022/12/image-308.png) Setting a custom metadata Now, run the following command to view your object’s `(kitten.png`) custom metadata. ```bash gsutil stat gs://my-first-bucket54621/kitten.png ``` If successful, you will see your newly-set custom metadata, as shown below. ![Viewing the custom metadata](https://adamtheautomator.com/wp-content/uploads/2022/12/image-309.png) Viewing the custom metadata ## Copying an Object to Another Bucket As an admin, making copies of objects for backup is crucial. But how exactly do you copy objects? Worry not. The gsutil `cp` command allows you to copy objects from one another to another quickly. To see the `cp` command in action, follow these steps: 1\. Create a second bucket as you did in the “Creating a Bucket with gsutil” section, and name the bucket as you like. But this tutorial uses `my-second-bucket1234` as the name for this second bucket. 2\. Next, run the below command to copy (`cp`) the object _`kitten.png`_ from your first bucket ( `my-first-bucket54621`) to your second bucket (`my-second-bucket1234`). ```bash gsutil cp gs://my-first-bucket54621/kitten.png gs://my-second-bucket1234/kitten.png ``` The object is copied to the second bucket, including its metadata. ![Copying an object to another bucket](https://adamtheautomator.com/wp-content/uploads/2022/12/image-310.png) Copying an object to another bucket 3\. Now, run the following command to verify you have successfully copied the object (`kitten.png`) to your second bucket (`my-second-bucket1234`). ```bash gsutil stat gs://my-second-bucket1234/kitten.png ``` If successful, you will see the object in your second bucket with all its metadata intact, including custom ones, as shown below. ![Verifying the object was successfully copied to another bucket](https://adamtheautomator.com/wp-content/uploads/2022/12/image-311.png) Verifying the object was successfully copied to another bucket ## Deleting Objects and Buckets Being charged for unnecessary storage consumption seem frustrating, especially when you set custom metadata. Why? Custom metadata is subject to [storage charges](https://cloud.google.com/storage/pricing#storage-notes) aside from the object. To avoid these charges, delete objects or buckets you no longer need with the following steps: 1\. Run the following gsutil command to remove (`rm`) or delete an object (`kitten.png`) from your first bucket (`my-first-bucket54621`). ```bash gsutil rm gs://my-first-bucket54621/kitten.png ``` You have deleted the _kitten.png_ object in your first bucket, but the same object still exists in your second bucket. ![Deleting objects from buckets](https://adamtheautomator.com/wp-content/uploads/2022/12/image-312.png) Deleting objects from buckets Next, run each command below to list the objects (`kitten.png`) in your first (`my-first-bucket54621`) and second (`my-second-bucket1234`) buckets. ```bash gsutil ls -l gs://my-first-bucket54621/kitten.png gsutil ls -l gs://my-second-bucket1234/kitten.png ``` The output below confirms that the _**kitten.png**_ object no longer exists in your first bucket but is still in your second bucket. ![Verifying an object has been deleted](https://adamtheautomator.com/wp-content/uploads/2022/12/image-313.png) Verifying an object has been deleted 3\. Run the below commands to delete your two buckets recursively. These commands delete all the objects in each bucket and their associated custom metadata. ```bash gsutil rm -r gs://my-first-bucket54621 gsutil rm -r gs://my-second-bucket1234 ``` Notice below that the first command immediately removed your first bucket, while the second command removed the object before removing your second bucket. ![Deleting buckets](https://adamtheautomator.com/wp-content/uploads/2022/12/image-314.png) Deleting buckets Finally, run the below [`gcloud projects`](https://cloud.google.com/sdk/gcloud/reference/projects) command to `delete` your Google Cloud project (`gsutil-project-demo`). This command deletes your project and all its associated resources. ```bash gcloud projects delete gsutil-project-demo ``` Input **Y** and press Enter when prompted to confirm deleting the project. > 💡 _After deleting a project, you will have a 30-day grace period if you ever change your mind and want to recover your project. Once the 30-day period has passed, the project and its resources are deleted and cannot be recovered._ ![Deleting a Google Cloud project](https://adamtheautomator.com/wp-content/uploads/2022/12/image-315.png) Deleting a Google Cloud project ## Conclusion As an alternative to the Google Cloud Console, gsutil lets you manage your Cloud Storage projects via Cloud Shell. And in this tutorial, you have learned how to use gsutil to create buckets, upload and copy objects, and delete unused ones to avoid recurring charges. At this point, you can easily manage your Google Cloud projects from the Cloud Shell. But this tutorial can only cover the tip of the iceberg regarding what gsutil can offer. With this newfound knowledge, why not use [gsutil with Boto](https://cloud.google.com/storage/docs/boto-gsutil) to automate your AWS cloud operations? Related:[Utilizing Boto3 to Manager AWS S3](https://adamtheautomator.com/boto3-s3/) Share this article [Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fgsutil%2F&text=Learning%20the%20gsutil%20Command%20Through%20Examples)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fgsutil%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fgsutil%2F) ## Related Posts ![](https://adamtheautomator.com/wp-content/uploads/2022/12/gcp-cloud-functions.jpg) ### [GCP Cloud Functions Customize Your Serverless Workflow](/gcp-cloud-functions/) Learn how to build and deploy scalable, serverless applications using GCP Cloud Functions. This tutorial covers essential concepts and best practices for streamlined workflows. ![](https://adamtheautomator.com/wp-content/uploads/2023/02/kubeadm.jpg) ### [How to Get Started Building Kubernetes Clusters with kubeadm](/kubeadm/) Learn how to create and manage Kubernetes clusters with the kubeadm command-line tool in this tutorial by ATA Learning! ![](https://adamtheautomator.com/wp-content/uploads/2023/01/google-cloud-storage-python.jpg) ### [How to Install & Use the Google Cloud Storage Python Client](/google-cloud-storage-python-client/) Discover how to use the Google Cloud Storage Python client in your projects and make your DevOps pipelines work even better! ## Categories * [IT Ops](/category/it-ops/) * [Cloud](/category/cloud/) * [DevOps](/category/devops/) * [Home Ops](/category/home-ops/) * [Information Security](/category/infosec/) * [Software Development](/category/software-development/) ## Site * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)