---
title: "Utilizing Boto3 to Manager AWS S3"
description: "Learn how to get started from scratch on copying files with Python and AWS S3 using the Boto3 S3 Python module."
canonical: "https://adamtheautomator.com/boto3-s3/"
---

# Utilizing Boto3 to Manager AWS S3

> Learn how to get started from scratch on copying files with Python and AWS S3 using the Boto3 S3 Python module.

Source: https://adamtheautomator.com/boto3-s3/

---

ATA Learning

Tap to hide

[

ATA Learning

](/)

*   [Home](/)
*   [Tutorials](/tutorials/)
*   [Instructors](/author/)
*   [Advertising](/advertising/)
*   [Recommended Resources](/resources/)
*   [About Adam](/about-adam/)

Search for:  

*   [](https://twitter.com/adbertram)
*   [](https://github.com/Adam-the-Automator)
*   [](https://www.linkedin.com/company/adam-the-automator-llc)
*   [](/feed/)

![Utilizing Boto3 to Manager AWS S3](https://adamtheautomator.com/wp-content/uploads/2021/04/Getting-Started-Managing-AWS-S3-with-Python-Boto3.jpg)

# Utilizing Boto3 to Manager AWS S3

[![](https://secure.gravatar.com/avatar/2beb65fca997135120ed98dc6a2e57dcdf1a7d7d2f5ff687b5d91dc7ccd7a6b5?s=192&d=mm&r=g)Sagar](https://adamtheautomator.com/author/shanky-mendiratta/)16 June 20216 min. read

Categories: [Cloud](/category/cloud/)

Tags:[AWS](/tag/aws/)[Python](/tag/python/)

Table of Contents

*   [Prerequisites](#h-prerequisites)
*   [Installing Boto3](#h-installing-boto3)
*   [Creating an AWS S3 Bucket with Boto3](#h-creating-an-aws-s3-bucket-with-boto3)
*   [How to List S3 buckets in AWS](#h-how-to-list-s3-buckets-in-aws)
*   [How to Upload A File to an AWS S3 Bucket](#h-how-to-upload-a-file-to-an-aws-s3-bucket)
*   [How to Upload Entire Folders with Boto3](#h-how-to-upload-entire-folders-with-boto3)
*   [How to Copy Files Between S3 Buckets with Boto3](#h-how-to-copy-files-between-s3-buckets-with-boto3)
*   [Conclusion](#h-conclusion)

If you need to copy files to an [Amazon Web Services (AWS) S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) bucket, copy files from bucket to bucket, and automate the process, the AWS software development kit (SDK) for Python called [Boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) is your best friend. Combining Boto3 and S3 allows move files around with ease in AWS.

In this tutorial, you will learn how to get started using the Boto3 Python library with S3 via an example-driven approach.

Let’s get started!

## Prerequisites

This post will be a step-by-step tutorial. If you’d like to follow along, ensure you have the following in place:

*   An [AWS account](https://repost.aws/knowledge-center/create-and-activate-aws-account)
*   An IAM user with an [access key ID and secret key](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html) [set up on your local machine](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html). You can find steps to set up IAM users Access key and Secret key from [here](https://docs.amazonaws.cn/en_us/cli/latest/userguide/cli-configure-files.html). This tutorial will use an IAM user called _myboto3user_.

> _Ensure the IAM user is set up for [programmatic access](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html) and that you assign it to the existing policy of AmazonS3FullAccess._

*   [Python](https://adamtheautomator.com/install-python-36/) v3.6 or later installed on your local machine. This tutorial will be using Python v3.9.2 on a Windows 10 machine.

Related:[How Do You Install Python 3.6?](https://adamtheautomator.com/install-python-36/)

*   The Python package manager, [pip](https://pip.pypa.io/en/stable/installing/).
*   A code editor. Even though you can use any text editor to work with Python files. This tutorial will be using [Visual Studio (VS) Code.](https://code.visualstudio.com/)

## Installing Boto3

Before you can begin managing S3 with Boto3, you must install it first. Let’s start off this tutorial by downloading and installing Boto3 on your local computer.

The easiest ways to install Boto3 is to use the [pip Python package manager](https://pip.pypa.io/en/stable/user_guide/). To install Boto3 with pip:

1\. Open a cmd/Bash/PowerShell on your computer.

2\. Run the `pip install` command as shown below passing the name of the Python module (`boto3`) to install.

```bash
pip install boto3
```

> [pip](https://pip.pypa.io/en/stable/user_guide/) is a Python package manager which installs software that is not present in Pythons standard library.

![Installing Boto3](https://adamtheautomator.com/wp-content/uploads/2021/04/Untitled-72-2.png)

Installing Boto3

Boto3 should now successfully installed!

## Creating an AWS S3 Bucket with Boto3

Once you have Boto3 installed, it’s time to see what it can do! Let’s now dive into some examples of working with AWS S3 starting with creating a new S3 bucket.

> _Boto3 supports two types of interactions with AWS; [resource or client levels](https://www.edureka.co/community/39618/difference-between-client-and-resource-in-boto3). The client level provides low-level service access while the resource level provides higher-level, more abstracted level access. This tutorial will use client access._

1\. Open your favorite code editor.

2\. Copy and paste the following Python script into your code editor and save the file as _main.py_. The tutorial will save the file as _~\\main.py_. The following code snippet creates an S3 bucket called _first-us-east-1-bucket_ and prints out a message to the console once complete.

```python
# Importing boto3 library to make functionality available
 import boto3
# Creating a client connection with AWS S3
 s3 = boto3.client('s3')
# Creating a bucket
 s3.create_bucket(Bucket='first-us-east-1-bucket')
 print("Bucket created succesfully")
```

3\. Open your terminal and execute the _main.tf_ script using `python`. If successful, you should see a single message of `Bucket created successfully`.

```bash
python ~\main.py
```

4\. Now, open your favorite web browser, navigate to the [AWS Management Console](https://aws.amazon.com/console/) and log in.

5\. Click on the search bar at the top of the console, search for ‘S3’, and click on the S3 menu item.

![Searching for the S3 service in the AWS Management Console ](https://adamtheautomator.com/wp-content/uploads/2021/04/Untitled-73-1.png)

Searching for the S3 service in the AWS Management Console

On the S3 page, you should now see your newly-created bucket as shown below.

> _The bucket is in the **AWS Region** US East because the [default region is set to us-east-1 in the AWS profile](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html)._

![Finding the created S3 bucket in the AWS Management Console](https://adamtheautomator.com/wp-content/uploads/2021/04/Untitled-74-2.png)

Finding the created S3 bucket in the AWS Management Console

## How to List S3 buckets in AWS

Now that you have at least one S3 bucket in your account, now confirm that not by using the AWS Management Console but by using Boto3. Boto3 can also list all S3 buckets in your account.

With your code editor open:

1\. Copy and paste the following Python code into your code editor and save it as _list\_s3\_buckets.py_. This script queries AWS and saves the response to the `list_buckets()` method in a file, loops (`for`) through an array (`response['Buckets']`), and returns the name (`bucket["Name"]`) of each S3 bucket it finds.

```python
# Importing boto3 library
 import boto3
# Creating a client connection with AWS S3
 s3 = boto3.client('s3')
# Storing the client connection within rep
 response = s3.list_buckets()
 
# Output the bucket names
 print('Existing buckets:')
 for bucket in response['Buckets']: # For Loop to list all the buckets
     print(f'{bucket["Name"]}')
```

2\. Execute the script and you should see each S3 bucket name displayed in your account.

```bash
python list_s3_buckets.py
```

## How to Upload A File to an AWS S3 Bucket

Now that you have an S3 bucket to work with, let’s begin creating some objects in it. To start, upload a file to your S3 bucket.

1\. Create a new file or pick an existing one on your local computer to upload. This tutorial will use a file called _ATA.txt._

2\. Assuming you still have your code editor open, create a new Python script and save it as _upload\_s3\_file.py_ by copying/pasting the following code. The script below [opens the _~/ATA.txt_ file for reading (`rb`)](https://peps.python.org/pep-0343/) and upload the file (`upload_fileobjj()`) to the `first-us-east-1-bucket`.

```python
# Importing boto3 library
 import boto3
# Creating a client connection with AWS S3
 s3 = boto3.client('s3')
# Read the file stored on your local machine
 with open('~/ATA.txt', 'rb') as data:
# Upload the file ATA.txt within the Myfolder on S3
 s3.upload_fileobj(data, 'first-us-east-1-bucket', '~/ATA.txt')
```

3\. Execute the script which should upload the file.

```bash
python upload_s3_file.py
```

4\. Assuming you still have the S3 page open in a browser, click on the bucket created earlier, and you should see the file has been uploaded successfully!

![Finding the uploaded file](https://adamtheautomator.com/wp-content/uploads/2021/04/Untitled-75-2.png)

Finding the uploaded file

## How to Upload Entire Folders with Boto3

Previously you uploaded a single file to the AWS S3 bucket but what if you need to upload a folder? There is nothing in the `boto3` library itself that would allow you to upload an entire directory. But, you can still make it happen with a Python script.

1\. Ensure you have a folder on your local computer with some files in it. This tutorial will use a folder called _ATA._

2\. Assuming you still have your code editor open, create a new Python script and save it as _upload\_s3\_folder.py_ by copying/pasting the following code. The script below uses the `os module` to walk the directory tree using `os.walk` and recursively add all the files of the folder to a zip file called _ATA.zip_ using the `ZipFile` module and upload it to the `first-us-east-1-bucket`.

```python
# Importing boto3, zipfile, and os library
 import boto3
 import os
 import zipfile
# Creating a client connection with AWS S3
 s3 = boto3.client('s3')
 
def zipdir(path, ziph):
     # ziph is zipfile handle
     for root, dirs, files in os.walk(path):
         for file in files:
             ziph.write(os.path.join(root, file))
 zipf = zipfile.ZipFile('ATA.zip', 'w', zipfile.ZIP_DEFLATED) 
 zipdir('ATA', zipf)   # Passing the ATA folder in the arguments 
 zipf.close()
# Upload the Zip file ATA.zip within the folder2 on S3
 with open('ATA.zip', 'rb') as data:
     s3.upload_fileobj(data, 'first-us-east-1-bucket', 'ATA.zip')g
```

3\. Execute the script which should upload the zip file of the `ATA` folder containing all your files in the bucket. `python upload_s3_folder.py`

```bash
python upload_s3_folder.py
```

## How to Copy Files Between S3 Buckets with Boto3

Previously, you worked with S3 from on-prem. Let’s now stay in the cloud and copy files from one S3 bucket to another.

1\. Create a second S3 bucket to transfer files to using the know-how you received from the earlier section. This tutorial will be using a bucket called _first-us-east-1-bucket-2_ as the destination bucket.

2\. Create a new Python script and save it as copy\*\_s3\_to\_s3.py.\* Copy and paste in the following code. The script assumes you still have the _ATA.txt_ file in the S3 bucket uploaded earlier. It will find the _ATA.txt_ file in the `first-us-east-1-bucket` and copy it to the `first-us-east-1-bucket2` S3 bucket.

> The _Resource() API provides a higher-level abstraction than the raw, low-level calls made by service clients. At times, you need to connect to resources directly rather than Service API._

```python
# Importing boto3 library
 import boto3
# Creating the connection with the resource
 s3 = boto3.resource('s3')
# Declaring the source to be copied
 copy_source = {
     'Bucket': 'first-us-east-1-bucket',
     'Key': 'ATA.txt'
 }
 bucket = s3.Bucket('first-us-east-1-bucket-2')
# Copying the files to another bucket
 bucket.copy(copy_source, 'ATA.txt')
```

3\. Execute the script which should upload the file.

```bash
python copy_s3_to_s3.py
```

You should now see the file has been copied from one bucket to the other.

![Successful copy from bucket to bucket](https://adamtheautomator.com/wp-content/uploads/2021/04/Untitled-76-1.png)

Successful copy from bucket to bucket

## Conclusion

In this tutorial, you learned how to install the Boto3 AWS SDK for Python and work with the AWS S3 service. Although many automation tools manage and work with various AWS services, if you need to interact with AWS APIs with Python, Boto3 is the answer.

Now that you have a AWS S3 bucket set up with a Boto3, what do you plan to manage next?

Share this article

[Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fboto3-s3%2F&text=Utilizing%20Boto3%20to%20Manager%20AWS%20S3)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fboto3-s3%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fboto3-s3%2F)

## Related Posts

![](https://adamtheautomator.com/wp-content/uploads/2021/05/Getting-Started-Managing-AWS-EC2-with-Python-Boto3.jpg)

### [Take Advantage of Boto3 EC2 To Manage AWS EC2 instances](/boto3-ec2/)

Learn how to use the AWS Boto3 EC2 Python SDK to create new EC2 instances, start, stop, terminate and query tons of information about EC2.

![](https://adamtheautomator.com/wp-content/uploads/2026/08/featured_image-1.webp)

### [Survive the Senior Cloud Engineer AWS Interview Loop](/senior-cloud-engineer-interview-loop/)

Prepare for a senior cloud engineer interview with AWS EKS and Control Tower system design reasoning, Terraform locking, and salary negotiation tactics.

![](https://adamtheautomator.com/wp-content/uploads/2026/02/featured_image-1-scaled.jpg)

### [Transition from SysAdmin to Cloud Engineer](/transition-sysadmin-cloud-engineer/)

System administrators already have the networking, Linux, and operational expertise that cloud engineering demands. This guide maps the philosophy shift, technical domains, certifications, and portfolio steps to make the transition.

## Categories

*   [IT Ops](/category/it-ops/)
*   [Cloud](/category/cloud/)
*   [DevOps](/category/devops/)
*   [Home Ops](/category/home-ops/)
*   [Information Security](/category/infosec/)
*   [Software Development](/category/software-development/)

## Site

*   [Home](/)
*   [Tutorials](/tutorials/)
*   [Instructors](/author/)
*   [Advertising](/advertising/)
*   [Recommended Resources](/resources/)
*   [About Adam](/about-adam/)

Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)