---
title: "AWS Batch: Your Answer to Efficient Job Scheduling & Execution"
description: "Seeking a scalable, streamlined solution for batch computing? Explore how AWS Batch automates job management, saves you time, and reduces operational complexity."
canonical: "https://adamtheautomator.com/aws-batch/"
---

# AWS Batch: Your Answer to Efficient Job Scheduling & Execution

> Seeking a scalable, streamlined solution for batch computing? Explore how AWS Batch automates job management, saves you time, and reduces operational complexity.

Source: https://adamtheautomator.com/aws-batch/

---

ATA Learning

Tap to hide

[

ATA Learning

](/)

*   [Home](/)
*   [Tutorials](/tutorials/)
*   [Instructors](/author/)
*   [Advertising](/advertising/)
*   [Recommended Resources](/resources/)
*   [About Adam](/about-adam/)

Search for:  

*   [](https://twitter.com/adbertram)
*   [](https://github.com/Adam-the-Automator)
*   [](https://www.linkedin.com/company/adam-the-automator-llc)
*   [](/feed/)

![AWS Batch: Your Answer to Efficient Job Scheduling & Execution](https://adamtheautomator.com/wp-content/uploads/2022/06/Effective-Job-Management-with-AWS-Batch.jpg)

# AWS Batch: Your Answer to Efficient Job Scheduling & Execution

[![](https://secure.gravatar.com/avatar/2beb65fca997135120ed98dc6a2e57dcdf1a7d7d2f5ff687b5d91dc7ccd7a6b5?s=192&d=mm&r=g)Sagar](https://adamtheautomator.com/author/shanky-mendiratta/)27 February 20247 min. read

Categories: [DevOps](/category/devops/)

Tags:[AWS](/tag/aws/)

Table of Contents

*   [Prerequisites](#prerequisites)
*   [Setting up the Compute Environment](#setting-up-the-compute-environment)
*   [Creating an AWS Batch Job Queue](#creating-an-aws-batch-job-queue)
*   [Creating an AWS Batch Job Definition](#creating-an-aws-batch-job-definition)
*   [Verifying the AWS Batch Job and Monitoring the Job](#verifying-the-aws-batch-job-and-monitoring-the-job)
*   [Conclusion](#conclusion)

Are you struggling to manage and configure many computing resources for your Amazon Web Services (AWS)’s infrastructure? Why not try [AWS Batch](https://docs.aws.amazon.com/batch/latest/userguide/what-is-batch.html)? AWS Batch allows you to run batch computing workloads in the AWS Cloud with no sweat.

With AWS Batch, you can submit batch jobs to provision capacity according to the requirements and reduce the compute cost and capacity issues. And in this tutorial, you’ll learn how to run an AWS Batch job and deploy it to the AWS cloud.

Read on and start automating jobs with AWS Batch!

## **Prerequisites**

This tutorial will be a step-by-step demonstration. If you’d like to follow along, ensure you have the following in place:

*   An [Amazon Web Service (AWS) account](https://repost.aws/knowledge-center/create-and-activate-aws-account).
    
*   An AWS IAM user, an [access key ID, and a secret key](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html) [set up on your local machine](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) with access to create and manage EC2 instances – This tutorial will use an IAM user called ec2_user_.
    

> _Ensure the IAM user is set up for [programmatic access](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html) and that you assign it to the existing policy of AmazonEC2FullAccess._

Related:[Learning Identity and Access Management (IAM) AWS Through Examples](https://adamtheautomator.com/iam-aws/)

*   Your compute environments and container instances will require an AWS [IAM role with administrator access](https://docs.aws.amazon.com/batch/latest/userguide/get-set-up-for-aws-batch.html#create-an-iam-role).
    
*   A private AWS ECR repository.
    

Related:[Running Scans on AWS ECR Repository](https://adamtheautomator.com/aws-inspector/#Running_Scans_on_AWS_ECR_Repository)

*   An AWS VPC.

Related:[Building an AWS Terraform VPC Step-by-Step](https://adamtheautomator.com/terraform-vpc/)

*   [](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)[AWS CLI configured](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html).

## Setting up the Compute Environment

AWS Batch runs jobs based on various networking components. These networking components are but are not limited to:

*   Resource utilization, such as CPU and memory
*   Infrastructure computes services, such as AWS Fargate, Amazon EC2, etc.

But before running your first job, setting up your compute environment is a crucial part of the process.

To create your AWS Batch compute environment:

1\. Sign in to the [AWS Management Console](https://aws.amazon.com/console/) with your AWS account.

2\. Next, search for **Batch**, select the **Batch** service, and your browser redirects to the AWS Batch page (step three).

![Opening the AWS Batch on the AWS Management Console](https://adamtheautomator.com/wp-content/uploads/2022/06/image-592.png)

Opening the AWS Batch on the AWS Management Console

3\. Once on the **AWS Batch** page, click on **Create a compute environment button,** as shown below, to initiate creating a compute environment.

Your browser redirects to a page where you’ll configure a compute environment (step four).

![Initiating creating the compute environment for an AWS Batch job](https://adamtheautomator.com/wp-content/uploads/2022/06/image-593.png)

Initiating creating the compute environment for an AWS Batch job

4\. Now, configure the compute environment with the following under the **Compute environment configuration** section:

*   **Compute** **environment type** – Select **Managed** so that AWS takes care of your instances.
    
*   **Compute** **environment name** – Provide an environment name you like, but this tutorial’s choice is **ATA-batch-env**.
    
*   **Service role** – Choose a service role that has permission to call other AWS services on your behalf. And even if you don’t have a service role, one is created on your behalf.
    

The **Service role** is set for this tutorial to the **Batch service-linked role**.

![Specifying the computer environment configuration](https://adamtheautomator.com/wp-content/uploads/2022/06/image-594.png)

Specifying the computer environment configuration

5\. Under the Instance configuration, set the following options:

*   **Provisioning model** – Select **Fargate**, which allows you to run containers without needing any instances you create.
    
*   **Maximum vCPUs** – Input 256, the default that the Fargate instance will use.
    

![Specifying the instance configuration for the compute environment](https://adamtheautomator.com/wp-content/uploads/2022/06/image-595.png)

Specifying the instance configuration for the compute environment

6\. Next, scroll down to the **Networking** section, and configure the following:

*   **VPC ID –** Select the VPC ID you created for this tutorial as a prerequisite.

Related:[Building an AWS Terraform VPC Step-by-Step](https://adamtheautomator.com/terraform-vpc/)

*   **Subnets –** There are subnets available for your AWS account you can select from the dropdown field. But if you wish to use the subnets of your choice, select the appropriate subnets where the compute resources will be launched.

This tutorial uses the Default subnets, as shown below.

> _The VPC ID and Subnets groups will be different for each account. The VPC and Subnets groups you see in this tutorial will differ from your AWS account._

*   Expand the **Additional settings: Security groups**, placement group option, and you’ll see the default security groups already selected. But again, you can associate the security group of your choice.

![Specifying the networking configuration for the compute environment](https://adamtheautomator.com/wp-content/uploads/2022/06/image-596.png)

Specifying the networking configuration for the compute environment

7\. Finally, click on the **Create compute environment** button (bottom-right), as shown below, to finalize creating the compute environment.

![Provisioning the compute environment ](https://adamtheautomator.com/wp-content/uploads/2022/06/image-597.png)

Provisioning the compute environment

After a few seconds, you’ll notice in the **Compute environments** tab that the environment you specified is now enabled, as shown below.

![Verifying the compute environment is enabled](https://adamtheautomator.com/wp-content/uploads/2022/06/image-598.png)

Verifying the compute environment is enabled

## Creating an AWS Batch Job Queue

You’ve just launched a compute environment for AWS Batch to run a job. But the compute environment is doing nothing until you create a job queue.

A job queue stores your submitted jobs until the AWS Batch Scheduler runs the job on a resource in your computing environment.

To create a job queue, follow these steps:

1\. On the AWS Batch Dashboard, click on Create job queue under the Job queue overview section. This action redirects your browser to a page where you’ll configure a job queue (step two).

![Creating a Job Queue in AWS Batch](https://adamtheautomator.com/wp-content/uploads/2022/06/image-599.png)

Creating a Job Queue in AWS Batch

2\. In the **Job queue configuration** section, provide the following details:

*   **Job queue name** – Enter a unique name, but this tutorial’s choice is **ATA-AWS-BATCHJOB**.
    
*   **Priority** – Enter an integer between 0 and 100 for the job queue. This tutorial uses the default value 1.
    
*   **Scheduling policy ARN** – Keep this option disabled since the scheduler manually keeps it disabled.
    
*   Adding tags is optional, so ignore the tags for this tutorial.
    

![Specifying the job queue configuration ](https://adamtheautomator.com/wp-content/uploads/2022/06/image-600.png)

Specifying the job queue configuration

*   Scroll down, and select the **ATA-batch-env** you created in step four of the “Setting up the Compute Environment” section as the compute environment.

After setting the compute environment, click the **Create** button to create the job queue.

![Specifying the compute environment in the job queue](https://adamtheautomator.com/wp-content/uploads/2022/06/image-601.png)

Specifying the compute environment in the job queue

4\. Finally, click on **Job Queues** in the left panel on the AWS Batch page to verify the newly-created job queue, as shown below.

![Verify the Job Queue on AWS Batch](https://adamtheautomator.com/wp-content/uploads/2022/06/image-602.png)

Verify the Job Queue on AWS Batch

## Creating an AWS Batch Job Definition

You now have a job queue where your job will reside. But one more component you need to create, which is [job definition](https://docs.aws.amazon.com/batch/latest/userguide/job_definitions.html), before finally submitting your job. The job definition contains all the job details, such as timeouts, AWS ECR links, etc.

To create the job definition:

1\. Navigate to the AWS Batch Dashboard, and click on Job definitions, as shown below, to access the list of job definitions.

![Accessing the list of job definitions](https://adamtheautomator.com/wp-content/uploads/2022/06/image-603.png)

Accessing the list of job definitions

2\. Next, click the **Create** button (top-right) to initialize a new job definition. Your browser redirects to a page where you’ll configure the new job definition (step three).

![Initiating creating a new job definition](https://adamtheautomator.com/wp-content/uploads/2022/06/image-604.png)

Initiating creating a new job definition

3\. On the **Create job definition** page, provide the following details:

*   Select the **Single-node** option under the **Job type** section since you’ll only run your job on a single node.
    
*   Provide a job’s **Name** as you like under the **General configuration** section. But this tutorial’s choice is **ATA-Job-Definition**.
    
*   **Execution timeout –** Enter the amount of time, in seconds, that an unfinished job terminates after running. The minimum timeout is 60 seconds.
    

![Specifying the job definition’s general configuration](https://adamtheautomator.com/wp-content/uploads/2022/06/image-605.png)

Specifying the job definition’s general configuration

4\. Scroll down to the **Platform compatibility** section, and configure the following:

*   **Platform type** – Choose **Fargate**, a server-less compute engine that allows you to build applications without managing a server.
    
*   Enable the **Assign public IP** option, so your AWS Batch job is accessible publicly if required.
    

Related:[How to Use PowerShell to Get an IP Address](https://adamtheautomator.com/powershell-get-ip-address/)

*   Enable the **Execution role** option, and choose a task execution role that lets Amazon Elastic Container Service (Amazon ECS) agents make AWS calls on your behalf. This tutorial uses the **ecsTaskExecutionRole** execution role.

![Specifying the platform compatibility configuration](https://adamtheautomator.com/wp-content/uploads/2022/06/image-606.png)

Specifying the platform compatibility configuration

5\. Now, set the following under the **Job configuration** section:

*   **Image** – Enter the Name of the image in _repository-url/image:tag_ format that will be used to launch the container. This tutorial uses the AWS ECR created as a prerequisite.

_Images in Amazon ECR repositories use the_ _registry/repository\[:tag\]__, and images in other online repositories use the domain name, such as_ _google.com/assemblyline/ubuntu__._

*   Choose **Bash** for the **Command syntax**, then specify the below **Command** to pass to the container. When you submit a new AWS Batch, the job container executes the below command.

```powershell
echo "hello world"
```

![Specifying the image, command syntax, and command](https://adamtheautomator.com/wp-content/uploads/2022/06/image-607.png)

Specifying the image, command syntax, and command

6\. Scroll down, and continue configuring the job definition with the following:

*   **vCPUs** – Specify the number of vCPUs to reserve for the container. The AWS Batch will not be too heavy, so this tutorial uses only one vCPU (1.0).
    
*   **Memory** – Specify the hard limit (in MiB) of memory (2 GB) to present to the job container.
    

![Specifying vCPUs and memory for the job](https://adamtheautomator.com/wp-content/uploads/2022/06/image-608.png)

Specifying vCPUs and memory for the job

7\. After specifying configurations in the job definition, click on the **Create** button (bottom-right) to create the job definition.

![Creating the job definition ](https://adamtheautomator.com/wp-content/uploads/2022/06/image-609.png)

Creating the job definition

8\. Navigate to the **Job definitions** page again to verify the newly-created job definition, as shown below.

![Verifying the Job definition ](https://adamtheautomator.com/wp-content/uploads/2022/06/image-610.png)

Verifying the Job definition

9\. Now, navigate to the **Jobs** page, and click on **Submit new job** (top-right) to submit your first AWS Batch job.

![Submitting a new AWS Batch Job](https://adamtheautomator.com/wp-content/uploads/2022/06/image-611.png)

Submitting a new AWS Batch Job

10\. Lastly, specify the following for the AWS Batch job you’re submitting:

*   **Name** – Provide a job name (**ATA-batch-Job**).
    
*   **Job definition** – Select the job definition you created in step three (**ATA-Job-Definition**).
    
*   **Job queue** – Select the job queue (**ATA-AWS-BATCHJOB**) you created in step two of the “Creating an AWS Batch Job Queue” section.
    
*   **Execution timeout** – Set the execution timeout to the minimum (**60**).
    
*   Click on **Submit** at the bottom of the page to submit the new job.
    

![Specifying the details for submitting a new AWS Batch Job](https://adamtheautomator.com/wp-content/uploads/2022/06/image-612.png)

Specifying the details for submitting a new AWS Batch Job

## Verifying the AWS Batch Job and Monitoring the Job

You’ve successfully submitted your first AWS Batch job, but how can you confirm if the job is working? AWS Batch creates logs for each running job, where you can verify whether your job is running or not.

Navigate to the **Jobs** page, and your job status says **SUCCEEDED** (bottom-right), as shown below. But if it’s not enough, you can get more information about your job’s status.

Click on your AWS Batch job (**ATA-batch-job**), and your browser redirects to your job’s status and logs.

![Verifying the new Job](https://adamtheautomator.com/wp-content/uploads/2022/06/image-613.png)

Verifying the new Job

Below, you can see the status also says **SUCCEEDED**.

![Checking the status of the AWS Batch job](https://adamtheautomator.com/wp-content/uploads/2022/06/image-614.png)

Checking the status of the AWS Batch job

Now, scroll to the bottom, and you’ll notice that the job ran successfully on the container, and executed the command you specified in the job definition.

![Verifying the AWS Batch ran successfully](https://adamtheautomator.com/wp-content/uploads/2022/06/image-615.png)

Verifying the AWS Batch ran successfully

## **Conclusion**

In this tutorial, you’ve learned how to use AWS Batch to run jobs against job definitions. AWS Batch Job provides a handy logging system that gives you in and out details of the Job that you execute.

The best part is that AWS batch is a free service that allows you to run the batch jobs periodically or on-demand while you only pay for the resources you use.

Now, why not use this newfound knowledge with other AWS services and build vital services?

Share this article

[Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Faws-batch%2F&text=AWS%20Batch%3A%20Your%20Answer%20to%20Efficient%20Job%20Scheduling%20%26%20Execution)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Faws-batch%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Faws-batch%2F)

## Related Posts

![](https://adamtheautomator.com/wp-content/uploads/2022/07/Move-Development-to-the-Cloud-with-the-AWS-Cloud9-IDE.jpg)

### [Simplify Development with AWS Cloud9: Your Cloud-Based IDE](/aws-cloud9/)

Simplify your development workflow, boost productivity, and unlock the power of collaboration with a cloud-based IDE. This AWS Cloud9 tutorial shows you how.

![](https://adamtheautomator.com/wp-content/uploads/2023/11/automating-tasks.jpg)

### [Automating Tasks Using Bash Scripts and Cron Jobs with AWS](/automating-tasks/)

Discover how to combine bash scripts with cron jobs and leverage the power of AWS for automating tasks in this ATA Learning tutorial!

![](https://adamtheautomator.com/wp-content/uploads/2023/10/aws-amplify.jpg)

### [AWS Amplify: Your Launchpad for Full Stack Apps](/aws-amplify/)

Learn how to create and deploy web applications with AWS Amplify and take your development to the next level in this ATA Learning tutorial!

## Categories

*   [IT Ops](/category/it-ops/)
*   [Cloud](/category/cloud/)
*   [DevOps](/category/devops/)
*   [Home Ops](/category/home-ops/)
*   [Information Security](/category/infosec/)
*   [Software Development](/category/software-development/)

## Site

*   [Home](/)
*   [Tutorials](/tutorials/)
*   [Instructors](/author/)
*   [Advertising](/advertising/)
*   [Recommended Resources](/resources/)
*   [About Adam](/about-adam/)

Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)