WORKLOAD AUTOMATION COMMUNITY
  • Home
  • Blogs
  • Forum
  • Resources
  • Events
    • IWA 9.5 Roadshows
  • About
  • Contact
  • What's new

Connect AWS Batch with Workload Automation

8/10/2022

0 Comments

 
Picture
Let us first understand about the AWS Batch, before moving on to AWS Batch plug-in and how it is benefits to our workload automation users. 
The AWS Batch is available on Automation Hub, download it to empower your Workload Automation environment.  
Picture
​AWS Batch enables developers, scientists, and engineers to run hundreds of thousands of batch computing jobs easily and efficiently on AWS. AWS Batch dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted. With AWS Batch, there is no need to install and manage batch computing software or server clusters that you use to run your jobs, allowing you to focus on analysing results and solving problems. AWS Batch plans, schedules, and executes your batch computing workloads across the full range of AWS compute services and features.
 
Now let us see how easy it is to start and monitor AWS Batch job.
Create a new job and select “AWS Batch” in the Cloud section. 
Picture
Figure 1: Job Definition page

Connection
 
First, establish a connection to the AWS server by entering the required details.
Then, you can test the connection to AWS server by clicking on Test Connection.
Picture
Figure 2: Connection page
Picture
Figure 3: Connection page > Test Connection

Action

After having successfully tested the connection, you can go to the Action tab and specify the details about the job queue and define what actions you want to perform. 
 
·      Job Name: Specify the name of the job, it must be unique among the running jobs. 
·    Job Queue: Jobs are submitted to a job queue where they reside until they can be scheduled to run in a compute environment. If you do not have any job queue, create one. You can choose job queue with scheduling priority or without scheduling priority.
·      Job Definition: It specifies how the jobs should run. You can create/use (if exist) single node job definition or multiple job definition. Select any one of the job definition.
·      Share identifier: Use any text as share identifier if you use only job queue with scheduling priority. Do not enter any text on share identifier field if you use job queue without scheduling priority.
Picture
Figure 4: Action page

Job Configurations:

VCPUs: Choose the number of vCPUs that your compute environment should launch with. As your job queue demand increases, AWS Batch can increase the desired number of vCPUs in your compute environment. The minimum vcpu you can use is 1.

Memory: Specify the memory hard limit (in MiB) present to the container.
 GPUs: There are different types of GPU available, specify the GPUs value. This is an optional field.
Picture
Figure 5: Action page > Job Configurations

Command:
 User can use different commands for AWS batch to display in job logs for Ex:
 User can write “echo” in first field

Click on + button

Then write any text like “Job Success”

This message should display in job logs after job success 
Environment Variable:
AWS Batch automatically sets specific environment variables in container jobs. These environment variables provide introspection for the containers inside jobs, and you can use the values of these variables in the logic of your applications. All variables that are set by AWS Batch begin with the prefix, AWS_BATCH_. 
For Ex:
Name: AWS_BATCH_JQ_NAME Value :Job queue name
Picture
Figure 6: Action page > Environment

Parameters:
 
When you submit a job, you can specify parameters that should replace the placeholders or override the default job definition parameters. Parameters in job submission requests take precedence over the defaults in a job definition. 
 
For Ex:
 
Name: command Value: [“abc” , “data”]
Picture
Figure 7: Action page > Parameters

 
Submitting your job 
​  
Submit your job into the current plan. You can add your job to the job stream that automates your business process flow. Select the action menu in the top-left corner of the job definition panel and click on Submit Job into Current Plan. A confirmation message is displayed, and you can switch to the Monitoring view to see what is going on. 
Picture
Figure 8: Submitting job

Monitor Page:
Picture
Figure 9: Monitor page
 
If Job is success in backend AWS Batch then status should be changed to successful.
Picture
Figure 10: Monitor page contd.

You can cancel the running job by clicking kill option.
Picture
Figure 11: Monitor page contd.
Picture
Picture
Figure 12: Job log

Authors Bio
Picture
Shivansh Verma – Senior Software Engineer at HCL Software
Works as a Plug-in Developer in Workload Automation. Proficient in Java, C#, Python, C++, SQL, AI/ML

Picture
D Krishna Kumar – Senior Software Engineer(Tester) at HCL Software
​
Working as a Tester in Work Load Automation For the different plugins. Acquired skills on Functional Testing, Regression Testing, Smoke Testing, java, selenium
0 Comments

Your comment will be posted after it is approved.


Leave a Reply.

    Archives

    March 2023
    February 2023
    January 2023
    December 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    March 2022
    February 2022
    January 2022
    December 2021
    October 2021
    September 2021
    August 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021
    December 2020
    November 2020
    October 2020
    September 2020
    August 2020
    July 2020
    June 2020
    May 2020
    April 2020
    March 2020
    January 2020
    December 2019
    November 2019
    October 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017

    Categories

    All
    Analytics
    Azure
    Business Applications
    Cloud
    Data Storage
    DevOps
    Monitoring & Reporting

    RSS Feed

www.hcltechsw.com
About HCL Software 
HCL Software is a division of HCL Technologies (HCL) that operates its primary software business. It develops, markets, sells, and supports over 20 product families in the areas of DevSecOps, Automation, Digital Solutions, Data Management, Marketing and Commerce, and Mainframes. HCL Software has offices and labs around the world to serve thousands of customers. Its mission is to drive ultimate customer success with their IT investments through relentless innovation of its products. For more information, To know more  please visit www.hcltechsw.com.  Copyright © 2019 HCL Technologies Limited
  • Home
  • Blogs
  • Forum
  • Resources
  • Events
    • IWA 9.5 Roadshows
  • About
  • Contact
  • What's new