Unlocking Business Value: Workload Automation and Azure Queue Storage

Introduction:
Let us begin with understanding Azure Queue Storage, what it is all about before moving to our Azure Queue Storage Plug-in, and how it benefits our workload automation users. To empower your Workload Automation environment, download the Azure Queue Storage[1] [2]  plug-in available on Automation Hub.

​Azure Queue Storage is a versatile and scalable service for managing asynchronous communication between different parts of an application. It enables decoupling components, managing workloads, and handling task scheduling effectively. This service provides a reliable and secure way to achieve these goals. Azure Queue Storage allows for the storage and retrieval of messages, facilitating communication within cloud applications by enabling asynchronous message queuing.
Use Cases:

• Order Processing Systems: In e-commerce platforms, Azure Queue Storage can be used to manage and process customer orders asynchronously. When an order is placed, it is added to a queue, allowing the order processing system to handle each order individually and sequentially, ensuring no order is missed or processed out of sequence.
• Background Task Processing: For applications requiring background processing, such as image or video processing, Azure Queue Storage can queue tasks for processing by worker roles. This decouples task submission from processing, enabling better load management and scalability.
• Log Management: Applications can utilize Azure Queue Storage to store log messages. This is particularly beneficial for distributed systems where logs from various components need to be aggregated and processed. Logs can be queued and then processed by a logging service that writes them to a database or file system.
• Message Buffering: In scenarios where message buffering is required between different parts of a system, Azure Queue Storage can act as an intermediary. For example, in a microservices architecture, one service can place messages in a queue, and another service can read and process them at its own pace.
• Event Sourcing: Azure Queue Storage can be used to implement event sourcing, where changes to the application state are stored as a sequence of events. These events can be queued and processed to update the application state or trigger other actions.

Key Features:
Azure Queue Storage offers several key features that make it a powerful tool for managing asynchronous communication in cloud applications:

• High Availability: Ensures continuous message accessibility, providing reliable communication between application components.
• Scalability: Handles millions of messages, suitable for applications processing high volumes of tasks. Enables application growth without performance bottlenecks.
• Message Lifespan Customization: Allows specifying message retention time in the queue before automatic deletion, enabling flexible message management.
• Security: Messages are securely stored and accessible only by authorized users. Integrated encryption and Shared Access Signatures (SAS) enhance message security and control access to queues.
• Asynchronous Communication: Enables different parts of an application to communicate asynchronously, improving scalability and reliability. Facilitates efficient workload management by allowing tasks to be queued and processed later, reducing system load during peak times.
• Global Access: Messages are accessible from anywhere globally via authenticated HTTP or HTTPS calls.
• Message Queuing: Stores a large number of messages, each up to 64 KB in size, accessible from anywhere via authenticated HTTP or HTTPS calls.
• Component Decoupling: Decouples different parts of an application, improving scalability and reliability. Components can operate independently, enhancing system resilience and maintainability.
• Cost-Effectiveness: Priced on a pay-per-use model, making it a cost-effective solution for managing message queues. Pay only for storage and operations used, without ongoing costs.
• Resilience: Makes applications more resilient to individual component failures. Messages are buffered and can be processed later when the system recovers, maintaining workload integrity.
• Handling Bursts of Traffic: Absorbs unexpected traffic bursts, preventing server overload from a sudden influx of requests, maintaining application performance and reliability.

Getting Started:
To get started with Azure Queue Storage Plug-in, follow these basic steps below and let us see how easy it is to start and monitor the jobs.
Create a new job and select “Azure Queue Storage” in the Cloud section.

​Connection:
First, connect to the Azure Queue Storage by providing the required information.
 
Account Name
The account name of Azure Queue Storage. This field is mandatory.
Account Key
The account key of Azure Queue Storage. This field is mandatory.
Queue Name
The Queue Name of Azure Queue Storage. This field is mandatory.
Add the content 

​After that, by selecting Test Connection, you can check the connection to the Azure Queue Storage.

​Action:
You may go to the Action tab and specify the job's specifics as well as the action you wish to take after the connection has been successfully tested.
Send
You can select the "Send" to send a message.
 
Message Text
In this section you can upload the message which you want to send.
Message Expire In
Select the check box to set the message expiration duration while sending the message.
Time Duration
This section can be used to set the time duration of the message to expire.
Duration Type
This section can be used to select the duration type of the message expire, as it supports only: - Second, Minute, Hour and Day.
Message Never Expire
Select the check box to set that message never expire while sending the message.

Receive
You can select “Receive" to receive a message.
 
Number of Messages to Receive
The section provides information about how many messages that you want to receive.
Note: by default, it accepts one message if the number of messages is not specified.
 
Save Function to Display Output to File
Select the save function button to enable the Output file Path.
Output File Path
Provide the location to save the function output files.

​Saving and submitting your job:
Submit your job into the current plan. You can add your job to the job stream that automates your business process flow. Select the action menu in the top-left corner of the job definition panel and click on Submit Job into Current Plan. A confirmation message is displayed, and you can switch to the monitoring view to see what is going on.

​Monitor Page:
Users can track the jobs in the monitor page.

​If the job completes successfully in the backend Azure Queue Storage, the status should be changed to successful.

Job Log Details:
 
Send

Receive

In Conclusion:
​
The Azure Queue Storage plugin is a powerful tool that enables you to leverage the benefits of message queuing within your Workload Automation environment. By integrating with Azure Queue Storage, you can build more scalable, responsive, and reliable applications.

---

Ernesto Carrabba, Product Manager, HCL Clara, HCL HERO and HCL Workload Automation 
 
Ernesto Carrabba is the Product Manager for HCL Clara, HCL HERO and HCL Workload Automation. Ernesto is a very dynamic product manager with experience in building and launching IoT products, combined with a master's degree in mechanical engineering and study researches on Augmented and Virtual Reality