Further innovation in workload automation: Plug-ins

In my previous post on innovation in workload automation, I talked about the challenges of job definitions and how end-to-end batch processing has helped over the last 50 years.  IBM Workload Scheduler  (IWS) can automate batch processing independent of the target platform to monitor and control workflow throughout the enterprise IT infrastructure. In this post, I want to continue talking about the evolution of IBM Workload Scheduler and how new innovations have helped with the challenges of job definition. ​

In the early 2000s, the ability to schedule simple scripts or commands was not enough anymore. In fact there were a small number of typical common applications, like SAP, Oracle, IBM Storage Manager and PeopleSoft, that had internal batch processing through jobs that were often written in a proprietary language. 
 
To include this type of batch processing in the whole workload automation production environment, the IBM Rome Lab created specific agents, one for each type of application. So, we had the IBM Workload Scheduler for SAP extended agent together with three other IBM Workload Scheduler extended agent software components. 
 
But information technology evolution never stops, so from the end of the last millennium until today we have had an exponential growth of several vendor applications, like Java, database, Cognos, VMware, IBM InfoSphere DataStage, Structured Query Language (SQL), Urban Code Deploy, file transfer, and many others—most of them also requiring scheduling capabilities coverage. 
  ​

Can you imagine how much development effort would be required today if the HCL Lab had to develop an extended agent for every new vendor application that came into the market? 
It would be an enormous challenge, but there is a lucky fact! Almost all new applications that come into the marketplace support the XML language. This means that an XML source file can be used to run proprietary application jobs. This is valid both when the job starts from a distributed engine and when it starts from a IBM z/OS scheduling engine. In this second case, which is more common, it is important to be able to produce XML jobs if we want to schedule application vendors’ jobs from z/OS and we are also dealing with the lack of XML skill (as discussed in my earlier post). 
 
 The importance of XML 
The XML language reached such a high penetration inside the world of IT that now it is impossible to give up using it, since as I said almost all existing emerging applications today support XML for executing a sequence of their native commands. Try to think how a typical end-to-end batch processing environment, independent of its size, can gain an advantage from this language. Many executions of a batch workload can be automated without caring about which applications should be targeted. This way different types of jobs are able to run in sequence or in parallel, remaining in the same batch flow. 
  
Now we can come back to the initial unresolved question from part 1: Who is going to write these XML jobs to be initiated from z/OS and executed outside it in order to run proprietary application batch jobs? This issue can be solved with the Workload Automation Plug-in feature, which is in my opinion one of the most innovative enhancements we had in our IBM workload automation solution during the last 10 years. 
  
The workload automation plug-in innovation 
 I started this series of blog posts from the conclusion by showing you an XML job source member in z/OS. Finally, now I can show you the beginning of the story by looking more closely at the heart of this very strong innovation: workload automation plug-ins. Considering that a new extended agent for every emerging application has a high cost in effort and is not practical anymore, the solution now is interfacing an application’s batch through XML executions. The bottleneck of this solution is writing such jobs. The workload automation plug-in feature eliminates this bottleneck.

As you can see in the previous example, the IWS workload console provides a very user-friendly screen, where we can specify the library where the XML job will be placed and the parameters that will be used to build the XML sequence of commands, including security information. The result of this task is the complete creation of the XML job. The example shows a file transfer plug-in. Even if a plug-in is valid for every type of platform, imagine how important this can be for a mainframe-centric workload automation environment. 

There is no longer a need to develop an extended agent for every emerging application. It is enough to provide the new plug-in, which requires a very small development effort and at the same time provides a much easier implementation for customers.

​If you work with z/OS, I know you needed to write a JCL (Job Control Language) at least one time, for example, for copying one library to another. If you work with UNIX or Linux environments, I am sure you had to prepare a scripting language source file in order to execute several commands at the same time. But did you ever try to write an XML source file to execute a job, especially if you have a typical mainframe background? 

It would be great if you would share with us your experiences with different types of jobs, especially if you had to deal with emerging applications. Is there anything more you would suggest to HCL laboratory to enhance innovation in workload automation? Please share your ideas with me on Twitter, @nicochillemi.

Nico Chillemi
Executive IT Specialist

Domenico Chillemi, known as Nico, is an Executive IT Specialist in IBM Italy, with the role of Europe IOT Technical Leader for zSystems IT Service Management and Security technical sales. He is also the IBM Italy Technical Specialist Profession Leader, very careful to colleagues career and eminence. He belongs to the IBM Academy of Technology (AoT) Leadership Team as AoT Core Team member and also to the IBM zChampion team, leading all zSystems initiatives inside the IBM Academy of Technology. His main expertise is Workload Automation, especially in mainframe and end to end, and he tightly collaborates with HCL Technologies, being one of the IBM contacts point for customers with HCL. Moreover, he has also a strong knowledge of zStorage tools, System Automation and Security on zSystems.