Integrating IBM Z Workload Scheduler with IBM CDP: How to easy access Audit Info

IBM Z Workload Scheduler collects many information about its workload execution and data base definitions in different formats. You can obtain info from messages issued on SYSTEM and/or on IzWS product LOG, or you can obtain it from AUDIT function reports. 
​
Starting from this info, an analysis can be done to better tuning workload, prevent problems and so on.  

So, what is the problem?

We could summarize it in the following key pain areas:

• The amount of info, that can be very big, makes analysis difficult.
• Info evaluation is more meaningful together with other kind of info, like mails, system messages and so on.
• AUDIT reports are written an “old style” not so easy to be immediately understood.

The analysis is a task for an analytic tool.  That’s the point. The conclusion is that we need to make IzWS info available to analytic tools (Splunk, Elastic Search, IBM IOAz or others).

This is what we have done in IzWS 9.5 release. 
We make our data available to analytic tools by means of CDP (IBM Common Data provider) integration:

1. IzWS 9.5 collects (if wished) the above-mentioned info in centralized place (unix files)  
2. IBM CDP read this info from unix files and transmit to defined analytic tools
   
3. Analytic tools get the info and can use them 

How To Set It Up?

To enable the Integration, you can simply:

1. Create a Unix directory where to store the Unix files used to communicate with CDP
2. Add the new parameter CDP(YES) and the CDPPATH just defined to OPCOPTS in the initial parameters on controller side
3. Customize CDP to use the IzWS Unix files as an input to be considered for data collection

Following these three steps IWSZ uses the CDP integration defaults, that is:

• Log info about actions performed on data base                          - > CDPDB(YES)
• Log info about Modify Current Plan actions                                  - > CDPMCP(YES)
• Log info about status change to complete and Start                    - >  CDPJOBSTATUS(CS)
• Switch the Unix file where to log info every 1000 records           - >  CDPSWITCHLOGLIM(1000) 

CDPDB(YES|NO)
Set NO to avoid logging of events related to actions performed in the controller database. 

CDPMCP(YES|NO)
Set NO to avoid logging of events related to MCP actions. 

CDPJOBSTATUS (CS|changed operation statuses|NO)
Specify the string including all status changes you want to log. 
E.g. CEAR* for complete, error and  ready. 
CDPSWITCHLOGLIM(number of records|1000)
Specifies how many records must be written in the CDP log before the CDP log switching function is started. 

You can create samples for OPCOPTS CDP parameters by using EQQJOBS customization panels:

INFO STORED IN UNIX FILES
Every record of Unix files is filled up with a textual message having always the same structure:

1. Date Time         
   1. format is YY/MM hh.mm.ss
2. Controller subsystem name
3. Message id       
   1. EQQ8nnnn for record corresponding to MCP / DB actions logged on JT data sets
   2. Original message id for messages taken from Controller MLOG
4. Message text

Not all the IzWS messages are logged into the Unix files.
​
By default, IzWS logs all the error messages and the following subset of informational messages: 

​In any case, you can always add other messages by setting in them CDP=YES.

CDP INFO

You can find all needed info about CDP on “IBM Common Data Provider for z Systems User Guide” from where we extract the following summary info:

CDP Provides z operations data in a consumable and consistent form for multiple subscribers, such as 

• IBM Operations Analytics for z Systems (IOAz)
• IBM DB2 Analytics Accelerator for z/OS (IDAA)
• 3rd-party analytics platforms, including Splunk.

It includes the following basic components: 
1) A Configuration Tool for defining the sources from which you want to collect operational data
2) The data gatherer components (System Data Engine and Log Forwarder) for gathering different types of operational data
3) A Data Streamer for streaming all data to its destination.

​The following picture summarize how we use it:

CONCLUSION

In conclusion, enabling the CDP is very easy and provides IzWS info to analytic tools in a simple way.

Even if you do not have CDP the function has a value: 
the Unix files can be easily processed by your own tool for your own purpose., because IzWS info are centralized in one place and in a textual format.

You can archive the inactive Unix files, once switched, and then use it in the way you need. 

AUTHOR’S BIO
ROSSELLA DONADEO – Advisory Software EngineerGraduated with a bachelor degree in mathematics in 1982. Technical leader of Workload Scheduler for z/OS. 
She worked for a couple of years in a small software house, and then in 1984 joined IBM. 
Since then she worked in level 3, development and verification.
Since 1996 she has been focused on Workload Scheduler for z/OS product. 
She successfully led the development of ZWS 950. 
She is a Mindfulness instructor, fond of trekking, yoga, tai chi, cooking and practices Vipassana meditation.
She is also a writer and had a book published.
 

---