WORKLOAD AUTOMATION COMMUNITY
  • Home
  • Blogs
  • Forum
  • Resources
  • Events
    • IWA 9.5 Roadshows
  • About
  • Contact
  • What's new

Passing Variables from an Event Rule to a Job Stream

10/22/2020

0 Comments

 
Picture
Event Rules are an extension of Workload Automation (WA) capabilities that enable events occurring external to the scheduling environment to trigger actions on scheduling objects within WA.  An ideal use of this capability is to detect the arrival of a file and then trigger an action to submit a Job Stream containing jobs to process the data contained in that file.  This capability has been available for a while and is widely used.  In this article, a hidden feature is explored where the name of the file and other properties related to the file are passed as variables to a Variable Table associated to the Job Stream being submitted as ac action where any Job within that Job Stream can retrieve those variables and process the data in the file. ​
Create an Event Rule with the Event, File Created, and Action, Submit Job Stream 

​Follow the steps below to create an Event Rule 
  1. Login to the Dynamic Workload Console  
  2. Navigate to Design à Create Event Rules 
  3. Click on Create new à Event Rules 
  4. Click on the field, <not_named> and the General Info panel is displayed on the right 
  5. Click on the Name field, and enter the name, IF_FILE_EXISTS_PROCESS_FILE 
  6. Click on Save as draft to disable it 
  7. Click on Add Events à File Monitor à File created (click on + sign) à Close 
  8. Click on the event just added, the Properties panel opens on the right 
  9. Fill in the values in the fields as shown below: 
​​​         a. File name: /tmp/datafile.*.txt              An existing file with a matching name will not satisfy the condition, a new file with
                                                                           a
  matching name must be created. 
         b. Sample interval: 300                           Indicates how often in seconds to check for the file 
         c. Workstation: EEL ​
Picture
10. Click on Add Actions à IBM Workload Scheduler actions à Submit job stream
11. Click on the Action just added, the properties panel opens on the right
12. Fill in the values in the fields as shown below:
     a. Job stream name:         PROCESS_FILEThe name of the Job Stream to be submitted that contains jobs to process the
​                                                contents of the file.
      b. Workstation name:       EEL
Picture
       c. ​ Custom parameter 1:          Click on the Variables icon on the right and choose fileCrtEvt1 à File name
                                                         FILE_NAME=%{fileCrtEvt1.FileName} will be entered
                                                         The value of File name will be passed to the Variable, FILE_NAME
Picture
        d. Custom parameter 2:      Similarly, click on Variables icon on the right and choose any other variable that needs to be
                                                     passed as Variables, e.g.
         e. Custom parameter 3:     WORKSTATION=%{fileCrtEvt1.Workstation}​
         f. Custom parameter 4:      TIME_STAMP=%{fileCrtEvt1.TimeStamp}​​​
         g. Custom parameter 5:     HOSTNAME=%{fileCrtEvt1.Hostname}​
         h. Custom parameter 6:     IP_ADDRESS=%{fileCrtEvt1.IPAddress}
         i. Custom parameter 7:      EVENT_RULE_ID=%{fileCrtEvt1.EventRuleId}
Picture
13. Click on the selection box next to the event Rule name and click on Save on the menu bar.
14. Click on the X to close the successful saved message.
15. The following is the Event Rule definition is composer format.
<?xml version="1.0"?>
<eventRuleSet   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xmlns="http://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules"
                xsi:schemaLocation="http://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules http://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules/EventRules.xsd">
        <eventRule name="IF_FILE_EXISTS_PROCESS_FILE" ruleType="filter" isDraft="no">
                <eventCondition name="fileCrtEvt1" eventProvider="FileMonitor" eventType="FileCreated">
                        <scope>
                                /TMP/DATAFILE.*.TXT ON EEL
                        </scope>
                        <filteringPredicate>
                                <attributeFilter name="FileName" operator="eq">
                                        <value>/tmp/datafile.*.txt</value>
                                </attributeFilter>
                                <attributeFilter name="SampleInterval" operator="eq">
                                        <value>300</value>
                                </attributeFilter>
                                <attributeFilter name="Workstation" operator="eq">
                                        <value>EEL</value>
                                </attributeFilter>
                        </filteringPredicate>
                </eventCondition>
                <action actionProvider="TWSAction" actionType="sbs" responseType="onDetection">
                        <scope>
                                SBS EEL#PROCESS_FILE
                        </scope>
                        <parameter name="JobStreamWorkstationName">
                                <value>EEL</value>
                        </parameter>
                        <parameter name="Parm_1">
                                <value>FILE_NAME=%{fileCrtEvt1.FileName}</value>
                        </parameter>
                        <parameter name="Parm_2">
                                <value>SAMPLE_INTERVAL=%{fileCrtEvt1.SampleInterval}</value>
                        </parameter>
                        <parameter name="Parm_7">
                                <value>EVENT_RULE_ID=%{fileCrtEvt1.EventRuleId}</value>
                        </parameter>
                        <parameter name="Parm_5">
                                <value>HOSTNAME=%{fileCrtEvt1.Hostname}</value>
                        </parameter>
                        <parameter name="Parm_6">
                                <value>IP_ADDRESS=%{fileCrtEvt1.IPAddress}</value>
                        </parameter>
                        <parameter name="Parm_3">
                                <value>WORKSTATION=%{fileCrtEvt1.Workstation}</value>
                        </parameter>
                        <parameter name="JobStreamName">
                                <value>PROCESS_FILE</value>
                        </parameter>
                        <parameter name="Parm_4">
                                <value>TIME_STAMP=%{fileCrtEvt1.TimeStamp}</value>
                        </parameter>
                </action>
        </eventRule>
</eventRuleSet>


​16. Select the Job Stream Name and choose Select an Action à Submit Job Stream into Current Plan

​Create a Variable Table, Job, and Job Stream with to process the file

Follow the steps below to create the objects
  1. Login to the Dynamic Workload Console 
  2. Navigate to Design à Manage Workload Definitions
  3. Click on Create New à Variable Table
  4. In the General tab, enter in the Name field, PASS_VAR
  5. The variable to hold the properties of the file monitored are created and referenced internally, as such, no Variables need to be created.  
  6. Click on the Save icon to save the Variable table.
  7. The following is the Variable Table definition is composer format.
VARTABLE PASS_VAR
  MEMBERS
END

​8. Click on Create New à Job Definition à UNIX
9. In General tab, enter in the Name filed, FILE_RECEIVED_N_PROCESSING
10. In the Task tab, Select Script and enter in the Script name field:

/bin/echo -e "\nThe properties of the file received are: \n\nFile Name=^FILE_NAME^ \nSample Interval=^SAMPLE_INTERVAL^ \nWorkstation=^WORKSTATION^ \nTime stamp=^TIME_STAMP^ \nHostname=^HOSTNAME^ \nIP Address=^IP_ADDRESS^ \nEvent Rule ID=^EVENT_RULE_ID^ \n\nThe next job in the Job Stream can now start processing the file.\n"
11. Notice that the same variable names are used to retrieve the values as they are assigned to the respective properties of the file monitored in the Event Rule, IF_FILE_EXISTS_PROCESS_FILE.

​12. Click on the save icon to save the Job 
13. The following is the Variable Table definition is composer format.
EEL#FILE_RECEIVED_N_PROCESSING
 SCRIPTNAME "/bin/echo -e \"\nThe properties of the file received are: \n\nFile Name=^FILE_NAME^ \nSample Interval=^SAMPLE_INTERVAL^ \nWorkstation=^WORKSTATION^ \nTime stamp=^TIME_STAMP^ \nHostname=^HOSTNAME^ \nIP Address=^IP_ADDRESS^ \nEvent Rule ID=^EVENT_RULE_ID^ \n\nThe next job in the Job Stream can now start processing the file.\n\""
 STREAMLOGON iwadmin
 DESCRIPTION "A Job to demo how to retrieve variables passed from an Event Rule"
 TASKTYPE UNIX
 RECOVERY STOP

14. Click on Create New à Job Stream
15. In General tab, enter in the Name filed, PROCESS_ FILE
16. Scroll down to the Variable table field and enter PASS_VAR to associate the variable table to the Job Stream.
17. In the Details view, right click on the Jobs and choose Add Jobs
18. Enter in the Job definition field, FILE_RECEIVED_N_PROCESSING, and click on Add
19. Click on the save icon to save the Job Stream
Picture
20. The following is the Variable Table definition is composer format.
SCHEDULE EEL#PROCESS_FILE 
DESCRIPTION "A Job Stream to demo how to retrieve a variable passed from an Event Rule"
VARTABLE PASS_VAR
:
EEL#FILE_RECEIVED_N_PROCESSING
END

Monitor the execution of the Event Rule

​
Follow the steps below to create the file, monitor the execution of the Event Rule, submission of the Job Stream, and the Job log to view the variable passed from the Event Rule to the Job Stream.

1. Login to server where the file is monitored and create a file to simulate the arrival of the file cd /tmp
touch datafile.100120201202.txt
2. On the MDM, monitor the contents of the messages.log cd /opt/ibm/iwa/iws/TWSDATA/stdlist/appserver/engineServer/logs
tail -f messages.log
3. Once messages are displayed about the Event Rule being executed, login to the Dynamic Workload Console and navigate to monitoring the Job stream.
4. The Job Stream, PROCESS_FILE, is submitted.  
5. Go into Jobs view and monitor the status of the Job, FILE_RECEIVED_N_PROCESSING
​6. Once it is successful, view the Job Log.
Picture
7. Notice that the Variables are successfully passed from the Event Rule to the Job Stream and the Job is able to retrieve them from the Variable Table.
8. Also notice that the Variable Table has not been populated with the variables but were mapped to it internally.
VARTABLE PASS_VAR
MEMBERS
END
The same process can be used to create an Event Rule with Modification completed file monitor event.
​
Want to learn more? Write to me at Sajjad.Kabir@hcl.com or schedule a demo.
Author's Bio
Picture
​Sajjad Kabir, Solutions Architect, HCL Software
 
Sajjad Kabir is an Information Technology professional with over 25 years of diverse industry experience with primary emphasis in IT Architecture and Service Management. Extensive experiences in management, architecture, design, development, implementation, and systems integration solutions in multiple industries, platforms, and network environments. 
View my profile on LinkedIn
0 Comments

Your comment will be posted after it is approved.


Leave a Reply.

    Archives

    March 2023
    February 2023
    January 2023
    December 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    March 2022
    February 2022
    January 2022
    December 2021
    October 2021
    September 2021
    August 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021
    December 2020
    November 2020
    October 2020
    September 2020
    August 2020
    July 2020
    June 2020
    May 2020
    April 2020
    March 2020
    January 2020
    December 2019
    November 2019
    October 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017

    Categories

    All
    Analytics
    Azure
    Business Applications
    Cloud
    Data Storage
    DevOps
    Monitoring & Reporting

    RSS Feed

www.hcltechsw.com
About HCL Software 
HCL Software is a division of HCL Technologies (HCL) that operates its primary software business. It develops, markets, sells, and supports over 20 product families in the areas of DevSecOps, Automation, Digital Solutions, Data Management, Marketing and Commerce, and Mainframes. HCL Software has offices and labs around the world to serve thousands of customers. Its mission is to drive ultimate customer success with their IT investments through relentless innovation of its products. For more information, To know more  please visit www.hcltechsw.com.  Copyright © 2019 HCL Technologies Limited
  • Home
  • Blogs
  • Forum
  • Resources
  • Events
    • IWA 9.5 Roadshows
  • About
  • Contact
  • What's new