WORKLOAD AUTOMATION COMMUNITY
  • Home
  • Blogs
  • Forum
  • Resources
  • Events
    • IWA 9.5 Roadshows
  • About
  • Contact
  • What's new

Deep dive into Workload Automation Dynamic Scheduling

3/28/2019

1 Comment

 
Picture
This blog addresses some common questions related to Dynamic Scheduling in Workload Automation.

​Make a dynamic agent work
A question that I often hear from Workload Automation customers is:
​
​"
I would like to start using Dynamic Scheduling, but I only have FTAs in my environment. Do I have to install at least one dynamic agent for each FTA?"

It is not necessary to install a new dynamic agent if an FTA is already installed on the system, since a dynamic agent component is already installed together with the FTA. However, sometimes it is not working. The reason might be that the dynamic agent component hasn’t been started, or it is not properly configured, or the broker component on the MDM has not been well configured too.

Let’s start investigating the Broker component on the MDM side. We have to make sure that the Broker database contains the correct information. On the MDM run the following commands from <TWA home>/TDWB/bin:
root@EU-HWS-LNX110:/opt/IBM/TWA/TDWB/bin# ./exportserverdata.sh -dbUsr <db user> -dbPwd <db password>
Setting CLI environment variables....
AWKCLI168I Server data successfully exported.
root@EU-HWS-LNX110:/opt/IBM/TWA/TDWB/bin# cat server.properties
https://EU-HWS-LNX110.NPROD.HCLPNP.COM:31116/JobManagerRESTWeb/JobScheduler
The command exportserverdata saves in the file server.properties the list of Broker component addresses, typically the addresses of MDM and all the Backup MDMs. In this case the backup master is not set and EU-HWS-LNX110.NPROD.HCLPNP.COM is the hostname of the MDM while 31116 is WebSphere port for incoming requests.

If addressed are missing, or not correctly set, it is possible to correct them by editing the file server.properties,  importing the new values using the script <TWA home>/TDWB/bin/importserverdata.sh and passing the updated version of server.properties.

It is possible to verify if the import operation was really successful running again the script exportserverdata.sh and checking that the new addresses are now saved in server.properties output file.

​Now, it is necessary to connect the agent system where the FTA and the dynamic agent are installed. The dynamic agent processes are:
tws94fp2  2262     1  0 Sep03 ?        00:00:06 /home/tws94fp2/TWS/ITA/cpa/ita/agent
tws94fp2  2355  2262  0 Sep03 ?        00:55:35 /home/tws94fp2/TWS/bin/JobManager
i.e. agent (the watchdog process) and JobManager.

If they are running, stop them using the script:  <TWA home>/TWS/ShutDownLwa

After they have been stopped, make sure that the agent port to accept requests from the Broker is set. It is written in <TWA home>/TWS/ITA/cpa/ita/ita.ini. In [ITA] section.

​If you find the following settings:
tcp_port = 0
ssl_port = 31114
It means that secure communication on port 31114 is in place. If the port 31114 (the default one at installation time) is not free and cannot be used, it is possible to change this value to the proper port number. Then, edit the file <TWA home>/TWS/ITA/cpa/config/JobManager.ini and check that the following parameters are correctly set in [ResourceAdvisorAgent] section:
FullyQualifiedHostname = EU-HWS-LNX110.NPROD.HCLPNP.COM
ResourceAdvisorUrl = https://EU-HWS-LNX110.NPROD.HCLPNP.COM:31116/JobManagerRESTWeb/JobScheduler/resource
ComputerSystemDisplayName = EU-HWS-LNX110_1
Where:
  • FullyQualifiedHostname is the hostname of the system hosting the agent.  
  • ResourceAdvisorUrl must point to the hostname (or ip address) and port number that have just been set using importserverdata.
  • ComputerSystemDisplayName is the cpu name as we want it to be defined in the database after the agent registration.
Then, it is necessary to delete the lines with the key AgentID and BackupResourceAdvisorUrls: this will force a new agent registration if the agent successfully connects to the Broker component. Restart the dynamic agent with the script <TWA home>/TWS/StartUpLwa.

After the agent has been restarted, check the agent logs, in

<TWA home>/TWS/stdlist/JM/JobManager_message.log,
A sequence of messages like the following should be present:
2018-11-01 18:50:08.187-04:00|2108348224|29408|EU-HWS-LNX110.NPROD.HCLPNP.COM|Start Logger
2018-11-01 18:50:08.188-04:00|INFO|JobManager|2108348224|29408|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA047I Starting subagent "JobManager".
2018-11-01 18:50:08.282-04:00|INFO|JobManager|2108348224|29408|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA090I The Job Status Notification Manager started.
2018-11-01 18:50:08.310-04:00|INFO|JobManager|1786758912|29408|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA050I The agent is open for e-business.
2018-11-01 18:50:08.340-04:00|INFO|JobManager|2108348224|29408|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA257I The Event Driven Workload Automation component was created.
2018-11-01 18:50:08.343-04:00|INFO|JobManager|2108348224|29408|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA257I The Event Driven Workload Automation component was created.
2018-11-01 18:50:14.040-04:00|INFO|JobManager|1778366208|29408|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA083I Resource information was sent to "https://EU-HWS-LNX110.NPROD.HCLPNP.COM:31116/JobManagerRESTWeb/JobScheduler/resource".
The last message, AWSITA083I is the message that certifies that the agent is correctly started and is able to connect to the Broker component. Also, you can verify that the following entries have been added to JobManager.ini and updated in [ResourceAdvisorAgent] section:
AgentID = 2FAC9900CA2711E7BAE78921BEAF999E
BackupResourceAdvisorUrls = https://EU-HWS-LNX110.NPROD.HCLPNP.COM:31116/JobManagerRESTWeb/JobScheduler/resource
Note that BackupResourceAdvisorUrls must contain the same list of Broker server addresses as previously configured with exportserverdata.sh/importserverdata.sh scripts.
It is not necessary to define a dynamic agent cpu in the database. It is automatically defined after a successful registration.
CPUNAME EU-HWS-LNX110_1
  DESCRIPTION "This workstation was automatically created."
  OS UNIX
  NODE EU-HWS-LNX110.NPROD.HCLPNP.COM SECUREADDR 31114
  TIMEZONE America/New_York
  FOR MAESTRO HOST EU-HWS-LNX11_DWB
    AGENTID "2FAC9900CA2711E7BAE78921BEAF999E"
    TYPE AGENT
    PROTOCOL HTTPS
END
The dynamic agent cpu is automatically created with the AGENTID and cpu name that are present in the configuration file JobManager.ini hosted by the Broker workstation. The workstation will be ready for the scheduling activities after next FINAL or after JnextPlan –for 0000.
 ​

​Make the Gateway work
Another question I often hear from customer is: 

"I want to configure a dynamic agent, already installed and running, to communicate with the master domain manager through a gateway. Do I have to install a local or remote gateway?"

If you need to use a remote gateway, it is necessary to install it remotely. Otherwise, you can leverage the installation of the dynamic agent that is currently in use and manually configure the agent to communicate with the master domain manager through the local getaway already installed.

The local gateway configuration file is located in
<TWA installation directory>/TWS/ITA/cpa/config/JobManagerGW.ini
and is used to manage the gateway process JobManagerGW whose lifecycle is handled by the watchdog agent named “agent”.

To configure the local gateway, you need to know the local port of the agent process: it is the value of tcp_port or ssl_port attributes in the configuration file
 <TWA installation directory>/TWS/ITA/cpa/ita/ita.ini, in [ITA] section.

Suppose this value is NNNNN. Then, follow these steps to complete the configuration:
1. Stop the dynamic agent
2. Edit the configuration file JobManagerGW.ini and in [ITA] section make sure that the following settings are in place (consider that GW1 is the gateway ID that has been selected, and EU-HWS-LNX110.NPROD.HCLPNP.COM is the system hostname)
autostart = yes
JobManagerGWID = GW1
JobManagerGWURIs = https://EU-HWS-LNX110.NPROD.HCLPNP.COM:NNNNN/ita/JobManagerGW/JobManagerRESTWeb/JobScheduler/resource
3. Edit the configuration file JobManagerGW.ini and in [ResourceAdvisorAgent] section make sure that the following settings are in place, considering that the port is WebSphere https port
ResourceAdvisorUrl = https://<MDM hostname>:<port>/JobManagerRESTWeb/JobScheduler/resource
BackupResourceAdvisorUrls = https://<MDM hostname>:<port>/JobManagerRESTWeb/JobScheduler/resource
ResourceAdvisorUrl = https://<MDM hostname>:<port>/JobManagerRESTWeb/JobScheduler/resource

BackupResourceAdvisorUrls = https://<MDM hostname>:<port>/JobManagerRESTWeb/JobScheduler/resource
4. Edit the configuration file JobManager.ini and set the following attributes in the following way, considering that EU-HWS-LNX110.NPROD.HCLPNP.COM is, as above, the hostname of the system where the dynamic agent is installed
​ [ResourceAdvisorAgent]
ResourceAdvisorUrl = https://EU-HWS-LNX110.NPROD.HCLPNP.COM:NNNNN/ita/JobManagerGW/JobManagerRESTWeb/JobScheduler/resource
BackupResourceAdvisorUrls = https://EU-HWS-LNX110.NPROD.HCLPNP.COM:NNNNN/ita/JobManagerGW/JobManagerRESTWeb/JobScheduler/resource
5. Restart the dynamic agent.
​After the dynamic agent restart, the following processes are up and running:
tws94fp2 13166     1  0 15:50 ?        00:00:03 /home/tws94fp2/TWS/ITA/cpa/ita/agent
tws94fp2 13180 13166  0 15:50 ?        00:00:05 /home/tws94fp2/TWS/bin/JobManager
tws94fp2 13181 13166  0 15:50 ?        00:00:10 /home/tws94fp2/TWS/bin/JobManagerGW
In JobManager_message.log it is possible to check that JobManager is sending the system resources to the gateway address:
2018-11-25 15:50:17.969-05:00|INFO|JobManager|911644480|13180|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA047I Starting subagent "JobManager".
2018-11-25 15:50:18.190-05:00|INFO|JobManager|595584768|13180|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA050I The agent is open for e-business.
2018-11-25 15:50:24.992-05:00|INFO|JobManager|585017088|13180|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA083I Resource information was sent to "https://EU-HWS-LNX110.NPROD.HCLPNP.COM:3212/ita/JobManagerGW/JobManagerRESTWeb/JobScheduler/resource".
If a job is submitted, it is possible to follow the job lifecycle and the job status update that is sent to the gateway:
2018-11-25 15:52:16.978-05:00|540923648|13180|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA006I The job with ID "AGT94#JOBS.PP1.JNUM-804527197" was successfully created.
2018-11-25 15:52:17.059-05:00|INFO|JobManager|738891520|13180|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA031I The job with ID "AGT94#JOBS.PP1.JNUM-804527197" started.
2018-11-25 15:52:17.074-05:00|INFO|JobManager|738891520|13180|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA034I The job with ID "AGT94#JOBS.PP1.JNUM-804527197" completed successfully.
2018-11-25 15:52:18.437-05:00|INFO|JobManager|729966336|13180|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA109I The notification of the status of the job with ID "AGT94#JOBS.PP1.JNUM-804527197" was succesfully sent to the URI "https://EU-HWS-LNX110.NPROD.HCLPNP.COM:3212/ita/JobManagerGW/JobManagerRESTWeb/JobScheduler/job".
Also, in JobManagerGW_message you can see that system resources are forwarded to the master domain manager:
2018-11-25 18:40:24.518-05:00|INFO|JobManagerGW|1733740288|13181|EU-HWS-LNX110.NPROD.HCLPNP.COM|AWSITA307I Resource information was sent to "https://EU-HWS-LNX110.NPROD.HCLPNP.COM:31116/JobManagerRESTWeb/JobSchedulerGW/resourceGW".
Note that it is always possible to revert this configuration and manually pass from the communication through gateway to the direct connection to the master domain manager. This can be achieved in the following way:
1. Stop the dynamic agent.
2. Edit the configuration file JobManagerGW.ini and in [ITA] section make sure that the following settings are in place:
autostart = no
3. Edit the configuration file JobManager.ini and in [ResourceAdvisorAgent] section make sure that the following settings are in place, considering that the port is the WebSphere https port:
ResourceAdvisorUrl = https://<MDM hostname>:<port>/JobManagerRESTWeb/JobScheduler/resource
BackupResourceAdvisorUrls = https://<MDM hostname>:<port>/JobManagerRESTWeb/JobScheduler/resource
4. Restart the dynamic agent.

After the above steps the dynamic agent is directly connected to the master domain manager without passing through the gateway.
​
If you have any feedback or want to know more about Dynamic Scheduling, contact Paolo Salerno,  paolo.salerno@hcl.com or comment below.
​
Picture

​Paolo Salerno

Lead Specialist 

Paolo is a Senior Software Engineer. He has been working in the Workload Automation area since 2011 and in the last 6 years has been covering the position of L3 Support Team Leader.
1 Comment
Noppadol
1/29/2020 10:41:22 pm

Hi Paolo

Good article but I've a problem for register DA to the Broker due to some wrong configuration while install the agent

then I try reinstall DA but I don't get the parameter BackupResourceAdvisorUrls in JobManager.ini

I try put it manually and restart DA many time but problem still exist

I try delete AgentID and restart DA but result is the same

Can you guide me what should I do to fix this problem

Thank you

Reply

Your comment will be posted after it is approved.


Leave a Reply.

    Archives

    March 2023
    February 2023
    January 2023
    December 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    March 2022
    February 2022
    January 2022
    December 2021
    October 2021
    September 2021
    August 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    February 2021
    January 2021
    December 2020
    November 2020
    October 2020
    September 2020
    August 2020
    July 2020
    June 2020
    May 2020
    April 2020
    March 2020
    January 2020
    December 2019
    November 2019
    October 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017

    Categories

    All
    Analytics
    Azure
    Business Applications
    Cloud
    Data Storage
    DevOps
    Monitoring & Reporting

    RSS Feed

www.hcltechsw.com
About HCL Software 
HCL Software is a division of HCL Technologies (HCL) that operates its primary software business. It develops, markets, sells, and supports over 20 product families in the areas of DevSecOps, Automation, Digital Solutions, Data Management, Marketing and Commerce, and Mainframes. HCL Software has offices and labs around the world to serve thousands of customers. Its mission is to drive ultimate customer success with their IT investments through relentless innovation of its products. For more information, To know more  please visit www.hcltechsw.com.  Copyright © 2019 HCL Technologies Limited
  • Home
  • Blogs
  • Forum
  • Resources
  • Events
    • IWA 9.5 Roadshows
  • About
  • Contact
  • What's new