In this Blog , we would look at a Banking Case Study wherein we are going through the “New Client Adoption and Reverse Check” Business process which is very important process in the Banking Industry
As we look into the Business Process , we are also going to look at the associated Business KPIs and how they are tracked by iControl at a Business Level as the Whole process is Automated in HCL Workload Automation and how any potential KPI Breach is addressed by HWA in real time and the how the Automated Recovery is carried out in HWA preventing a real SLA Breach from happening.
New Client Adoption and Reverse Check Business Process :
This is an important process for the Banking Industry realized as nightly batches to Screen Potential Customers and existing Customers and keep a Tab on Customers who are part of different Sanctions lists , highlighted below are the Steps in the Business Process :
There are various KPIs Associated with this process which need to be tracked in real time and acted upon so that we don’t breach any Business SLA and are able to get all the processes to be finished on/before time.
Below are some of the KPIs that need to tracked in real time during this Process :
The Screening Records for Politically exposed individuals must be received by Agreed Thresholds.
The Screening Records for Adverse Media Watchlists must be received by Agreed Thresholds.
The Screening Records for “Other Sanctions Lists” must be received by Agreed Thresholds.
The number of watchlists not loaded successfully must be equal to the agreed thresholds.
All Watchlists should be loaded into the System within agreed thresholds.
The Entire Business Process is to carried out prior to Business hrs everyday and should be ready by the morning time 09:00am .
Solution Realization in HCL Workload Automation and iControl :
The Entire Flow is Automated within HCL Workload Automation such that below steps of the Business Process are realized in the below way :
In each of the above Steps , KPIs are tracked in real time through iControl .
Flow within iControl to drive the Anti Money Laundering Process :
Flow within HCL Workload Automation to drive the Anti Money Laundering Process:
Flow in action within HCL Workload Automation :
In the job log , we find issues with the Feed and Exceptions and need to fetch a New Replacement File for the same . The Replacement File once obtained in the System , the job is rerun and the job starts executing again , this time the job finishes Successfully and is able to fetch the Adverse Media Watchlists :
KPIs Breached in iControl :
iControl tracks the associated KPIs that are affected due to the failure :
“The Screening records for adverse media must be received by the agreed threshold”
Business is aware and kept informed that the KPI affected due to this Batch Job Abend .
Successful Run post receiving the Replacement File :
The PEP Screening Job also completes Successfully within a Few mins and so does the Sanctions Lists Job.
The File Transfer job goes next and downloads watchlists for Adverse Media, PEP Screening and Sanctions Record Pull etc. from a Repository exposed by a Regulatory Authority, starts around 08:30pm IST and finishes by 08:34pm IST.
The Load Client Record Job starts around 08:34 pm IST and finishes within a Few Seconds.
The Load Watchlists Job starts running by 08:34pm IST and after downloading the PEP List and Adverse Media Listing , the DB Connections get exhausted and are stuck :
= JOB : MASTER_DA#NEW_CLIENT_ADOPT[(0000 01/23/23),(0AAAAAAAAAAAAJSW)].LOAD_WATCHLISTS
= USER : unixda
= JCLFILE : /home/unixda/LoadWatchLists.sh
= TWSRCMAP :
= AGENT : MASTER_DA
= Job Number: 584130431
= Mon 01/23/2023 20:34:17 IST
Loading WatchLists into Databases
[09/09/22 16:12:02:835 ICT] 00000001 com.ibm.ws.kernel.launch.internal.FrameworkManager I CWWKE0002I: The kernel started after 0.935 seconds
[09/09/22 16:12:02:881 ICT] 0000002c com.ibm.ws.kernel.feature.internal.FeatureManager I CWWKF0007I: Feature update started.
[09/09/22 16:12:03:366 ICT] 00000020 com.ibm.ws.security.ready.internal.SecurityReadyServiceImpl I CWWKS0007I: The security service is starting...
[09/09/22 16:12:04:053 ICT] 00000024 com.ibm.ws.cache.ServerCache I DYNA1001I: WebSphere Dynamic Cache instance named baseCache initialized successfully.
[09/09/22 16:12:04:053 ICT] 00000024 com.ibm.ws.cache.ServerCache I DYNA1071I: The cache provider default is being used.
[09/09/22 16:12:04:053 ICT] 00000024 com.ibm.ws.cache.CacheServiceImpl I DYNA1056I: Dynamic Cache (object cache) initialized successfully.
[09/09/22 16:12:04:460 ICT] 00000038 com.ibm.ws.security.token.ltpa.internal.LTPAKeyCreateTask I CWWKS4105I: LTPA configuration is ready after 0.070 seconds.
[09/09/22 16:12:04:460 ICT] 00000020 com.ibm.ws.security.ready.internal.SecurityReadyServiceImpl I CWWKS0008I: The security service is ready.
[09/09/22 16:12:06:006 ICT] 00000020 com.ibm.ws.sib.utils.ras.SibMessage I CWSID0108I: JMS server has started.
[09/09/22 16:12:06:116 ICT] 00000020 com.ibm.ws.security.jaspi.AuthConfigFactoryWrapper I CWWKS1655I: The default Java Authentication SPI for Containers (JASPIC) AuthConfigFactory class com.ibm.ws.security.jaspi.ProviderRegistry is being used because the Java security property authconfigprovider.factory is not set.
[09/09/22 16:12:06:381 ICT] 00000020 org.apache.cxf.bus.osgi.CXFActivator I Adding the extensions from bundle com.ibm.ws.org.apache.cxf.cxf.rt.ws.addr.2.6.2 (206) [org.apache.cxf.ws.addressing.policy.AddressingAssertionBuilder, org.apache.cxf.ws.addressing.policy.UsingAddressingAssertionBuilder, org.apache.cxf.ws.addressing.policy.AddressingPolicyInterceptorProvider, org.apache.cxf.ws.addressing.impl.AddressingWSDLExtensionLoader, org.apache.cxf.ws.addressing.WSAddressingFeature, org.apache.cxf.ws.addressing.MAPAggregator]
[09/09/22 16:12:06:397 ICT] 00000020 org.apache.cxf.bus.osgi.CXFActivator I Adding the extensions from bundle com.ibm.ws.org.apache.cxf.cxf.rt.bindings.soap.2.6.2 (209) [org.apache.cxf.binding.soap.SoapBindingFactory, org.apache.cxf.binding.soap.SoapTransportFactory]
[09/09/22 16:12:06:397 ICT] 00000020 org.apache.cxf.bus.osgi.CXFActivator I Adding the extensions from bundle com.ibm.ws.org.apache.cxf.cxf.rt.bindings.xml.2.6.2 (215) [org.apache.cxf.binding.xml.XMLBindingFactory, org.apache.cxf.binding.xml.wsdl11.XMLWSDLExtensionLoader]
[09/09/22 16:12:06:397 ICT] 00000020 org.apache.cxf.bus.osgi.CXFActivator I Adding the extensions from bundle com.ibm.ws.org.apache.cxf.cxf.rt.frontend.jaxws.2.6.2 (217) [org.apache.cxf.jaxws.context.WebServiceContextResourceResolver]
[09/09/22 16:12:06:397 ICT] 00000020 org.apache.cxf.bus.osgi.CXFActivator I Adding the extensions from bundle com.ibm.ws.org.apache.cxf.cxf.rt.management.2.6.2 (219) [org.apache.cxf.management.InstrumentationManager]
[09/09/22 16:12:06:397 ICT] 00000020 org.apache.cxf.bus.osgi.CXFActivator I Adding the extensions from bundle com.ibm.ws.org.apache.cxf.cxf.rt.transports.http.2.6.2 (223) [org.apache.cxf.transport.http.HTTPTransportFactory, org.apache.cxf.transport.http.HTTPWSDLExtensionLoader, org.apache.cxf.transport.http.policy.HTTPClientAssertionBuilder, org.apache.cxf.transport.http.policy.HTTPServerAssertionBuilder, org.apache.cxf.transport.http.policy.NoOpPolicyInterceptorProvider]
[09/09/22 16:12:06:397 ICT] 00000020 org.apache.cxf.bus.osgi.CXFActivator I Adding the extensions from bundle com.ibm.ws.org.apache.cxf.cxf.rt.ws.policy.2.6.2 (225) [org.apache.cxf.ws.policy.PolicyEngine, org.apache.cxf.policy.PolicyDataEngine, org.apache.cxf.ws.policy.AssertionBuilderRegistry, org.apache.cxf.ws.policy.PolicyInterceptorProviderRegistry, org.apache.cxf.ws.policy.PolicyBuilder, org.apache.cxf.ws.policy.PolicyAnnotationListener, org.apache.cxf.ws.policy.attachment.ServiceModelPolicyProvider, org.apache.cxf.ws.policy.attachment.external.DomainExpressionBuilderRegistry, org.apache.cxf.ws.policy.attachment.external.EndpointReferenceDomainExpressionBuilder, org.apache.cxf.ws.policy.attachment.wsdl11.Wsdl11AttachmentPolicyProvider, org.apache.cxf.ws.policy.mtom.MTOMAssertionBuilder, org.apache.cxf.ws.policy.mtom.MTOMPolicyInterceptorProvider]
[09/09/22 16:12:06:819 ICT] 00000020 com.ibm.ws.security.wim.VMMService I CWIMK0009I: The user registry federation service is ready.
[09/09/22 16:12:08:585 ICT] 00000024 com.ibm.ws.transport.iiop.internal.ORBWrapperInternal A CWWKI0001I: The CORBA name server is now available at corbaloc:iiop:localhost:2809/NameService.
[09/09/22 16:12:09:319 ICT] 00000045 com.ibm.ws.app.manager.AppMessageHelper I CWWKZ0018I: Starting application TWSEngineModel.
[09/09/22 16:12:09:319 ICT] 00000039 com.ibm.ws.app.manager.AppMessageHelper I CWWKZ0018I: Starting application SchedulerEAR
[09/09/22 16:12:09:319 ICT] 00000045 bm.ws.app.manager.ear.internal.EARDeployedAppInfoFactoryImpl I CWWKZ0135I: The TWSEngineModel application is using the expanded directory at the D:\Program Files\wa\usr\servers\engineServer\apps\TWSEngineModel.ear location.
[09/09/22 16:12:09:319 ICT] 00000039 bm.ws.app.manager.ear.internal.EARDeployedAppInfoFactoryImpl I CWWKZ0135I: The SchedulerEAR application is using the expanded directory at the D:\Program Files\wa\usr\servers\engineServer\apps\SchedulerEAR.ear location.
[09/09/22 16:12:09:600 ICT] 00000038 com.ibm.ws.webcontainer.osgi.webapp.WebGroup I SRVE0169I: Loading Web Module: RESTAPIDiscoveryUI.
[09/09/22 16:12:09:600 ICT] 00000031 com.ibm.ws.webcontainer.osgi.webapp.WebGroup I SRVE0169I: Loading Web Module: PublicRESTAPIDiscoveryServlet.
[09/09/22 16:12:09:600 ICT] 00000054 com.ibm.ws.webcontainer.osgi.webapp.WebGroup I SRVE0169I: Loading Web Module: com.ibm.ws.rest.api.discovery.subscription.
[09/09/22 16:12:09:600 ICT] 00000040 com.ibm.ws.webcontainer.osgi.webapp.WebGroup I SRVE0169I: Loading Web Module: ibm/api.
[09/09/22 16:12:09:600 ICT] 0000003a com.ibm.ws.webcontainer.osgi.webapp.WebGroup I SRVE0169I: Loading Web Module: PublicRESTAPIDiscoveryUI.
[09/09/22 16:12:09:600 ICT] 00000040 com.ibm.ws.webcontainer I SRVE0250I: Web Module ibm/api has been bound to default_host.
[09/09/22 16:12:09:600 ICT] 00000038 com.ibm.ws.webcontainer I SRVE0250I: Web Module RESTAPIDiscoveryUI has been bound to default_host.
[09/09/22 16:12:09:600 ICT] 00000054 com.ibm.ws.webcontainer I SRVE0250I: Web Module com.ibm.ws.rest.api.discovery.subscription has been bound to default_host.
[09/09/22 16:12:09:600 ICT] 00000031 com.ibm.ws.webcontainer I SRVE0250I: Web Module PublicRESTAPIDiscoveryServlet has been bound to default_host.
[09/09/22 16:12:09:600 ICT] 0000003a com.ibm.ws.webcontainer I SRVE0250I: Web Module PublicRESTAPIDiscoveryUI has been bound to default_host.
Export: Release 220.127.116.11.0 - Production on friday, 09 September, 2022 16:18:01
Copyright (c) 2018, 2019, Oracle. All rights reserved.
Connected to: Oracle Database 10g Express Edition Release 18.104.22.168.0 - Production
Starting SYSTEM.SYS_EXPORT_SCHEMA_01: SYSTEM/******** SCHEMAS=hr
DIRECTORY=dmpdir DUMPFILE=schema.dmp LOGFILE=expschema.log
Estimate in progress using BLOCKS method...
Processing object type SCHEMA_EXPORT/TABLE/TABLE_DATA
Total estimation using BLOCKS method: 448 KB
Processing object type SCHEMA_EXPORT/USER
Processing object type SCHEMA_EXPORT/SYSTEM_GRANT
Processing object type SCHEMA_EXPORT/ROLE_GRANT
Processing object type SCHEMA_EXPORT/DEFAULT_ROLE
Processing object type SCHEMA_EXPORT/TABLESPACE_QUOTA
Processing object type SCHEMA_EXPORT/PRE_SCHEMA/PROCACT_SCHEMA
Processing object type SCHEMA_EXPORT/SEQUENCE/SEQUENCE
Processing object type SCHEMA_EXPORT/TABLE/TABLE
Processing object type SCHEMA_EXPORT/TABLE/INDEX/INDEX
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type SCHEMA_EXPORT/TABLE/COMMENT
Processing object type SCHEMA_EXPORT/PROCEDURE/PROCEDURE
Processing object type SCHEMA_EXPORT/PROCEDURE/ALTER_PROCEDURE
Processing object type SCHEMA_EXPORT/VIEW/VIEW
Processing object type SCHEMA_EXPORT/TABLE/CONSTRAINT/REF_CONSTRAINT
Processing object type SCHEMA_EXPORT/TABLE/TRIGGER
Processing object type SCHEMA_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS
. . exported PEP.WATCHLIST 6.093 GB 225 rows
. . exported ADVERSEMEDIA.WATCHLIST 6.640 GB 227 rows
[09/09/22 16:12:09:600 ICT] [ERROR] [pool-3-thread-3] riodicNotificationService - Error org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is org.apache.tomcat.dbcp.dbcp.SQLNestedException: Cannot get a connection, pool exhausted
Associated KPIs on iControl Side :
“The number of watchlists not loaded successfully must be equal to the Agreed thresholds”
“All watchlists should be loaded in the System within the Agreed Threshold”
So Business is aware on the associated KPIs that are affected given the Long Running Job.
An Event Rule named CONNPOOL_EXTEND is defined which in turn would execute when the Load Watchlists Job goes beyond the Max Duration is running long which inturn executes the Recovery Action to Extend the Connection Pool to accommodate more connections:
Next CARRYOUT_SCREENING Job is executed to carryout the Screening at around 08:39pm and completes by 08:41pm IST.
This completes the Whole AML Chain within the SLA with every Step Tracked closely for Business KPIs as well as individual Steps within HWA.
Using a Solution comprised of HCL Workload Automation and HCL iControl , we are able to completely Automate each Step in the Anti Money Laundering Process while also tracking each Step in real time , incase of any Job Failures or Long Runners , we are able to readily track the Business KPIs in realtime that are affected and intimate the relevant Business Contacts/Owners directly through iControl while also Automating the Recovery in each Step of the Process through HWA via Event Rules or Conditional Dependencies.