OMS Transaction Framework - Nomenclature and more

After a long break following my first post - longer thanks to the distractions of the Euro Cup and Wimbledon - I am back with a little tidbit on nomenclature related to the Sterling transaction framework. If you have been stumped by whether a Sterling process is an agent server or integration server then this info would help you make the right call. 

For many years as I have worked with various customers, colleagues and partners I would hear people using various Sterling terms - agent server, integration server, transaction interchangably. Although the Sterling OMS world is not what it was in 2000 and as the lines are getting blurred as traditional "agent" processes are being implemented as services I figured I should tackle this topic in my blog. Earlier this week when one of my colleagues mentioned that this was a topic he too had explained for the n-th time to a new customer and pinged me looking for such a write-up I figured that it was time to put pen to paper or rather finger to keyboard. (For illustrations do refer the Sterling product documentation guides - ftp://public.dhe.ibm.com/software/commerce/doc/ssfs/85/Application_Platform_Configuration_Guide.pdf)

Grab a cup of your favorite beverage as this post does get a little long..

Transactions - In software parlance, a transaction usually means a sequence of information exchange and related work (such as database updating) that is treated as a unit for the purposes of satisfying a request while ensuring data integrity. Transactions may be synchronous such as those running in the UI or Asynchronous such as the batch jobs.  In the Sterling world the product extensibility and flexibility make the boundaries of a seemingly similar transaction vary from implementation to implementation even if the project teams and customers may call it the same such as Create Order Transaction or even dropping the transaction and referring to it as Create Order. In Sterling these transactions are defined either as an agent criteria or a Service via the Application Configurator. These transactions are executed in background JVMs known as agent servers or integration servers (also referred to as batch jobs) or directly from the Sterling UI (traditional console or thick client) or a Webservice call from external systems on the Application server JVM. Transactions consist of the underlying API and its associated events, user exits and conditions. A successful transaction results in the changes being committed that usually involves a combination of Database updates and messages being written and read from a queue or file. Either the entire transaction is successful or an error is thrown which causes the entire transaction to be rolled back or an error to be raised for subsequent reprocessing.  

In Sterling MCF we can classify processes into the following types of transactions :-

1. Time-Triggered transactions or Agents- which are triggered on a scheduled basis to perform repetitive actions.  Actions typically include invoking APIs to perform database updates. for e.g. consolidation of orders to shipments that may need to happen around 30 minutes apart so the Consolidate To Shipment time triggered transaction can be configured to trigger every 30 minutes. Most of the time triggered transactions are driven by records in YFS_TASK_Q table or based on the pipeline. Time triggered transactions are defined by the Transaction Name and the Agent Criteria.  They can be run in single or multi thread mode and are also called agents and the servers in which they run being called agent server.  Three types of Time-triggered transactions are :-

i. Business Process transactions - Responsible for majority of processing entities such as orders (sales/purchase/transfer) and shipments. The entities in every implementation will require one or more business process transactions such as CONSOLIDATE_TO_SHIPMENT, CLOSE_ORDER to complete their lifecycle. Understanding limitations of the Sterling transaction framework and designing for your business needs can help you get the most out of the solution.

ii. Purge transactions - Archive data from live (transaction) tables to history tables or delete that data that does not require archiving. Helps to mitigate unrestricted growth of the OMS  transactional database. Frequently underestimated in value and in development+testing efforts and overlooked in most implementations leading to application performance issues.

iii. Task Q Syncher Time-Triggered Transactions – A relatively new addition to the fold and is used to update the task queue repository table with the latest list of open tasks to be performed  by the corresponding each transaction, based on the latest pipeline configuration. 4 of these transactions are available - Load Execution, Order Fulfillment, Order Delivery and Order Negotiation

iv. Monitors – These are circumstance driven transactions that watch for processes or circumstances that are out of bounds and then raise alerts. Common monitors are those for Order, Shipment, Inventory Availability and Exceptions. Monitoring jobs can be a huge system hog if the data is not being purged often and if excessive stale entities exist such as abandoned or erroneous orders.

2. Services or Flows – Transactions that are executed NOT in pre-defined times are called services or flows. In the Database and Configuration screen titles this name is also used for every transaction in the SDF. Services can be invoked via use of broadly available transports - Web service/SOAP, HTTP, JMS, MSMQ, DB, flat file etc. A service can invoke other services to make a longer chain of services.  A service could include invoking APIs (product or custom), evaluating conditions, making DB updates etc. The services are processed continuously subject to thread and resource availability and are not triggered at any particular time. They can be run in single or multi thread mode and the servers in which they are executed are called integration servers. The most common scenario is the use of services to read messages from an inbound queue to Sterling for example to Create orders flowing in from a web channel.

3. Externally-Triggered Transactions - An externally-triggered transaction is used to map a service invoked to a Sterling transaction and to leverage the transaction framework. Seldom used in the real world as implementations prefer to just use a service/flow minus the transaction instead.

4. User-Triggered Transactions - A user-triggered transaction is invoked manually through the Application Consoles, a configured alert queue, or an e-mail service.  Never seen it used in the field.  So if you are implementing this or the externally-triggered do let me know how it goes.

Composite services – A construct to enable invocation of multiple services in parallel. A very useful concept ever since its addition to the SDF but needs careful testing as implementations could run into issues stemming from funky exception handling or inadequate logging.

Agent Criteria – An element that describes attributes that are specific to a time-triggered transaction. These attributes include the selector criteria such as Organization code, Manual or Auto triggered, trigger interval and server name. A particular transaction may have one or more agent criteria for processing data for different organizations or other logical grouping. E.g. Schedule Order agent criteria could be used to run scheduling for different organizations at different intervals.

Agent Server - Server JVM on which one or more agent criteria (commonly referred to as agents) can run. Invokes the com.yantra.integration.adapter.IntegrationAdapter class and is started typically by a startIntegrationServer.sh script provided as part of the product installation.

Integration Server - Server JVM on which one or more integration services or flows (commonly referred to as services or mistakenly called agents) are run. Invokes the com.yantra.integration.adapter.IntegrationAdapter class and is started by a startIntegrationServer.sh. 

Yes, you read it right! Both agents and integration services are started by the same class and script but the server name, service name or agent criteria name and definition controls the behavior.

Trigger agent - This is the process that is typically invoked via Cron or Ctrl-M jobs to trigger a certain time triggered transaction at certain points in time using the triggeragent.sh or triggeragent.cmd script.  For e.g. to Create Waves at certain hours of the day in a WMS implementation the trigger agent job could be invoked to trigger Create Wave agent or to run a nightly purge of sales order we could trigger the Order Purge agent.

Events – Help accomplish certain specific actions executed upon a certain business event occurring. For e.g ON_SUCCESS of Create Shipment we could have an event to send an e-mail to the customer with the shipment details or ON_BACKORDER of Schedule Order could used to raise Alert to the Inventory Control Business team. Event Handlers are configured to associate the required Actions to a particular event. Conditions are often used to further customize the action taken. Event handlers can invoke any service to e-mail, or raise exception alert;  Publish XML to external queues/database or Invoke custom services.  Actions associated are triggered any time the transaction is raised and when applicable so use it with caution. Excessive number of and complicated actions can prolong a transaction so use them wisely and tune them well.

User Exits – These enable transactions to invoke custom logic to interact with external systems synchronously to complete processing. A classic example is in the Payment Agent for credit card authorization. Frequently a source of issues when not implemented well and only care while designing and testing can avoid myriad issues post production.