After a long break following my first post - longer thanks to the distractions of the Euro Cup and Wimbledon - I am back with a little tidbit on nomenclature related to the Sterling transaction framework. If you have been stumped by whether a Sterling process is an agent server or integration server then this info would help you make the right call.
For many years as I have
worked with various customers, colleagues and partners I would hear people
using various Sterling terms - agent server, integration
server, transaction interchangably. Although the Sterling OMS world is not what
it was in 2000 and as the lines are getting blurred as traditional
"agent" processes are being implemented as services I figured I should
tackle this topic in my blog. Earlier this week when one of my colleagues mentioned that this was a topic he too had explained for the n-th time to a new
customer and pinged me looking for such a write-up I figured that it was
time to put pen to paper or rather finger to keyboard. (For illustrations
do refer the Sterling product documentation guides - ftp://public.dhe.ibm.com/software/commerce/doc/ssfs/85/Application_Platform_Configuration_Guide.pdf)
Grab a cup of your favorite beverage as this post does get a little long..
Transactions - In software parlance, a transaction usually
means a sequence of information exchange and related work (such as database
updating) that is treated as a unit for the purposes of satisfying a request
while ensuring data integrity. Transactions may be synchronous such as those
running in the UI or Asynchronous such as the batch jobs. In the Sterling world the product
extensibility and flexibility make the boundaries of a seemingly similar
transaction vary from implementation to implementation even if the project
teams and customers may call it the same such as Create Order Transaction or
even dropping the transaction and referring to it as Create Order. In Sterling
these transactions are defined either as an agent criteria or a Service via the
Application Configurator. These transactions are executed in background JVMs
known as agent servers or integration servers (also referred to as batch jobs)
or directly from the Sterling UI (traditional console or thick client) or a
Webservice call from external systems on the Application server JVM.
Transactions consist of the underlying API and its associated events, user
exits and conditions. A successful transaction results in the changes being
committed that usually involves a combination of Database updates and messages
being written and read from a queue or file. Either the entire transaction is
successful or an error is thrown which causes the entire transaction to be
rolled back or an error to be raised for subsequent reprocessing.
In Sterling MCF we can
classify processes into the following types of transactions :-
1. Time-Triggered transactions or Agents- which are triggered on a scheduled
basis to perform repetitive actions. Actions typically include invoking APIs to perform database
updates. for e.g. consolidation of orders to shipments that may need to happen
around 30 minutes apart so the Consolidate To Shipment time triggered
transaction can be configured to trigger every 30 minutes. Most of the time
triggered transactions are driven by records in YFS_TASK_Q table or based on
the pipeline. Time triggered transactions are defined by the Transaction Name
and the Agent Criteria. They can be run in
single or multi thread mode and are also called agents and the servers in which
they run being called agent server. Three types of Time-triggered transactions are :-
i. Business Process transactions - Responsible for majority of processing
entities such as orders (sales/purchase/transfer) and shipments. The entities
in every implementation will require one or more business process transactions
such as CONSOLIDATE_TO_SHIPMENT, CLOSE_ORDER to complete their lifecycle.
Understanding limitations of the Sterling transaction framework and designing
for your business needs can help you get the most out of the solution.
ii. Purge transactions - Archive data from live (transaction) tables to
history tables or delete that data that does not require archiving. Helps to
mitigate unrestricted growth of the OMS transactional database. Frequently underestimated in value
and in development+testing efforts and overlooked in most implementations
leading to application performance issues.
iii. Task Q Syncher Time-Triggered Transactions – A relatively new addition to the fold and is
used to update the task queue repository table with the latest list of open
tasks to be performed by the
corresponding each transaction, based on the latest pipeline configuration. 4
of these transactions are available - Load Execution, Order Fulfillment, Order
Delivery and Order Negotiation
iv. Monitors – These are circumstance
driven transactions that watch for processes or circumstances that are out
of bounds and then raise alerts. Common monitors are those for Order, Shipment,
Inventory Availability and Exceptions. Monitoring jobs can be a huge system hog
if the data is not being purged often and if excessive stale entities exist
such as abandoned or erroneous orders.
2. Services or Flows – Transactions that are executed NOT in
pre-defined times are called services or flows. In the Database and Configuration
screen titles this name is also used for every transaction in the SDF. Services
can be invoked via use of broadly available transports - Web service/SOAP,
HTTP, JMS, MSMQ, DB, flat file etc. A service can invoke other services to make
a longer chain of services. A service could
include invoking APIs (product or custom), evaluating conditions, making DB
updates etc. The services are processed continuously subject to thread and
resource availability and are not triggered at any particular time. They can be
run in single or multi thread mode and the servers in which they are executed
are called integration servers. The most common scenario is the use of services
to read messages from an inbound queue to Sterling for example to Create orders
flowing in from a web channel.
3. Externally-Triggered
Transactions - An externally-triggered
transaction is used to map a service invoked to a Sterling transaction and to
leverage the transaction framework. Seldom used in the real world as
implementations prefer to just use a service/flow minus the transaction
instead.
4. User-Triggered
Transactions - A user-triggered
transaction is invoked manually through the Application Consoles, a configured
alert queue, or an e-mail service. Never seen it used in the field. So if you are implementing this or the
externally-triggered do let me know how it goes.
Composite
services – A
construct to enable invocation of multiple services in parallel. A very useful
concept ever since its addition to the SDF but needs careful testing as
implementations could run into issues stemming from funky exception handling or
inadequate logging.
Agent Criteria – An element that describes attributes that are
specific to a time-triggered transaction.
These attributes include the selector criteria such as Organization code,
Manual or Auto triggered, trigger interval and server name. A particular transaction
may have one or more agent criteria for processing data for different
organizations or other logical grouping. E.g. Schedule Order agent criteria
could be used to run scheduling for different organizations at different
intervals.
Agent Server - Server JVM on which one or more agent criteria
(commonly referred to as agents) can run. Invokes the
com.yantra.integration.adapter.IntegrationAdapter class and is started
typically by a startIntegrationServer.sh script provided as part of the product
installation.
Integration Server - Server JVM on which one or more integration
services or flows (commonly referred to as services or mistakenly called
agents) are run. Invokes the com.yantra.integration.adapter.IntegrationAdapter
class and is started by a startIntegrationServer.sh.
Yes, you read it right!
Both agents and integration services are started by the same class and script
but the server name, service name or agent criteria name and definition
controls the behavior.
Trigger agent - This is the process that is typically invoked
via Cron or Ctrl-M jobs to trigger a certain time triggered transaction at
certain points in time using the triggeragent.sh or triggeragent.cmd script. For e.g. to Create Waves at certain hours
of the day in a WMS implementation the trigger agent job could be invoked to
trigger Create Wave agent or to run a nightly purge of sales order we could
trigger the Order Purge agent.
Events – Help accomplish certain specific actions
executed upon a certain business event occurring. For e.g ON_SUCCESS of Create
Shipment we could have an event to send an e-mail to the customer with the
shipment details or ON_BACKORDER of Schedule Order could used to raise Alert to
the Inventory Control Business team. Event Handlers are configured to
associate the required Actions to a particular event. Conditions are often used
to further customize the action taken. Event handlers can invoke any service to
e-mail, or raise exception alert; Publish XML to external queues/database or Invoke custom services. Actions associated are triggered any time
the transaction is raised and when applicable so use it with caution. Excessive
number of and complicated actions can prolong a transaction so use them wisely
and tune them well.
User Exits – These enable transactions to invoke custom
logic to interact with external systems synchronously to complete processing. A
classic example is in the Payment Agent for credit card authorization.
Frequently a source of issues when not implemented well and only care while
designing and testing can avoid myriad issues post production.
