Reliability Patterns with Active MQ & Mule

Applications which use a transactional transport such as JMS, VM or JDBC are guaranteed that messages will not be lost during the course of processing. Transactional support in these transports ensures that messages are delivered reliably from an inbound endpoint to an outbound endpoint or between processors in a flow. If however you have a web application accessing your flow via a non-transactional transport such as HTTP you can ensure reliable messaging in your flow by implementing a reliability pattern. Reliability patterns couple a reliable acquisition flow with an application logic flow. A reliable acquisition flow can be combined with a series of asynchronous flows to decompose applications into decoupled, reliable segments. Applications which access multiple remote systems as part of their processing which may take a variable amount of time to respond and maybe unreliable are good candidates for this architecture.

This blog discusses this approach using a fictions example which accepts contact information over an inbound HTTP endpoint which needs to be processed by a web service and then synchronizes the contact information with multiple backend systems such as Microsoft Dynamics CRM and Salesforce. These are systems you may have no control over but we’d like to respond to the calling application as soon as possible with a confirmation that the contact information was received and is being processed. This is done by accepting the request over HTTP, generating a contact ID and transaction ally submitting this to a JMS queue for processing.  For purposes of this example we’ll be using Active MQ as our messaging broker. 

The first step is the front end data submission - the contact information is submitted in the form of a SOAP request to an HTTP inbound endpoint and generates a unique contact ID which can then be returned to the calling application for tracking. 


The contact information is transactionally transmitted to a JMS outbound endpoint which then sends it to another queue for backend processing.

<async doc:name="Async">
<jms:outbound-endpoint queue="contact.submit" connector-ref="Active_MQ" doc:name="JMS">
<jms:client-ack-transaction action="ALWAYS_BEGIN"/>
</jms:outbound-endpoint>
</async> 



This flow accepts the contact information and submits it to two separate branches of a scatter-gather router which concurrently sends this information to Microsoft Dynamic CRM and Salesforce endpoint. 

<jms:inbound-endpoint queue="contact.submit" connector-ref="Active_MQ" doc:name="JMS">
         <jms:transaction action="ALWAYS_BEGIN"/>
</jms:inbound-endpoint>

If at any point during the processing the application crashes, message is redelivered to the endpoint and processing continues. In the case of transactional rollback, a rollback exception strategy is used to redeliver the messages a pre-configured number of times. If the message cannot be processed, it will be sent to a DLQ. One thing to note is that this approach may end up causing a duplicate record to be created for the contact so it may be prudent to have an extra step to lookup the contact in the CRM as well as Salesforce before persisting it. 

<jms:outbound-endpoint queue="contact.complete" connector-ref="Active_MQ" doc:name="JMS">
         <jms:transaction action="JOIN_IF_POSSIBLE"/> 
</jms:outbound-endpoint>

Finally, once the contact has been saved, a message is sent on a queue to indicate completion of the transaction as shown in the following flow.

<jms:inbound-endpoint queue="contact.complete" connector-ref="Active_MQ" doc:name="JMS">
          <jms:transaction action="BEGIN_OR_JOIN"/>
</jms:inbound-endpoint>


Comments

Post a Comment

Popular posts from this blog

Anypoint MQ access using POSTMAN

Image
Mulesoft's Cloud Based message offering Anypoint MQ provides you APIs which enable REST based access to send and receive messages, administer queues and message exchanges, and analyze results. This blog goes over the usage of these API's when running POSTMAN on Windows. Details around usage with CURL are already available on the Mulesoft Website. With the Client ID and Client Secret handy, do a POST to the following endpoint - make sure you are passing the following parameters as x-www-form-urlencoded format to the endpoint https://mq-us-east-1.anypoint.mulesoft.com/api/v1/authorize client_id - this the client ID for your client app client_secret - this the client secret for your client app client_id - this the client ID for your destination grant_type - client_credentials Your Content-Type will be application/x-www-form-urlencoded ( surprise, surprise :) If everything above is correct, you will get back a JSON response similar to the following { "
Read more

Legacy to API Led Connectivity with Mulesoft

When building out green field migrations from existing legacy code to Mule soft, a bottom up approach is recommended for creating new integration applications. An ESB such as Mule ESB is an important enabler in this type of a migration facilitating the creation and orchestration of services without the need for an application server or other infrastructure components. Given that, migrations to this type of architecture is a two-step process – the first step involves creation of endpoints to expose business logic as services wrapped inside Mule flows. The next step is to create service compositions that represent the actual business processes. These steps lay the foundation for a ‘System API’ layer which connects to the various SOR’s hosting the most important asset of an organization – namely the data – enabling access from well-defined API’s. While building out these integrations, developers need to ensure some foundational design principles are followed. Separate implementatio
Read more

Publish – Subscribe Messaging with Anypoint MQ

Image
Well, after three years of working almost exclusively on the Mulesoft Anypoint Platform, solving critical business problems for both on-premise as well as cloud based customers I’ve finally decided to start my own blog. As part of an ongoing client engagement, I got the opportunity to work with Mulesoft’s cloud based messaging solution, AnypointMQ – during the project we implemented point to point messaging for the most part but the publish-subscribe messaging is a powerful paradigm to connect distributed systems so I thought it would be a good idea to explore it as part of this blog. Anypoint MQ is a multi-tenant, cloud based messaging service which allows integrations deployed on Mulesoft’s Cloud Hub environment to talk to each other using either point to point or publish-subscribe messaging. Other alternatives for cloud based messaging include the Azure Service Bus and Amazon SQS messaging service both of which you can integrate seamlessly with your Mulesoft applications. Whil
Read more