Breaking free from on-premises

Two three years ago now (wow, time flies!), I was investigating how we could transition from our on-premises message based architecture to having services running in Azure. This article briefs you on parts of our journey thus far.
In this post

Background

On premises, we were running NServiceBus with its default MSMQ transport. We were relying on DTC to ensure that we had transactional guarantees for message delivery, and we had grown comfortable with this support. As such, we were looking for a solution where we could extend our current mesh of on-premises services into the cloud, having them work together for an extended period of time, until we finally had moved them all.


Also, due to dependent software components, for a given set of services, we were tied down to an older version of NServiceBus (4), whereas we - in the cloud (and for our new services) - wanted to use the latest stable version we could get our hands on, which at the time was NServiceBus 6.

We tried various approaches and one of these were to introduce lightweight components that could participate in any transaction and bridge our calls from on-premises to the cloud and back again. These were lightweight in the sense that they were templated projects without business logic, whose only purpose was to transport the messages. Thus, we could rely on them being stable, once we had gotten the configuration right - i.e. verifying that messages were written as expected. Since we could communicate from our premises to the cloud, we decided to place the bridge components on premises as well:


Okay, that's a lot of boxes and arrows! Let's break it down a bit by looking at our messaging scenarios:

Extending our first on-premise service to the cloud - a scenario

The team responsible for our first on-premise service (really - the underlying business capabilities, which are encapsulated by a number of messaging services, web apps and more), wants to add new functionality to their solution. This added functionality fits well in the cloud, why they create *Cloud Service 1*. In order to communicate with this service - and maintain the transactional guarantees they had before - they send their command (or event) to the *On-premise Service 1 to Cloud Service 1* (OPS1-To-CS1) message bridge, which they also maintain as part of their solution. The message gets written to the services' outgoing queue (courtesy of NServiceBus and MSMQ) and will be rolled back if subsequent messages / actions fails within the same transaction.
All the message bridge is doing, is to take the message and forward it to the target service, which it does by reading the message from the queue (again, courtesy of NServiceBus) and writing it to Azure Service Bus. Here, we minimize risk of message loss by letting the bridge has exactly one responsibility, minimizing the risk that we introduce code in the future that would interfere with message delivery. We also rely on Azure Service Bus' Message Deduplication feature, ensuring that we - in the case of errors concerning double delivery, only ever let our target service see one message. All in all, we accomplish once and only once delivery, which we are so accustomed to when running in our own datacenter.

Finally, the Cloud Service 1 reads the message from Azure Service Bus, using the NServiceBus 6 library.
Want the nitty-gritty? Check out our published code.

Reference material

Comments

Post a Comment

Popular posts from this blog

Auto Mapper and Record Types - will they blend?

Image
TL;DR: Yes, they will. Thank you for your time. 😄 Background At Redgate, engineers enjoy a great benefit in that we get are expected to spend 10% of our time to improve on our craft.  Having suffered this week with manually having to copy a bunch of properties from a domain object to its HTTP representation, I decided to spend this week's 10% time to see if I could answer the following questions: Can use AutoMapper to simplify mapping between two types?  Does it also allow me to map between a regular type and a record type? What about the other way around? Can I use this to reduce the maintenance burden in my product? Mapping between two types [automapper fundamentals] To started and get a basic mapping done, created a .NET 5 ASP.NET Web API project, added the below package references and configured automapper : < PackageReference Include ="AutoMapper" Version ="10.1.1" /> < PackageReference Include ="AutoMapper.Extensions.Microsoft.DependencyInj
Read more

Unit testing your Azure functions - part 2: Queues and Blobs

In our last installment , we discovered how to start unit testing our Azure Functions, looking at the HTTP Trigger (and evaulating the function's HTTP response). This time, we'll take a look at using the QueueTrigger, getting supporting data from table storage and outputting Blobs. Yup, I'm tossing you into the deep end! What are we trying to accomplish? This time, we will test a function that performs some arbitrary business logic on bank accounts. The function responds to a storage queue request, logs each command and ensures that each received command is executed exactly once. Almost like a real-world function, huh? ;-) In the function above, we are introduced to three types that we need to deal with in our unit tests - the IQueryable interface, IAsyncCollector and the CloudBlockBlob concrete type. IQueryable is easy enough to work with: Introducing the Testable Async Collector When I first started exploring this topic, I figured I could just use my favourit
Read more

Testing WCF services with user credentials and binary endpoints

Image
This article discusses the following error messages that you might receive from a WCF service An error occurred when verifying security for the message HTTP/1.1 415 Cannot process the message because the content type 'application/soap+xml;charset=UTF-8;action="..."' was not the expected type 'application/soap+msbin1' When doing analysis work on existing services, invoking them in a safe test environment is a handy way of getting an understanding of their operations, especially if the documentation has gone missing (or was never written). SoapUI is a tool that can be used to test WCF Services, but did you know that Visual Studio also comes with its own set of tools? Let's start with SoapUI, which is available from  http://www.soapui.org/ . Its operations are quite straight-forward; however, you might run into issues when trying to access services that are protected by user credentials, or when accessing services with binary endpoints. I
Read more