Sunday, August 11, 2013

How can I run NServiceBus with SQL Server instead of RavenDB?

NServiceBus is reliable by default.  What does that mean?

Subscribers subscribe to the messages that are of interest to it with the publisher. Should the publisher remember who the subscribers are and for what message types even after the publisher process has been restarted?

Should Sagas continue to remember the state for all of the Saga instances after a process restart?

A saga requests a timeout or let's say a message is deferred, should the endpoint still receive the deferred message or the timeout message after the said time has past even after the process has been restarted?

A resounding Yes, Yes and Yes!

To accomplish this, NServiceBus uses RavenDB as its default persister, so your endpoints can be reliable from the get go.

But I don't want to use RavenDB, I want SQL Server - So...
It's easy peasy lemon squeezy. NServiceBus makes life easy to swap parts of the framework out! And swapping out default persistence is no different. Three simple steps and you are done!

1. Make sure that you've installed the nuget package NServiceBus.NHibernate. In your package manager console, type:
install-package NServiceBus.NHibernate -projectName <NameOfYourProject>

2. Add the following custom initialization
3. Specify the needed configuration for NHibernate in app.config, which by the way is super simplified in version 4.0

Now wasn't that easy? 

Wednesday, June 5, 2013

Why can't my machines talk nicely!? - Troubleshooting MSMQ, DTC and NServiceBus

You are already in the land of messaging, and have built your endpoints that use MSMQ as the transport of choice. Now comes the fun part, deployment!

You deploy your endpoints to your servers and give it a try.

The client is happy to send messages to the server and crickets on your server, or you get lovely transaction errors! And yes the machines are not playing nice with each other!

Here are some troubleshooting tips:

Can your machines ping each other?
Issue a ping from either machine to make sure that the machines can talk nice. Check by both IP address and name to make sure that name resolution works as expected.

Is your machine name too long?
For MSMQ to be able to send messages properly, the machine names must be 15 characters or less.

Do you have Windows firewall turned on?
If your firewall is turned on, ensure that Message Queueing and Distribution Transaction Coordinator are added to the list of Programs that are allowed in the firewall.

Are the ports configured correctly?
To verify: open ComponentServices. Expand Computers & right click on MyComputer → Properties. Click on Default Protocols tab. Click on Connection Oriented TCP/IP and then Properties and verify if the 5000-6000 port range is open. If not add the port range.

Is DTC configured properly?
To verify: In Component Services, expand My Computer and expand Distribution Transaction Coordinator. Check the properties on Local DTC to see if its configured like below:

Have you cloned your VMs? 
VMs are great. But if you have copied your VMs off of some image, then you need to know that there are some registry keys that cannot be the same for MSMQ to work properly.  Fore more details read this post.

Re-install DTC by following these steps:
  • Go to an administrative command prompt and type: msdtc -uninstall
  • Go to HKEY_CLASSES_ROOT/CID and delete any keys if they are present
  • Reboot the VM
  • Goto an administrative command prompt and type: 
                msdtc -install
                net start msdtc
  • If you had previously enabled Distribution Transaction Coordinator in firewall settings, you'll need to add it again to the allowed programs list. 
Generate a new QMId by following these steps:
MSMQ uses a unique Id for every machine. In order to generate a new identifier, follow these steps:
  • Stop MSMQ Service
  • Goto HKLM\Software\Microsoft\MSMQ\Parameters\MachineCache and delete the REG_BINARY QMId
  • Goto HKLM\Software\Microsoft\MSMQ\Parameters, add a DWORD called SysPrep and set it to 1
  • Restart MSMQ
For more details on the above read this post.

What does DTCPing tell you?
DTCPing is a great tool that can help troubleshoot MSDTC woes between servers. Download DTCPing and run DTCPing on both the server and the client and start the test.

If you receive a Access is Denied error on DTCPing:  make sure you add the following registry entries and try again.
  • Add a key called RPC under HKEY_LOCAL_MACHINE\ SOFTWARE\Policies\ Microsoft\Windows NT
  • Add a DWORD called RestrictRemoteClients under RPC and set the value to 0 
If you received a duplicate CID warning, follow the steps outlined above for cloning the VMs. Essentially re-install DTC and generate a new QMId.

For more information on the DTCPing errors, see here.

Using SQLServer and still having problems?
If running DTCPing on both machines produces the right results and you are still having trouble with transactions on the SQL Server, then try the DTCTester tool. The DTCTester establishes a connection to the SQL Server and creates a temporary table and enlists the connection in a transaction and does an insert on the temporary table and commits the transaction. It selects to read the inserted value and finally closes the connection.

For more details: Download DTCTester.

Hope these steps help in resolving your MSMQ, DTC, SQLServer woes!

Friday, February 15, 2013

A Journey from RunMeFirst.bat to the land of Powershell

A Brief History of Time!
During NServiceBus 2.x days, installing NServiceBus was done by installing runmefirst.bat /i. This basically checked if MSMQ was installed and if not, then it installed MSMQ and configured the Distribution Transaction Coordinator correctly and installed the CriticalTime performance counter. And who would have thought, installing an Enterprise Service Bus was as easy as installing runmefirst.bat!

Along with 3.0 came several bells and whistles and with RavenDB, a cool distributed document database that supports transactions as the default subscription and saga persistence store. Needless to say the infrastructure story needed a little bit more, MSMQ, DTC, RavenDB and the additional performance counters. With the addition of NSB Studio, a Visual Studio Extension to design endpoints, came the necessity to separate the SDK version. So the runmefirst.bat transformed into a MSI which installed the SDK, and the samples with the option to install the Studio Extension.

In order to install just the infrastructure on the servers for deployment, at some point in time, there was an additional switch added to the called the /installinfrastructure. So, when installing, you add the /installinfrastructure switch, it bootstraps the needed infrastructure and installs the endpoint.

Let's say we are deploying five services on the server, it does not make sense to call the /installinfrastructure switch everytime.  So in order to install infrastructure, the following steps were followed: /installinfrastructure (once on the server) followed by /install (with all of the other parameters for actually installing each host itself)

However this option did not have the modularity of installing just the needed components or a nice way of checking what's already installed.

Enter the land of Powershell - The Administrator's best friend:
NServiceBus introduced powershell commandlets to install the needed infrastructure in a modular way with options to check if the components are already installed.

You can find the official documentation here.

Deploying NServiceBus Infrastructure to Servers:
1. Copy NServiceBus.dll, NServiceBus.Core.dll, Log4Net.dll into a folder on the server (And when the dependency on Log4Net gets removed, there is bright light at the end of the tunnel, copying Log4Net wouldn't be necessary)
2. Open the all powerful powershell command prompt as an administrator and change directory to the correct folder and execute the following commands after executing, Import-Module .\NServiceBus.Core.dll
  • Install-MSMQ
  • Install-DTC
  • Install-RavenDB
  • Install-PerformanceCounters

Each of the above command can be used with a -whatif to tell you if these components are installed.

Some hiccups:
When trying to run the import-module if you see this ugly BadImageFormatException:

Then the we need to tell Powershell to load .Net 4.0 modules. This can be done by adding a config file called Powershell.exe.config and Powershell_ise.exe.config with the following contents to the C:\Windows\System32\WindowsPowerShell\v1.0 folder:

<?xml version="1.0"?>
 <startup useLegacyV2RuntimeActivationPolicy="true">
  <supportedRuntime version="v4.0.30319"/>
  <supportedRuntime version="v2.0.50727"/>
For more details on this, read Cornelius' blog post.

After adding the config files, restart powershell and don't forget to start in administrator mode, the module should be imported without any problems and installing infrastructure is as easy as 1, 2, 3, 4!

Last But Not Least:
The /installinfrastructure is going to be deprecated in 4.0
Embrace powershell.

Monday, February 4, 2013

Be Explicit -- Commands and Events in NServiceBus

In order for the systems we build to be reliable and scalable,  it usually involves queueing and messaging.  Having removed temporal coupling out of the equation, (Blocking synchronous RPC has its place, but just not everywhere) messaging is an important part of the puzzle and

  • How we classify these messages become important
  • How we communicate between "things" via these messages become important (using the right messaging pattern, ie. request-response, publish-subscribe, and the other 150 messaging patterns described in Gregor A Hophe and Bobby Woolf's Enterprise Integration Patterns book)
  • And not but least, naming these messages more important.

Types of Messages:
If we can classify messages, then we could classify them into Commands & Events as NServiceBus does.
There are those that fail:
Commands are messages that can be sent to one specific logical endpoint to fulfill a certain request.  The endpoint that handles this message has the ultimate authority to reject or fulfill this request. Therefore it makes sense to name these messages in the active tense, for Example, PlaceOrder
E = mc2:
And then there are those messages that convey a certain something happened that is of significant importance to the business. And are published as a statement of fact. These messages are classified as events. Since these messages convey a certain something happened, it is named in the passive past tense, i.e. OrderAccepted.  Since these have a significant importance, there are often multiple parties interested in when this happens. And since multiple parties are acting on this, as soon as it is broadcasted, they cannot be revoked. When you publish the OrderBilled, Shipping is going to ship the items to the customer. There is no oops, no, Order did not get billed without bad consequences.

Tiny business of naming:
When naming these messages, i.e these specific commands and events, let's for one brief moment, throw away all technical details out of the window and focus entirely on the business. Does business understand CreatePdfDocument? Or IPdfDocumentCreated? If the whole purpose of writing software is so that it adds business value, and the software to "align" with the business,  then the first step is to use the domain language, language understood by the business rather than use the language we techy geeks love. My seven year old has the right idea, geeks are cool people that solve problems and he kindly gave me a huge compliment. However domain experts in the business know the business and its needs. Techy people try to understand these requirements and come up with their own language and design to convey the same. Ummm, just why is it that we need an extra translation layer? Why not call a spade a spade? Why not describe our system in the language of the domain. Why is OrderPlaced, getting morphed to some type of IPdfDocumentSent? The blue book (aka Eric Evans' Domain Driven Design) talks about the importance of the Ubiquitous language.

The patterns of messaging and messages that are classified correctly and named right can easily fit into the business of software development and IT alignment!

Let's say that Orders can be placed via web, mail or phone.
Preferred customers get a certain discount
Customers are made preferred, if their sales volume exceeds a certain limit.

An extremely over simplified solution :
Order system's endpoint receives the PlaceOrder command (website exposed to customer, internal intranet site that customer service reps use will send the PlaceOrder command) and once it succeeds based on some criteria, publishes the event, "OrderAccepted"

Let's say the CRM service, listens to the OrderAccepted events. Knows which customer placed what order, and based on the order amount, can publish the CustomerStatusUpgraded event.

The Order service also listens to CustomerStatusUpgraded and CustomerStatusDowngraded events and based on that sets the discount appropriately.

In this model, the order service does not send commands to the CRM to GetCustomerStatus and waits for a response to compute the discount before it can publish the OrderAccepted event. A simple twist of how  pub/sub is used here by making the order service listen to the status changed event,   removed the need for request/response and made these two parts of the system autonomous. If or when the event is published, the ordering service will upgrade its discount percent. When orders are placed, it will always use what it has to compute the discount.

And a bigger benefit, when business says, "When a customer's status is upgraded, they get a complimentary fee waived one year Hertz Gold membership" with all that jazz of 3rd party integration with Hertz, all is needed is to add a an endpoint that subscribes to the event and to fulfill business' request. And the biggest benefit,  these new subscribers can be added without having to modify existing working system. And can be upgraded without having to impact other pieces of the puzzle as well.

Last but not least:
Starting from version 3.0, NServiceBus has removed the framework dependency on these messages assemblies and offers the unobtrusive conventions for declaring these messages. Define your commands and events as POCOs. For example:

namespace Business.Sales.Commands
    public class PlaceOrder
        public Guid OrderId { get; set; }
        public List<int> ItemsOrdered { get; set; }

namespace Business.Service.Events
    public class OrderPlaced
        public Guid OrderId { get; set; }
        public DateTime OrderPlacedTime { get; set; }

and when consuming them in the endpoints, initialize the conventions as Marçal describes here:

Thursday, November 29, 2012

Screw it, Let's do it!

I totally stole Richard Branson’s line for the captivating title. While Richard Branson was talking about something entirely different on his tweet, it just fit so right on with my zen experience yesterday at the Open Space forum at the Agile Island 2012 conference at Reykjavik that I shamelessly stole it.

It’s been a magical couple of days here in Iceland.  Here’s what I call a super awesome vacation. Yet another dream trip to Iceland and this time with an added bonus - the Agile Island 2012 conference. What more could I possibly want in a vacation, doing things that I absolutely love - Travel and technology!?

A day packed with goodness.  Jurgen Appelo’s keynote started the day making you think about enhancing the quality of work life and how getting away from a more traditional style of management into something that allows experimentation, and a more collaborative style can effectively affect the ordinary unhappy Melly’s who hate their job but do it anyway because they have to, into happy and productive Melly’s. Udi Dahan’s talk on balancing architecture and agile practices was keeping it real as always and how being "Agile" and architecture are both important and they help alleviate completely different risks, both important to address. While being agile, helps to focus on mitigating the risk of, “are we developing the right thing?” architecture focuses on building the software right with emphasis to reliability.  Also, it’s not a good thing if you can’t scale your system and you end up having to rewrite it! And other interesting talks like Culture Hacking by Petur Orri Saemundsen and Daoi Ingolfsson of Sprettur, and a whole lot more.

Eye Openers:
  • 70 % of the projects fail.
  • You need to balance good architecture and agile practices. You need both, it’s not one or the other as each of them are fundamentally managing a different risk.
  • Management is often the biggest hindrance to the adoption of being agile (aka collaborative) style of software development.
Jurgen’s statement that Management being the top 4 reasons for biggest hindrance to Agile / collaborative style of development adoption in an organization, brought up this burning question in my head:  “So then, how do we convince them!?”  If Management is the culprit then how can Developer(s) help change it? Another question that was brought up along the same lines was how does Culture Hacking come into play into the culture of the organization? The discussion was great, and what I learned even more inspiring. 

Two stories.  Vastly different.  Same results - Better practices, better ways
Story 1:
A senior member in the team was able to get buy in from the management to try the new Agile / Collaborative methodology in just one project as an experiment and let the team to implement the process that the team thought best.  After management consented, they team experimented. The team was now no longer just theorizing, but actually experimented this, and was able to show effective results. Needless to say, it was adopted and it was the thing.
Story 2:
The development and the testing team had great collaboration already. One of the developer(s) had brought up lean thinking and lean methodology as discussion to the group. The entire team really loved the idea. To the point where they started a book club and read about Lean Development. The thing is, "Management" was invited to their book club meetings but they were a no show. But the group continued with their book club and put practices into place. The D-Day came, and the team informed the management that they are going to work pronto this way, because it works and if they have any questions, they should read the book on lean development first.  Screw it, Let’s do it – In this case, a simple and yet an effective culture hack was born turning the organization more effective.

“To improve is to change, to be perfect is to change often”
Regardless of which story you draw your inspiration from, the hope is that, it is possible. Experimentation is a good thing.
When management realizes that having a whole lot of unhappy Mellys’, who just show up for their jobs to pay their mortgages is terrible and the more employees are involved, then good things happen!

My Takeaways:
  • The biggest takeaway of the open space discussion was Culture is bigger than management.
  • Management’s job is to help identify the vision and help facilitate the organization reach the goals and empower its people to achieve this effectively.
  • The more open, management is to experiments and collaboration, awesome things are waiting around the horizon!
Now onto the equally exciting travel part of the vacation!

Thursday, November 1, 2012

Why does it matter?

Writing good software is a thing of beauty and a thing of beauty is a joy forever.
What constitutes good software? Code that looks good? Three lines of code as opposed to lines and lines of code?

There was a time, I was blissfully unaware of a fundamental coupling in code called, "Temporal Coupling". In my mind, being Service Oriented was JBOWS. Just a bunch of WCF/Web services, that call each other to fulfill something. These services were grouped into Capability Services, Activity Services, and Entity Services. In my mind,  Services equaled WCF. And yes, though WCF offers a whole bunch of bindings. I had stuck to using Tcp. I was doing full fledged synchronous blocking request response without being aware of consequences. 

I was designing a thing of beauty. The software was going to be being oriented into these blocks of services!  Calling these services was simple too! As MSDN samples show, it was just  three simple steps:

//Step 1: Create an instance of the WCF proxy.
CalculatorClient client = new CalculatorClient();
// Step 2: Call the service operations. Call the Add service operation.
double value1 = 100.00D;
double value2 = 15.99D;
double result = client.Add(value1, value2);
//Step 3: Closing the client gracefully closes the connection and cleans up resources.

The first problem I ran into was timeout exceptions, and the problems didn't end there. Needless to say, it was far from beautiful and the devil is always in the details. Those three lines of code that looked beautiful hid a whole lot of things that I needed to make explicit in my design. 

What If?
What if the service was not up?
What if the client called the method and the service operation is executing and the operation timed out on the client side, but the operation completed on the server side?
What if this was a web service in a land far far away?
Do I need to retry the failed operation? 
Can I safely retry the operation?

My questions led to the first answers, led to the fact that these so called services were temporally coupled. What does that mean? Well, it means that one service cannot complete its function if the other service is not around to perform what it's asked to do. So, if Service A calls Service B, and Service B is down, well good luck to the million dollar order that is trying to be placed into the system. No Bueno! 

So, this whole idea of the activity service calling a capability service calling another service tumbled quickly into the abyss. More thought went into, how can these services be on their own without being dependent on each other to fulfill these capabilities and a design pattern that has been around forever emerged into sight. Event Driven Architecture and the Publish-Subscribe pattern.

Enter a different way of thinking
Well the real world works on events. A low battery event on your computer makes you react and plug in the power source when the event occurs. You react to events and perform certain actions when they occur. And yet why is that we write code on a command-control basis? Our programs mostly tend to, call method, perform action, retrieve more information, do this do that as if nothing else can go around these actions. 

What if we turned that around and made software react to events? An order placed event. The billing system can react to it and charge the customer and publish CustomerCharged event. The shipping system can then react to this event and ship the item. There is no service calling service, instead a loosely coupled model where services can be truly autonomous emerges.

Software "talking" to software via messages to convey a certain something happened. These messages being durable, i.e. in case of something bad happens, they can be still around, so when the process is restarted, the service is aware of "I still need to process this message". Messaging makes things explicit of all the things, as a software person I need to consider and account for, exactly what those three steps of code was masking.

How can I build this awesome pub/sub as the backbone for the software I write?
Juval Lowy's WCF book has a whole appendix worth 33 pages that describes, how you can build your own pub/sub system using WCF. All I can say is,  May the force be with you. To start anything is easy, but to build it and build it right takes time. And the pressure is on you to make sure that this works well, and you've taken care of every teeny tiny detail, since this will be a fundamental piece of the system. The last thing you want is events gone AWOL and the CEO watching you while you are trying to frantically debug your system! Talk about pressure!

There is a much easier way!
When I was on this quest to find a framework that can help me build software in a loosely coupled, event driven manner, my path crossed  with NServiceBus.

NServiceBus is a light weight framework has been around for a long while and offers a robust and a relatively simple way of using pub/sub in your software. You can have pub/sub in 5 minutes, no kidding.  I've haven't looked back since and I don't miss WCF much! My experience with NServiceBus has been fantastic. I could now focus on my business problem and not worry about the intricacy of how the underlying messages get transported and the infrastructure aspect. IJW (It just works!) and yes, Atlantis does exist! 

When you leave behind  RPC request response blocking calls and start thinking in terms of event driven programming and how messaging and good architecture makes your software more reliable and autonomous, I say, you've arrived in the land of beautiful software and yes, Keats was right. A thing of beauty is a joy forever.