Living in the Tech Avalanche Generation

On the back of running his ADSD course in Sydney this year Udi Dahan will be also running a one day NServiceBus introductory course. The one day course is a highly condensed event so its sure to pack a wallop and if you are interested in taking NServiceBus for a spin and see what all the fuss is about then I suggest you quickly book yourself a place.

Mark and I will also be sharing our experiences of how NServiceBus has served us as an enabler in designing and delivering successful solutions for our clients as an ISV and as consultants.

This (now series of) posts(s) will be something of a stream of consciousness, an exploration that has no single objective, rather a journey of discovery with no guarantee (sought nor otherwise) of any practical application. Something tells me that there is some niche lurking behind the premise however beyond my initial instinct I am prepared to let any hidden benefits (if there are any) reveal themselves to me along the way.

A while back I posted on how it was possible to self host an NServiceBus endpoint in a custom built IronPython / WPF REPL process. Since that time NServiceBus has experienced several feature enhancements and now supports a rich hosting model of its own, the Generic Host. I wont regurgitate what has already been written about the Generic Host as its been covered in several places and if you wont to know more about it I suggest you look at these links:

The Generic Host

The Profiles

Sample

So Why an IronPython Host

In short, the ability to throw ad-hoc messages at your NServiceBus ecosystem and observe your systems behaviour (eye to eye) and some dynamic debugging! The typical debugging scenario has me sitting with four or more console windows open acting as either clients, servers and or publishers and some of those windows will be waiting for input (from me), the kind of input you typically see in a console application “Press y to send a message” (usually a fairly statically crafted and compiled one). When speaking about clients and subscribers sometimes (going into production) that just might be a windows desktop application.

In the world of dynamic languages developers have come to expect the ability to test / play with / mess with / debug their code in a Repeat Evaluate Print Loop which has some significant benefits.

Start-up

Using the NServiceBus Full Duplex sample I added a reference to my hacked version of the Generic Host and set the project to start-up the executable as per the usual approach (when you have chosen not to self host in your own executable).

After starting the Server project the console fires up as expected and is then followed by the IronPython console where the hosted Bus (instance) has been made available to the script scope context we have started in.

Next I proceed to hack out a message and send it on to the Bus. What isn’t clear below is that Message1 is actually an already dialled up instance of a RequestDataMessage as defined in the message project that is part of the Full Duplex sample that comes in the NServiceBus download.

Here is the Message Handler (in the Server project) for the RequestDataMessage (in the script above Message1 is an instance of this type.

Once the script above gets executed then the REPL reports back to us (I have hijacked the STD/IO and redirected it to the REPL’s TextWriter.

Ok, so it seems obvious enough that the Generic Host is now starting up in the context of our Python REPL and I can adhoc hand craft messages and observe the behaviour of the hosted endpoint, but there are some still some issues, one immediate and obvious one is that the REPL Host is doing two things or rather taking on two independent roles namely the Client and Server. If you run the Full Duplex sample you will note that there are two processes in the solution, a client configured AsA_Client and another configured AsA_Server; when I run the Server project using the Generic Host Shell I am freeing the console window and instead running the REPL window in its place. What this means quite literally is that in this instance, the REPL Host is allowing me to send messages to the Server Endpoint (which it too is hosting), without the Client endpoint running at all, thereby effectively spoofing some of the behaviour found in the Client process (project) without actually loading the client endpoint at all.

Alternatively I can run both projects (Client and Server) as I normally would, with the exception that they are both being hosted in the IronPython host. This (arguably) offers more fidelity with respect to the two services operating entirely within their own process which is in line with a production deployment.

What the image above demonstrates is that we now have two REPL Hosts running simultaneously, one endpoint configured AsA_Client and the other AsA_Server endpoint.

I am going to leave part 1 here for now and end with an idea that has come to mind: perhaps it would be useful to run a solution (furnished with hosted endpoints) and have all the REPL hosts start up within a container, perhaps an MDI host where all the REPL windows tile / cascade? It also occurs that perhaps such this REPL host would be useful for inspecting and returning source messages to their queues or performing functions like those found in the Grid UI or perhaps even something akin to an integrated shell where an Administrator might walk up to a machine hosting NServiceBus endpoints and issue those endpoints with administrative messages? Anyway I will meditate on it for a while and perhaps stay open for some outside input.

In the next post I will talk about some of the issues facing me currently and should be able to include a link to the working example by then. In the meantime you can find the old post here and download the REPL as it was back then.

Finally, the prototype IronPython console I built some time ago is really just that, a prototype and given that I plan to use it as a live runtime tool embedded in my commercial products (at some point in time) then its really time to take it from prototype to working draft and that means its time to get serious about styling the window beyond battleship grey.

After having recently sat through Udi’s 5 day course on Advanced Distributed  Systems Design (ADSD) and after several days of quiet introspection punctuated with some random reconnecting to the web full of it’s opinion and ideas, I found myself ever more at peace with the challenges that lie ahead. During a conversation with Udi whilst he was here in Melbourne recently, he recalled a twitter comment made by my colleague Mark wherein he quoted my flippant remark that having NServiceBus at my disposal could be likened to being armed with nuclear weapons. Perhaps it wasn’t the most elegant analogy but I think it conveys the sentiment, I feel able to leverage and harness a great deal energy and power. This sentiment can also be attributed to the many great ideas that you find yourself being exposed to in the ADSD course

In these times of rampant software development technology proliferation, it would be easy to feel overwhelmed and I know from speaking with colleagues that many do, yet somehow I feel more at peace than ever. Technology stacks, frameworks, buzzwords, paradigm shifts, they are all hurtling towards us at a pace most of us have never known and the inertia alone is staggering.

Many of us need help in separating the stuff from the fluff. As Neo said to the architect, “the question is choice”! Well, is it? What are the choices? Do technology choices alone get your system built? No of course they don’t but many have fallen into the trap of behaving as though they do. Ever caught yourself saying something like “oh if I just use MVVM or ASP.NET MVC, my system will be perfect because of benefits X, Y and Z”? Other developers occasionally ask me, “what technology should I learn next” or “how do I future proof myself”? I think they are somewhat surprised by my answer; my advice is to stay as far away as possible from attempting to build value as a vendor stack expert and focus attention on maturing and refining your design and communication skills. It seems obvious really, after all once you have well developed your design and architectural skills, then you are far better placed to make appropriate technology choices. The ensuing  implementation experience will sort out any technology knowledge gaps that you feel the job’s market is causing you to feel downward pressure from. This doesn’t mean ignore technology all together, by all means pull it apart and analyze it, see how it fits but not in isolation – you need to give it it’s context. This became quite plain to me after having been through the course two years ago.

The DDD / SOA course was a lynch pin for me it was the key that has opened many doors. Without the learning in principles first, no technology will stop you from making a mess of things. Again it’s pretty obvious when you say it but I am sure I am not alone as someone who came to the course with a whole set of assumptions based on layered designs and technology fads as a bag of best practice tools and tricks.

I have now had the privilege of sitting this course three times and it never get’s old. Since I first experienced it two years ago, the content has changed a little and become even more refined. Each time I sit through the course I feel more empowered than the last time, the lessons are learned more deeply and start to become part of my psyche.

Some of my fellow attendees have posted their thoughts upon having completed the course just recently in Melbourne and I suggest you consider their feedback as well. If you haven’t heard there is now a mailing list dedicated to discussing the courses content and if you are planning on attending in London or the US sometime this year you should definitely check out the mailing list.

I would like to leave you with this; if you have not done this course and you have even the vaguest inkling that you might stand to gain something from learning what what some of the best architects in world know but rarely share in great detail, then do whatever it takes to get yourself a seat when next ADSD rolls into town or find a way of bringing Udi and the course to you.