Living in the Tech Avalanche Generation

Many ERP / CRM products are coming to market spouting Web Services as their customisation and integration saviour. These products are exposing their innards to the world via a SOAP Web Service layers, a design which is often tightly coupled to their more familiar units of extensibility or API’s. What does that mean? As an example, NAV 2009 will allow you to expose known development artefacts such as ‘Pages’, ‘Code Units’ etc as SOAP Endpoints (Web Services) and the Sage Accpac products also encourages integration via a web service layer.

So, Integration…..yes I guess so….. and custom solutions interfacing with a given system, well yes a bit of that too, but scalable, that I am not too sure about. One way of making such systems more scalable is to introduce a bit of publish and subscribe, some good old HTTP and a little resource oriented Architecture. By simply monitoring events in the system and by publishing those events and their data we can do interesting things.

Building Around the Edges

It’s a popular approach to customise an alternative User Interface for OOTB ERP systems. There are many simple things you can do to make your custom system more scalable - for instance, when fetching semi static data you might think about publishing it as a HTTP resources and putting REST to work, the net effect of which should greatly reduce  the all too common syndrome we have observed in systems that make constant and arguably redundant request / response calls to the systems API and consequently over working it’s database system. This approach was detailed very elegantly a while back by Udi.

As percentage, the number of request messages (request / response) made when there has been little or no change to semi static data is often alarmingly high. Nonetheless, it is commonplace to see applications employ this ‘ask and wait’ routine, which places a cascade of over utilisation across all the systems involved in the call chain. By publishing semi static data when events occur you can save precious resources and gain significant scale.

Udi recently reminded me that NServiceBus provides another way of approaching this problem and of course as usual he’s right. If developers can get their heads around PUB / SUB and the fundamental shift in approach it requires without getting sidetracked by everything that they have done in the past, then this can be a very satisfying way of working. The outcome is that we can build a system that customises user experience with OOTB CRM or ERP systems, without placing the same kind of request / response stresses that we would usually create if we choose to use the vendors Web Service option.

It has just recently come to my attention that Udi will soon be running his advanced distributed systems design course in Austin Texas with Jeffrey Palermo’s crew at Headspring. I was fortunate enough to take this very course earlier this year in Australia and I can truly say that my thinking, planning, designing and output have all been enhanced in the most profound ways since doing so, it was one of those great eye opening experiences that can move you to another plane of thinking.

Judging from the course outline on Udi’s site, it would appear that some new material has found it’s way onto the syllabus (I’m jealous), which covers Ultra High scaling possibilities by leveraging REST and after hearing Udi’s podcast on that topic I think I have a feel for what that promises and can only say that I wish I could make the trip to the US to be there.

Whilst I gained enormously from the SOA and DDD content, I also took away some wonderful stuff from the ‘Smart Client’ sections of the training. My team have been using our own home grown MVP framework for some time now and Udi certainly expanded our thinking there, particularly with regard to multi threading issues - greatly useful stuff.

When I had the privilege of sitting through this training I did so with my team and cant say enough about how it invigorated, improved and turbo charged the mind-set and output of all who attended. Udi is a very skilful presenter and a wonderful teacher and significantly has amassed some great wisdom that anyone who is serious about building Service Oriented systems should go out of their way to experience and imbibe. If your in Austin Texas or anywhere in the United States for that matter, I thoroughly recommend you take this course, it is worth far more than the advertised price. Prepare to have some of your beliefs challenged and come with a spirit of learning and you will enjoy the rewards - I know I have and so have many of my colleagues. I know that I personally often find enormous value in listening and learning from people of the quality of Udi and Randy, so imagine spending a week in the company of one such teacher!!!

With PDC around the corner and the buzz around Oslo beginning to form into some louder noise, it occurs to me that we (developers and architects) should be helping shape the discussion and hopefully make an impression.

If I could hand the Oslo team my single most precious wish it would read like this:

“Can I please have a framework that allows me to build applications that support durable messaging using the publish & subscribe pattern and communicate over a bus!”

That’s the main thing that I crave. In the meantime we can use nServiceBus, SimpleServiceBus, Mass Transit and a few others, including rolling our own with WCF but lets not forget that WCF does not give us this out of the box, in fact far from it.

To date we have heard about how integral Windows Communication Foundation and Workflow Foundation are to Oslo and that XAML activation looms large along with ‘the Repository‘ which might be storage for workflow and service discovery. It would seem that the idea of the ‘Repository’ is about assisting with architecture, but I’m not yet clear about how deep the Repository lives in the API and whether I can leverage it to promote or provide durable pub /sub messaging with a bus architectural pattern. Let’s wait and see what emacs.net, the repository and the so-called new process server bring to the table.

Parts 1 and 2 of this post.