Living in the Tech Avalanche Generation

Well it hasn’t taken long to get some excellent bang for buck from IronPython. The entire team with whom I currently work are all furiously learning Python courtesy of an almost Googlesque free learning time and I have already documented my learning path with respect to Python here before.

As I said at the top of this post, we are already reaping some benefit and there is certainly not an Python guru amongst us (yet). Most recently we were asked for a fairly trivial application (as far as complexity goes) but one that was very valuable to the business nonetheless. Without going into the gory details, what was requested was a desktop application that digitally processed documents (imaging) and produced the output fast, and I mean really fast. The last part of the request was for a design that would allow the document list to be compiled from almost any source, database, xml, Microsoft Office files, txt, etc etc. Let’s deal with each of these concerns in order of their being mentioned.

Steroidal Processing Speed

This part was really not that hard to design and the choice was a simple one, in one word, NServiceBus. What was needed essentially was many workers all doing the same work on a given set of files and this of course mapped brilliantly to one way messaging using NServiceBus and it’s distributor to manage balancing the load amongst the workers. By spreading out the work across any number of worker nodes, our project managers could scale up and down according to their needs to adhere to a given SLA. If you want to read more about the Distributor in NServiceBus please check out Ayende’s review.

Convention to the Rescue

As I said earlier, we were asked to provide a method of very quickly adding or “plugging in” new ways to access data that would produce the list of files for processing  and furthermore this needed to work without needing to recompile any part of the application. Given these conditions and the convention that our messages were always derived from data that included a list of files to be processed, it was pretty clear that IronPython was a perfect fit.

The UI itself was built using C# with the exception that the list box contents would be populated by an IronPython script and the same script would call-back into the C# code with the list of results (file paths). Having IronPython scripts call-back into managed code is trivial, you just need to pass a function from the executing managed assembly and have that function stored as a variable on the executing Script Scope. Then you just call the function. For example:

public void callback()
{
    Debug.WriteLine(“Test Callback method got called”);
}

public string funcback(string calledwith)
{
    Console.WriteLine(“Test Callback function got called with “ +
        calledwith);
    return calledwith;
}

private void SetDefaultScopeVariables(ScriptScope scope)
{
    scope.SetVariable(“test_callback_method”,
        new Action(callback));
    scope.SetVariable(“test_callback_function”,
        new Func<string, string>(funcback));
}

What the sketch of the UI (above) points out (see the red arrow), is that by dropping scripts into the application we were able to load up any number of ways in reading data to produce the list of files that would be used in creating our messages (sent via the distributor). So by simply dropping in a new IronPython script, another item would be loaded into the list box, offering a new method of accessing data requiring processing. Every scripted plug-in would provide an input, a description and it’s own particular method of accessing data (which could be anything).

On a side note, the UI sketch was produced using SketchFlow in Expression Blend 3 which looks quite promising.

I have an itch. A modelling domain specific language itch. I must say that my interest in new languages has been reinvigorated of late and this has seen a flurry of activity around the DLR, IronRuby, IronPython and now most recently the M family of languages via my curiosity in what Oslo has to offer. I haven’t yet delved nearly as far as I intend into M and Oslo, however from what I have seen I must admit I am interested in the possibilities that (small) DSL’s might offer, particularly in the vertical I happen to work in.

HOWEVER

My grandfather had a great saying for when he or other people seemed to find themselves in a state of confusion, he called it “oogle boogled” and I have to say that after Doug Purdy’s latest post I am feeling a bit that way right now. My read on things is that Microsoft seem to be placing “M” and Oslo adjacent to a war cry of  “DATA, DATA, DATA, DATA” and hence the announcement that:

“the modelling platform is aligned in a deep and fundamental way with the data programmability stack (ADO.NET, EF/EDM, Astoria, etc”

Doug also goes on to announce that the teams have come closer (much) together, with:

“we made a decision to merge the Data Programmability team (EDM, EF, Astoria, XML, ADO.NET, and tools/designers) and the “Oslo” team (“Quadrant”, Repository, “M”) together”

What Interested me in the first place?

The piece that interested me mostly about Oslo was MGrammar and building textual DSL’s. The reason was simple, I work in a industry where we have a throng of ‘Data Managers’, where I (and others) are engaged in building tools that help these guys manage data. These ‘Data Managers’ are by definition pretty data savvy and tech literate. The potential for the DSL’s in this space largely revolves around tool aids and even the ability to code gen workflow and deployment instructions. So in short I look forward to seeing what the possibilities are with writing DSL’s that produce navigable models and from there it’s a land of unknown promise. Of course to the Eclipse community this is nothing new, but for an old Visual Studio hack like me, it’s a brave new world.

My concern is that if the focus swings deep enough into the “DATA DATA DATA DATA” mantra, somehow the potential for what excited me about the Oslo initiative will perhaps be diminished in it’s significance. I am certainly not stating this to be case, only that it’s a concern of a dilatant, (me), on his way to becoming enlightened. Perhaps misunderstanding on my part and perhaps lost in the articulation on theirs?

Techavalanche is currently planning to host a training event not to be missed. The event is “Advanced Distributed Systems Design using SOA & DDD” and the course author and presenter is Udi Dahan of nServiceBus fame.

Interest in the event is growing quickly and we hope to be in a position to lock down the date very soon. We are getting close to having the numbers for a definite date of the week of October 26, therefore if you are interested in taking part, now is the time to start thinking about aligning the stars and putting your ducks in a row (so to speak).

If you are interested in attending the event, please contact Techavalanche to enquire with your details and register your interest on the poll and we can make sure you don’t miss out on what should be a week to remember.