Living in the Tech Avalanche Generation

A big thanks to the people at DreamHost for getting the server that this site is located on back up and running after some problems this week. Normal transmission will continue from this point forth. Coming up next some more BizTalk, Design Patterns (with IronRuby to add some spice) and some chatter about Oslo and SOA.

One of the long time criticisms of WCF and ASMX web services has been the lacking of support for Publish / Subscribe. Yes I know that WS-* includes specification(s) to enable this, however if you can show me where on this wonderful planet of ours exists some decipherable prescribed guidance on putting together a store and forward architecture with durable messaging and discoverable endpoints with pub / sub capabilities, I would be very grateful. In the meantime, something that some of my colleagues and I are now considering is how we can leverage the little known and equally poorly documented System.Net.PeerToPeer library in helping us put a reliable messaging framework together that will carry messages over any of the WCF transports available. Now it’s true that WCF includes a Peer to Peer binding that leverages this library, however I have something else in mind. At this stage we envisage a design where we can deploy WCF endpoints across our network and have them use PNRP to advertise message contracts that they publish for other peers (services) to subscribe. This is very exciting and potentially removes any need to use a registry like UDDI or some home baked thing.

This wont be enough by itself however, issues such as durable messaging, the ability to support idempotent message handling and long running correlated workflow’s, all will require some more infrastructure code that we are going to have to write. The good news is that with some considerable work all these things are possible. My friends and I are planning on using SQL Server Express (Service Broker for it’s queue’s only) at the endpoints to serve a durable store, this also doubles nicely with smart client requirements should they exist in any applications that leverage such a framework. We also plan to utilise Workflow Foundation to manage the long running processes (state machine’s) inside the service. By service I am NOT referring to the WCF endpoints. The endpoints are simple entry & exit points for messages and it’s the services that host the endpoints and their message handlers that we consider make up our services.

This is what an endpoint might look like:

What this diagram doesn’t show is that there is quite a bit of infrastructure code required to glue all this together, perhaps a Bus like API where messages are published and subscribed to and where this ‘Bus’ persists it’s messages to it’s local queue. All messages will be read, sent and removed from queue to discovered endpoints inside a transaction, therefore if any transport of messages from point to point fails the transaction will rollback and return the message to the queue. I have previously defined what the WCF endpoints would potentially look like (at a very rudimentary code level) in a previous post.

To sum up, unless Oslo has something special in store that isn’t making itself plainly obvious then I am quite up beat about this design providing some value as a lite weight framework. Counting down the days but I am not holding my breath, perhaps somewhere deeper in the products release cycle.

I am spending more and more time with my head wrapped around BizTalk in one way or another and very recently I am glad to say that I successfully navigated the MCTS exam and certified as a BizTalk specialist. Does this mean that I expect to do a lot of work in BizTalk from this point on? Good question. Well I do expect to do some yes and I am hoping that I will be able to offer the measured position of being from the camp that does not reach for BizTalk to solve every SOA problem looking for a solution.

There are alternatives

I do appreciate many of the features of BTS 2006 but I must confess that sometimes I do hear some absolutely mad assertions coming from those who would throw BizTalk at every problem where messaging was a clear choice. I have heard voices hold forth that ALL business rules should be managed in one central location (in BizTalk) and that all applications and line of business applications should make BizTalk their messaging target endpoint. I remember  objecting to this idea firstly on the grounds that it was out of alignment with the budget constraints of a given project and secondly that it was clearly propagating fallacy number eleven of the fallacies of distributed computing. Now in this particular case my argument was pretty solid but I should also point out that it’s not always that cut and dry. BizTalk may well be the right architectural and financial choice in many cases where we know that scaling out on mass wont be a future consideration coupled with a short delivery window imperative or perhaps a business lacking in the capability to maintain a considerable software team required to maintain a complex custom SOA application. This does not exhaust the list no doubt but gives an idea.

SOA without the bells and whistles

I may want to build my own very light weight SOA framework or use NServiceBus, either option affording me the option to deploy multiple versions of the same service at different geographical points across my LAN. In this scenario my rules should stay with my service, I don’t fancy the idea of wearing the expense of BTS for every long running (or otherwise) service I write and I have all those nicely designed domain entities that have come out of my having been Domain Driven all along the way! Heck I may even be using WCF and WF in the most out of the box fashion and even then it makes very little sense to put all my business rules in one place just because I can. Centralizing things in my architecture expose me to a single point of failure, which we know isn’t a great idea. Some will argue that it’s easier to maintain in one place and if our environment is lacking in complexity then there are times where that does hold true, but over time as developers and architects begin to pile things into one place things might start to unravel. Imagine a BizTalk environment that begins to exhibit as a massive repository for your organisations business rules! Hmm, if I change this one rule here what happens to all the logic that uses my other 1300 rules and which one of them have dependencies on the rule I just changed? I should clarify here: I am not ready to throw out the baby with the bath water just yet however, I wouldn’t even begin to consider for a moment the notion that I should build tools for mapping or adapters for some of the LOB applications out there, these very mature aspects of BizTalk can exist very nicely behind my services rather at the front of my design.

Figure 1.0

Oslo is certainly suggesting architectural decisions like that exemplified above, where the BizTalk hosting environment continue to be used for integration services and the ‘Dublin’ Server be used for application services. This distinction has me curious; will Oslo’s process server be similar to BizTalk with respect to being a Broker? In the meantime as I become more adept at using the product, I feel more at peace with making up my mind about why, when and where BizTalk makes sense.

I have been playing around with IronRuby a bit just of late and I was curious to see whether it was at a stage where it was working with LINQ To SQL. It was clear from the outset that using LINQ To SQL in the most common way (with full UI experience) in VS.Net was simply a silly thing to expect and it became immediately apparent that any benefit of IEnumerable<T> or IQueryable<T> was out the window also. Given that that any attempt to access IEnumerable<T> from any instance of Table<T> was for now a journey into folly, I tried briefly to circumvent that approach and see if I could POCO my way to an outcome with Mapping files. Listing 1.0 shows this unfortunately doomed venture.

Attempt No # 1.0

First the C# POCO Entity class required in the RubyCustomer library.

using System;

namespace Org.TechAvalanche.Orm.Tests.Domain
{
    public class Customer
    {
        private string _CustomerID;
        private string _CompanyName;
        private string _ContactName;
        private string _ContactTitle;
        private string _Address;
        private string _City;
        private string _Region;
        private string _PostalCode;
        private string _Country;
        private string _Phone;
        private string _Fax;

        public Customer() { }

        public string CustomerID
        {
            get {return this._CustomerID;}
            set {this._CustomerID = value;}
        }

        public string CompanyName
        {
            get {return this._CompanyName;}
            set {this._CompanyName = value;}
        }

        public string ContactName
        {
            get {return this._ContactName;}
            set {this._ContactName = value;}
        }

        public string ContactTitle
        {
            get {return this._ContactTitle;}
            set {this._ContactTitle = value;}
        }

        public string Address
        {
            get {return this._Address;}
            set {this._Address = value;}
        }

        public string City
        {
            get {return this._City;}
            set {this._City = value;}
        }

        public string Region
        {
            get {return this._Region;}
            set {this._Region = value;}
        }

        public string PostalCode
        {
            get {return this._PostalCode;}
            set {this._PostalCode = value;}
        }

        public string Country
        {
            get {return this._Country;}
            set {this._Country = value;}
        }

        public string Phone
        {
            get {return this._Phone;}
            set {this._Phone = value;}
        }

        public string Fax
        {
            get {return this._Fax;}
            set {this._Fax = value;}
        }
    }
}

coupled with the following LINQ To SQL Mapping file

<?xml version=”1.0″ encoding=”utf-8″?>
<Database Name=”Northwind”
    xmlns=”http://schemas.microsoft.com/linqtosql/mapping/2007″>
  <Table Name=”Customers” Member=”Customer”>
    <Type Name=”Org.TechAvalanche.Orm.Tests.Domain.Customer”>
      <Column Name=”CustomerID” Member=”CustomerID” DbType=”NChar(5) NOT NULL”
          IsPrimaryKey=”true” CanBeNull=”false” UpdateCheck=”Never” />
      <Column Name=”CompanyName” Member=”CompanyName”
          DbType=”NVarChar(40) NOT NULL” CanBeNull=”false” UpdateCheck=”Never” />
      <Column Name=”ContactName” Member=”ContactName”
          DbType=”NVarChar(30)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”ContactTitle” Member=”ContactTitle”
          DbType=”NVarChar(30)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”Address” Member=”Address”
          DbType=”NVarChar(60)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”City” Member=”City”
          DbType=”NVarChar(15)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”Region” Member=”Region”
          DbType=”NVarChar(15)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”PostalCode” Member=”PostalCode”
          DbType=”NVarChar(10)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”Country” Member=”Country”
          DbType=”NVarChar(15)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”Phone” Member=”Phone”
          DbType=”NVarChar(24)” CanBeNull=”true” UpdateCheck=”Never” />
      <Column Name=”Fax” Member=”Fax”
          DbType=”NVarChar(24)” CanBeNull=”true” UpdateCheck=”Never” />
    </Type>
  </Table>
</Database>

and now our IronRuby attempt to talk DIRECTLY LINQ To SQL.

require ‘mscorlib’
require ‘System.Core, Version=3.5.0.0,
  Culture=neutral, PublicKeyToken=b77a5c561934e089′
require ‘System.Windows.Forms, Version=2.0.0.0,
  Culture=neutral, PublicKeyToken=b77a5c561934e089′
require ‘System.Data.Linq, Version=3.5.0.0,
  Culture=neutral, PublicKeyToken=b77a5c561934e089′
require ‘RubyCustomer, Version=0.0.0.0,
  Culture=neutral, PublicKeyToken=c1aba19d9dedea39′

begin

  #get the mapping into a stream
  stream = System::IO::StreamReader.new(“C:\\Users\\simon.segal” +
    “\\Desktop\\IronRuby\\ironruby\\bin\\DomainMapping.xml”)
  mapping_file =
    System::Data::Linq::Mapping::XmlMappingSource.
      FromStream(stream.BaseStream)
  #new up a DataContext with our mappings
  db = System::Data::Linq::DataContext.new(“server=localhost;initial “+
    “catalog=Northwind;user id=sa;password=supremo”, mapping_file)
  #new up a POCO customer from my C# library
  cust = Org::TechAvalanche::Orm::Tests::Domain::Customer.new
  #not sure if typeof works in IronRuby
  cust_type = cust.GetType()
  #load the table of customers from northwind
  custs = db.GetTable(cust_type)
  #tried to get enumerator but .Current gets a method error 
  #and so ends our journey
  list = custs.GetEnumerator()
  puts “The call to GetEnumerator returned a “ +
    list.GetType.ToString()
  currentCust = list.Current
  rescue Exception => ex
    puts “someone is missing a method!” + ex.to_s
    #just to prove that it’s a Table<Customer>
    puts “Just to show that we did get as far as “ +
           “getting the Table<Customer> : “ +
           custs.ToString()

end

Which yields the following output from standard IO (click to see full view):

So as you can see here, we get close but no cigar. Ready to give up - damn, not yet! Lets move all the LINQ To SQL code into our custom library and call it from IronRuby - that should work? We already had a POCO in our library referenced above (see the require ‘RubyCustomer’). So here we go.

Attempt No # 2.0

As a jumping off point for attempt # 2, I thought it might be time to check out Ruby In Steel, a third party integrated UI platform for VS.NET and IronRuby. I wanted to be able to keep my C# library with the LINQ To SQL code and the IronRuby consumer code in a single solution if possible and I was looking for a bit of holiday from writing Ruby Code in SciTE and NetBeans, therefore looking at Ruby In Steel had been on the agenda for a while so I figured now was as good a time as any. This gave me a more instantly gratifying experience in developing a Windows Form to present my LINQ To SQL data as well:

As you can see I have my C# project and my IronRuby project coupled together in a single VS.Net solution and whilst Ruby in Steel is not quite as stable or fully featured as you would expect it to be when IronRuby makes it’s official debut, it certainly helped speed things up quite a bit.

The LINQ To SQL code consisted of a run of the mill repository, POCO and mapping file (the entity itself remained as is - see above Customer Class).

using System.Data.Linq;
using System.Linq;

namespace CsharpLinqToSqlForIronRuby
{
    public class CustomerRepository
    {
        private readonly DataContext _context = null;

        public CustomerRepository(DataContext ctx)
        {
            _context = ctx;
        }

        public Customer[] All()
        {
            Customer[] retCustomers = null;
            using (_context)
            {
                retCustomers = _context.GetTable<Customer>().ToArray();
            }
            return retCustomers;
        }

        public Customer FindById(string custId)
        {
            Customer retCustomer = null;

            using (_context)
            {
                retCustomer = _context.GetTable<Customer>().
                    Where(c => c.CustomerID == custId).Single();
            }
            return retCustomer;
        }
    }
}

On the IronRuby side of things, we dialled up a class to handle calling our C# Repository library:

require ‘mscorlib’
require ‘System.Core, Version=3.5.0.0, ‘ +
    ‘Culture=neutral, PublicKeyToken=b77a5c561934e089′
require ‘System.Windows.Forms, Version=2.0.0.0, ‘ +
    ‘Culture=neutral, PublicKeyToken=b77a5c561934e089′
require ‘System.Data.Linq, Version=3.5.0.0, ‘ +
    ‘Culture=neutral, PublicKeyToken=b77a5c561934e089′
require ‘D:/simon.segal/LocalWorking/IronRubyProofs/’ +
    ‘IronRubyWithLinqToSqlForms/IronRubyWithLinqToSqlForms/’ +
    ‘CsharpLinqToSqlForIronRuby.dll’

class CustomerController

    def GetAllCustomers

        #get a stream of the mapping file
        stream = System::IO::StreamReader.new(“D:\\simon.segal\\” +
            “LocalWorking\\IronRubyProofs\\” +
            “IronRubyWithLinqToSqlForms\\” +
            “IronRubyWithLinqToSqlForms\\DomainMapping.xml“)
        #set the mapping source
        mapping_file = System::Data::Linq::Mapping::XmlMappingSource.
            FromStream(stream.BaseStream)
        #new up a DataContext with our mapping
        db = System::Data::Linq::DataContext.new(”server=localhost;initial ” +
            “catalog=Northwind;user id=sa;password=supremo”, mapping_file)
        #new up our repository
        cust_repos = CsharpLinqToSqlForIronRuby::CustomerRepository.new(db)
        #get all the customers
        custs = cust_repos.All()
        #print customers to the console window (in debug)
        custs.each {|cust| puts cust.CompanyName}

        return custs

    end

end

And what we needed now was an IronRuby WinForm to do the presentation work and data binding for us:

require ‘Form1.designer.rb’
require ‘CustomerController.rb’

class Form1

    #form load
    def Form1_Load(sender, e)
    end

    #button click event handler
    def button1_Click(sender, e)

        controller = CustomerController.new
        self.comboBox1.DisplayMember = “CustomerID”
        self.comboBox1.ValueMember = “CustomerID”
        custs = controller.GetAllCustomers.each {|customer|
            self.comboBox1.Items.Add(customer)}
        self.lstCustomer.DataSource = custs
        self.lstCustomer.DisplayMember = “CustomerID”
        self.lstCustomer.ValueMember = “ContactName”

    end

    #combo box selected index changed handler
    def comboBox1_SelectedIndexChanged(sender, e)
        cust = self.comboBox1.SelectedItem
        self.textBox1.Text = cust.CompanyName
        self.textBox2.Text = cust.ContactName
    end

    #list customer selected index changed handler
    def lstCustomer_SelectedIndexChanged(sender, e)
        cust = self.lstCustomer.SelectedItem
        self.textBox1.Text = cust.CompanyName
        self.textBox2.Text = cust.ContactName
    end

end

I am thinking. Stop that you say? Seriously, I have been wondering about combining a fluent interface with a some VS.Net VSX visual DSL tooling. I am currently working a small internal domain specific language and it occurred to me that rather than leaving the only method of access to the DSL as raw code, that perhaps by building out some visual entities around the entities expressed through the DSL, I might just be able to build a platform and not just the DSL on it’s own. So if we have code like:

3.Month.Old.Documents(“Contract”)

created by a designer like experience along of the lines of this:

Just having preliminary thoughts about this at the moment and I can already see that the vocabulary of DSL runs the risk of being influenced by the UI design in way that obscures and corrupts its meaning. For example should it read:

//makes sense
3.Day.Old.etc
//seems awkward and clunky
3.Day.InFuture

A quick revision to the Fluent Interfaces and Designer and we might get this:

Ok so we can start to think about all sorts of things when it comes to the readability of the language underneath the designer but my next question is what about constraints? Should we have any? Does it feel right to leave the user the ability to define a design that would produce this:

3.Years.Now.etc.etc

But I don’t want to start building constraints around what would amount to business logic concerns now, or do I? A lot more thought is required on this whole topic I think.