Living in the Tech Avalanche Generation

I’m not about to cut all ties with my relational past and abandon the RDBMS. No, indeed that’s not the appropriate response – well at least not at this stage. So an exploration is what is going to be required to help in clarifying whether MongoDB can find a place in my toolbox.

Watching Michael Dirolf recently sparked my imagination, I began to see a potential abundance of goodness that could be applied in many way – possibly alongside a relational database. The potential for its implementation on the query side in a CQRS architecture for instance, serving as a store for directly bound view model data. MongoDB at first glance offers some nice features that would blend well into an architecture that included serving data from inside the web tier; BSON (binary JSON) may resonate well, it’s unceremonious and lite weight. The future promise of sharding as a product feature also has it’s appeal for obvious reasons.

Figure 1.0 – Potential Architecture

My initial enquiry was grounded in the efficacy of putting MongoDB to work in a distributed publish / subscribe architecture where (N) subscribing instances of autonomous components in a logical service boundary could effectively update N instances of a dedicated view model database. Each view model perhaps sharing an affinity to a web server allowing scaling out both in tandem. In order to sustain eventual consistency, this architecture employs transactional queues. MongoDB achieves it’s scale efficiencies by ignoring some the properties of an RDBMS that are commonly associated with the difficulties in scaling them. What’s clear is that an architecture that heavily leverages publish / subscribe, one way messaging and transactional queues (aimed at providing high availability and scale), will also generally require it’s storage to support transactional behavioural properties. MongoDB advertises it’s replication feature set as part of it’s toolbox, this helps achieve eventual consistency across a shared grid of servers but these two worlds are not about to find a way to play well together. In effect, both seem to be coming to a similar problem with a different solution, with MongoDB addressing scale through it’s built-in auto sharding, focusing more intensely on the database alone. For MongoDB to work in this environment would require the ability to enlist database calls in distributed transactions, something that is not supported by design. It would be possible to add a transactional messaging gateway to solve this problem, however there are other database alternatives that would not require me to take that extra step and additional latency.

This of course in no way invalidates assigning value to looking more deeply at MongoDB (and the like), therefore as I plant sign posts along the road about to be travelled and in light of the friction described above, I will now change track somewhat. My next move forward is to get a better grip of what the developer experience is about (not every problem requires the same solution).

Me as Developer - Starting Out

I started out with a somewhat familiar approach, running the JavaScript based shell you can run ad-hoc queries somewhat akin to query analyzer in the SQL Server world. Once installed and having put the binaries in my path environment variable, running the following commands will start up the server and the shell.

   1: c:\users\simon.segal\mongod

   2: c:\users\simon.segal\mongo

Figure 2.0 – Mongo Shell & DB

Going from download, to install, to running the shell, inserting and querying for data, is really a relatively quick exercise. With just a dash of reading from the excellent documentation (which also has a friendly readable PDF download utility), you can be writing code in the shell very fast. Code like this doesn’t need a lot of explaining to get going.

Listing 1.0 – Mongo Shell

>customer = {firstName: "Jon", lastName: "Doe", DateJoined: new Date("2010/01/30")};
>db.Customers.save(customer);
>db.Customers.find();
{ "_id" : ObjectId("4b5016e65d3c000000006393"), "firstName" : "Jon", "lastName"
: "Doe", "DateJoined" : "Sat Jan 30 2010 00:00:00 GMT+1100 (AUS Eastern Daylight
 Time)" }

The Officially Supported Drivers

• C
• C++
• Java
• Javascript
• Perl
• PHP
• Python
• Ruby

The Unofficial Community Supported Drivers

• C#
• Cold Fusion
• Erlang
• Factor
• Fantom
• F#
• Go
• Groovy
• PHP
• Powershell
• Ruby

The C# Driver

You can see that Microsoft technology is fairly well represented in these lists but I want to focus on the C# driver and talk about using MongoDB from C# specifically. Here’s some code using the C# driver to help start getting a feel for the API.

public void CreateCustomer()
{
    var mongo = new Mongo("127.0.0.1", "27017");
    mongo.Connect();
    var db = mongo.getDB("HR");
    var employees = db.GetCollection("Employees");
    var employee = new Document();
    employee["FirstName"] = "Simon";
    employee["LastName"] = "Segal";
    employee["JoinDate"] = DateTime.Now;
    employees.Insert(employee);
    var cursor = employees.FindAll();
    foreach (var emp in cursor.Documents)
    {
        Console.WriteLine(string.Format(
            "Customer ID is {0}, first name is {1} and " +
            "last name is {2}", emp["_id"],
            emp["FirstName"], emp["LastName"]));
        Console.WriteLine(emp.ToString());
    }
}

After running this code in Visual Studio we get the following output which prints a formatted string and the JSON representation as is.

Figure 1.0

Issues

The first problem with writing code like this is that it’s about as far away from being Domain Driven as I can imagine, there is no concept of an Entity, just a Document. Well it’s a Document Database you say – so why don’t I just stop whining and get on with writing code like that above? Sure transaction script might be great for some scenario’s but when I want to work with a richer domain model I don’t think I should need to change to accommodate the drivers API. Let me be clear, I don’t think there is anything wrong with the API, it’s perfectly fine and in fact I think it’s more than pliable enough for me to use it in other ways that will suit me from time to time.

I alluded earlier in the discussion, to the fact that I originally wanted to proof of concept MongoDB for playing a role as an eventually consistent storage for the Query side of an implementation of CQRS, where views are being bound directly to view model data (as described here) via a RESTFul facade. In part 2.0 I am going to look at how we can build a Context object (think DataContext or ObjectContext) to wrap up the C# driver’s functionality and enable working directly with POCO’s, removing any direct requirement to work from the MongoDB API’s Document class. That’s it for now, let’s see where we end up in Part 2.0.

Over the course of the next month or so I will be posting a series on using a set of helper libraries which I refer to as NFetchSpec. I have taken the somewhat unusual course of creating this dedicated post that will act as an index of sorts for this series. Another reason for this aggregated index page is that the title of the series accurately reflects the intent of its content however not so much in a search friendly way and I want to make sure that it can be easily found by people who are interested in using Repositories, Fetching Strategies, Specifications and code only mapping.

NFetchSpec aims to provide a basis for using Repositories, Fetching Strategies, Specifications and Code Only mapping using a Role Driven methodology. Here is some example consumer code of NFetchSpec:

Listing 1.0

var customersStartWithA =
    new Specification<IMakeCustomerPrefered>
        (c => c.CompanyName.StartsWith(”A”));

var fetch = new FetchingStrategy<IMakeCustomerPrefered>(true);

fetch.AddIntentions(new IEagerFetchingIntention[]
{
    EagerFetchingIntention.CreateInstance<IMakeCustomerPrefered,
                                          ICollection<IOrder>>(c => c.Orders)
});

var repo = new CustomerBuyingStatusRepository();

var custs =
    repo.Get<IMakeCustomerPrefered>(customersStartWithA, fetch);

foreach (var cust in custs)
{
    cust.MakePrefered();
}

As described in Part 1.0, a role is an interface that describes a business or system event and can flag our  intention; in the case of the code above, the intention is to make the Customer Preferred. Using NFetchSpec, this style of working (using roles) seeps into all aspects of the code. This is certainly not a requirement to use NFetchSpec and I will give some examples to the contrary as we get deeper into the series of posts, but it is the key reason for developing it. In Listing 1.0 we are using NFetchSpec to create a search predicate using a Specification instance, a Fetching Strategy to determine the most appropriate loading for the role and then finally using the Repository to get a Customer to make preferred. The API of NFetchSpec supports writing code like this, where each of the moving parts can be written in line together, however as mentioned in Part 1.0, one of the benefits of working in this role driven way is that we can use some IoC techniques to discover the correct Specifications and Fetching Strategies based on their implementing a given role, i.e, the interface that is specified to the Repositories .Get<TRole>() method. This is closer to the intended proposed usage:

Listing 2.0

var repo = new CustomerBuyingStatusRepository();

var custs =
    repo.Get<IMakeCustomerPrefered>();

The Code here in Listing 2.0 will instruct the NFetchSpec infrastructure to find a fetching strategy and Specification for this role when the overloaded version of the .Get<TRole>() method is called. This approach makes for an extremely flexible approach, swapping out the most appropriate component – if I don’t like the fetching strategy for a given role, I can simply replace it, facilitating just in time performance tuning (in the case of fetching) as we observe the system over time.

Here are the links to the posts in the series. You might notice that some of the links are not complete meaning that that post is yet to come. I will update this list as new posts make there way out.

Helping the Entity Framework Play it’s <Role> Part 1.0 (posted)

Helping the Entity Framework Play it’s <Role> Part 2.0 (posted)

Helping the Entity Framework Play it’s <Role> Part 3.0 (posted)

Helping the Entity Framework Play it’s <Role> Part 4.0 (posted)

All the code will be available on conclusion with the final post. Just one small thing before signing off, I really want to re-emphasise that it’s a really good idea to watch Udi present the ideas that lead me to attempt accommodating this approach for Entity Framework.

To date the best first up learning materials I have come across for EF have been the DNR TV web casts with Dan Simmons and Julia Lerman’s tutorials which I posted a PDF consolidation of recently. Dan Simmons gives some very good rudimentary ‘meet the Entity Framework’ demonstrations and I am beginning to quickly feel at home with EF. Having said that I will add my initial observations on the product so far (bear in mind I was a huge fan of LINQ To SQL).

• A better and more functional designer experience than LINQ to SQL
• Lacks the POCO ability that I crave (perhaps in V 2.x something)
• I like having the ability to map single entities to multiple tables, which LINQ To SQL could not do.
• Curiously, I have found that EF supports rapid development scenarios with an extremely similar approach to that of LINQ To SQL, therefore the argument to use LINQ TO SQL if your in a ‘hurry’ doesn’t really stack up for me anymore and I am questioning the validity of that proposal altogether.

Find the DNR TV episodes here:

Episode 1

Episode 2

To get more deeply into the product I just now ordered Julia’s book which I will have to wait till Jan 2009 for but there’s plenty to do whilst waiting, I am going to have a go at getting the specification pattern working with the Entity Framework as I did with LINQ to SQL previously and think about how to deal nicely with Fetching Strategies in EF.