Living in the Tech Avalanche Generation

Just as I was about to embark on  a new project (codenamed PONGO), the IronPython team announced a CTP 2.0 drop of the IronPython tools for Visual Studio 2010 which is exciting on a number of different levels.

If IronRuby and IronPython eventually make it all the way to Visual Studio integration in the full sense (that is packaged as part of Visual Studio itself) that would be a mark of first class citizenry.

What about Expression Blend?

Personally I work in both Blend and Visual Studio and even though the Visual Studio designer is somewhat improved and it’s intellisense is far superior than that of Expression Blend, it still does not provide me with the same fidelity or fluency in the design experience as I get from Blend. This leaves me with a problem when it comes to building applications where IronPython is the choice language.

It would have been unrealistic (at this stage) to expect Blend to open IronPython projects created in Visual Studio and sure enough that is the not case. What might be useful is Blend having a “design XAML only mode”, enabling it to open XAML files individually, taking advantage of the design experience it has to offer. Still not ready to give up on using Blend with XAML files that were part of a Visual Studio IronPython project, I set out to find a way.

Symbolic Links

The first thing I tried out was to see if Symbolic and Hard linked files would make the difference? First up I tried a symbolic link using the MKLINK utility:

MKLINK [[/D] | [/H] | [/J]] Link Target

Before doing this I created two projects, the first one an IronPython Tools project generated in Visual Studio and the second one a C# project created in Blend.

It should be obvious from the naming which project is which. In the blend project I removed the default XAML file generated by Blend.

Next came the file linking, I used only the Link and Target command line options, creating a symbolic link using the same name as the .XAML file in my IronPython project.

MKLINK pongo.xaml d:\temp\PongoBlend\pongo.xaml

Having successfully created the soft linked symbolic file I tried to add it to the Blend project only to find that it would not open – which, to blunt I wasn’t expecting to work. I was a little more hopeful however when it came to the Hard Linked file which I setup as follows:

MKLINK \H pongo.xaml d:\temp\PongoBlend\pongo.xaml

This did in fact open correctly in Blend and worked for a brief moment, when I changed something in Blend – Visual Studio would prompt me that the file had been changed outside the editor, unfortunately my celebrations were short lived when I found that the files were not updating in both locations when one or the other was subject to an edit. After some investigation and testing it appears that once opening the file in both the IDE’s the links would somehow eventually become broken and so the symbolic linking idea was now officially a lost prospect.

Duelling Projects

Finally I decided to synchronize two separate .XAML files in each of the two different projects, so the challenge was to find the most unobtrusive way of working in both IDE’s and easily syncing the the .XAML files after edits had been made. Given that one of the IDE’s was running a dynamic language with a REPL built in, I thought it shouldn’t prove too difficult.

What I did was to add two .py source files into the project, each one with some code that would sync in one direction, either from Blend to Visual Studio and vice versa. Sure its  far from a perfect solution but it just might be a price I’m willing to pay in the short term to get the benefit of using Blend. In the long term I really do hope that as the dynamic languages gain support in Visual Studio we also get some integrated ways to work in both IDE’s.

Here’s the very same Pongo.xaml file (or copy of it to be precise) in Expression Blend, but before you get too excited, the “send to interactive console” context menu option seems to suffer some kind of inconsistent behaviour (on my machine any way), causing the File copy operation in my .py files to work on only some occasions.

Undeterred I decided to fall back to an even more disintegrated experience and opt for running the synchronizing .py files in IPY.exe itself. This is easily achieved by associating .py files to the IPY.exe in Visual Studio using the context menu’s “open with” option, which is a something existing Visual Studio users will be very familiar with.

Like I said its far from a perfect solution and to be frank it’s somewhat annoying however my frustration is another thing altogether if I’m faced with doing all my layout and design in Visual Studio – so I will live with it for the moment. Hopefully the IronPython team and or the Expression Blend Team can find a solution that flows changes through more seamlessly in the short term and the perhaps in the long term allow us to open IronPython and IronRuby projects in Blend.

By the way if your wondering about the codename of the project that drove this adventure (PONGO) – the answer is yes if you guessed that it’s something that revolves around IronPython and MongoDB and I will be blogging about that more in the coming weeks.

This is the final part of the series on using IronPython together with the Entity Framework.

Part 1.0

In this first instalment we setup the discussion on what we were trying achieve, how to use the __clrtype__ class in IronPython 2.6 to create PODO’s (Plain Old Dlr Objects) ala POCO, for integration with the Entity Framework. Our aim is to use the Entity Framework using the Beta 2 of Entity Framework 4.0.

Part 2.0

We dig in deep and explore the internals of what’s required to create a PODO (POCO for the DLR). We also build an IronPython ObjectContext and determine how to manage mapping between the Entities and Database. We achieve a successful test run and prove that we can in simple cases get IronPython and Entity Framework to worth together in this fashion. Finally in part 2.0 we take steps to verify lazy loading for an entity graph and we strike our first hurdle, leaving us to look forward to part 3 to see if the problems encountered might be overcome.

Part 3.0

We start to examine the issues with scalar / native types and see how proxies fit into the puzzle. We check out eager loading and whilst we do have some success we still face a problem in rounding out the whole story. Finally we prepare to journey into debugging a crash dump file for part 4.

Part 4.0

Using Visual Studio 2010 we successfully load up our crash dump file in the hope of jumping over the final hurdle. We discover that the specialized version of IronPython 2.6 does not have publicly available debug symbols but we can manage to see where the active thread was when the fatal crash occurred.

The State of Play

The current set of problems unfortunately leaves us short in terms of using the Entity Framework from IronPython, however I did previously demonstrate (in two parts here and here) how this can be achieved by way of introducing an interop layer with C# addressing the Entity Framework.

Part 1.0

Part 2.0

Part 3.0

Part 4.0

It’s my hope that this post will make the previous posts easier to find given my recent problems with a blog scraping site. Finding the original content has been difficult in light of my successfully having the scraping site in question remove my content.

We left off with IPY.exe crashing and leaving us a dump file. The code that forced the crash:

orders = ctx.Orders.Where(“it.CustomerID > ‘C’“)
for order in Orders:
    print Order.CustomerID

I speculated that something other than member assignment (made by the Entity Framework) to string types in the underlying CLR type of the Python entity was failing and you might remember some proof of concept code was demonstrated to show a variety of ways in specifying types to _clrfields and property decorators.

The Dump File

So decided to re-watch this timely video presentation by Tess Ferrandez and hosted by Scott Hanselman, to see if I could get myself up to speed quick enough to debug the crash dump in VS 2010.

Not that it’s relevant to this topic per`se but did find something odd in locating the dump file and thought it’s worth mentioning here: when the system notifies you of a crash that produces a dump file (on Vista 64 in this case), it went through a process of:

1. Prompting with the message that IPY.exe had indeed crashed
2. Where to find the dump file

After noting the path of the dump file and clearing these two dialogs, I went to retrieve the dump file only to find it was not at all where the dialog said it would be. I ran the process again and watch the target folder for the dump file and it appears that when the second dialog is cleared the dump file will be deleted – so make a copy before that happens!

Anyway back to the problem at hand. Once having secured a copy of the crash dump file I loaded into VS.NET 2010 (very slow with downloading the debug symbols for the first time). Eventually this is what I was presented with:

The information of most note here is the exception information:

“The thread tried to read from or write to a virtual address for which it does not have the appropriate access”

Obviously it’s important to know what was happening on the active thread when debugging for a system that crashed. When prompted to break by visual studio when all the debug symbols have been loaded I received the following message:

An unhandled exception of type ‘System.ExecutionEngineException’ occurred in Unknown Module.

Not very useful so far huh! When I open the parallel call stacks I can see that eight threads were active at the time of the crash and that execution stopped in NT.DLL and in particular on the call listed below.

ntdll.dll!_ZwWaitForSingleObject@12()  + 0×15 bytes

I’m certainly not familiar with debugging dump files but here is what I garnered from the video with Tess. As we look through Call Stack (above) in Visual Studio, we can see that DLR, IronPython and Entity Framework 4.0 functions were called just prior to the failure however it would seem that debug symbols for all three are not currently available on the Symbol Servers from Microsoft. I haven’t yet been able to find them nor been able to locate the source for the special IronPython 2.6 Beta Version for .NET framework 4.0. So we appear to be at impasse, I will continue to see if I can get any deeper into the why and wherefore of the crash and hope to find that one day Plain Old DLR Objects (PODO’s) and Entity Framework can play together in harmony.

Conclusion

It this point in time I would certainly say that using Entity Framework with IronPython will require you build a managed wrapper class(es) that roll up what you want from the entity framework. I showed this to some extent previously with IronRuby in these two posts. Perhaps with a bit of collaborative work the DLR and ADO.NET teams can do a mind meld and move that bit close to making the DLR languages have a more compelling story in this endeavour, however I suppose the question should be asked, is this kind of interop that they have in mind for the dynamic languages?