Advanced Domain Model queries using Linq

I read a fantastic blog article from Nick Blumhardt yesterday which kicked off some crazy ideas in my head.

The Problem

With the following class, I can query on IsOverdue using linq... but not using linq to sql (or linq to nhibernate).  How frustrating!

class Loan
{
  // Member details omitted
  public bool IsOverdue
  {
    get 
    {
       return DateReturned == null && 
          (DateTime.Now - DateBorrowed).TotalDays > LoanPeriod;
    }
  }
}

This is a fundamental issue in relational to object mapping systems, but we can fix it using some linq trickery.

Disclaimer

I think my solution is very cool, but I'm not sure how acceptable it is for a production environment. 

Please leave feedback if you like it / makes you want to vomit.

The change

Let's modify the Loan class implementation of IsOverdue:

#region IsOverdue Property
 
public bool IsOverdue
{
  get { return IsOverdueProperty.GetValue(this); }
}
 
static public readonly QueryProperty<Loan, bool> IsOverdueProperty = QueryProperty.Register<Loan, bool>(
    x => x.DateReturned == null && (DateTime.Now - x.DateBorrowed).TotalDays > x.LoanPeriod);
 
#endregion

Here we move the logic for IsOverdue into a separate QueryProperty member that takes care of the nasty details.

Hopefully you'll agree that the code is still fairly readable, and encapsulated with all the logic still bundled inside the Loan class.

By moving this logic into a lambda expression, we can now write queries for both database and object collections! e.g

// Database Access
using (var db = new RepositoryDatabase<Loan>())
  var overdue = db.Where(y => y.IsOverdue && y.LoanPeriod == 5);
 
// Collection Access
using (var list = new RepositoryCollection<Loan>())
  var overdue = list.Where(y => y.IsOverdue && y.LoanPeriod == 5);

More Goodness

When coding these query properties it can be very handy to reference other query properties. e.g.

#region DaysOverdue Property
 
public double DaysOverdue
{
  get { return DaysOverdueProperty.GetValue(this); }
}
 
static public readonly QueryProperty<Loan, double> DaysOverdueProperty = QueryProperty.Register<Loan, double>(
  x => x.IsOverdue ? (DateTime.Now - x.DateBorrowed).TotalDays - x.LoanPeriod : 0);
 
#endregion
 
#region OverdueFee Property
 
public decimal OverdueFee
{
    get { return OverdueFeeProperty.GetValue(this); }
}
 
static public readonly QueryProperty<Loan, decimal> OverdueFeeProperty = QueryProperty.Register<Loan, decimal>(
  x => (decimal)(x.DaysOverdue * 0.45));
 
#endregion

This allows us to create very complicated queries in a very simple manner.

The Catch

Too good to be true?  You're right; there's a devil in the details. 

You might have picked it up already - how does linq interpret my query property correctly?

By modifying the expression tree!

My Repository class implements the Where() and translates the expression; replacing any QueryProperty references with a call to the actual lambda expression.

The really ugly part is that it's a string based lookup ie. "<name>Property".  Though I also check the type to be extra safe.

It's really not a bad trade off, considering it's very similar to how WPF DependencyProperty's work.

The Conclusion

This is a really easy and seamless way to implement custom queryable properties in your domain objects.

I've only touched on a fraction of the possibilities with this technique, but it shows the power and flexibility of linq.

Make sure you check out the sample application for the full code.


ed: Continue to Part II

Loop optimization trickery

A while ago I was playing around with C# 3.5 lambda expressions trying to work out how it behaves with variable scope.

What do you think is written to the console?

class Program
{
    static void Main(string[] args)
    {
        // Write number to console in worker thread
        for (int i = 1; i < 9; i++)
            DoLater(() => Console.Write(i));
    }
     
    // Call action delegate on separate thread
    static void DoLater(Action action)
    {
        // I'm avoiding using another lambda here to simplify test
        new Thread(new ParameterizedThreadStart(DoLater)).Start(action);
    }
    
    // Calls the action after waiting for a second
    static void DoLater(object action)
    {
        Thread.Sleep(1000);
        ((Action)action)();
    }
}

If you initially guessed (like me) that it displays the numbers 1-9 in some random order (eg. '14385726') - you'd be wrong...   it actually produces '999999999' every time.

So I cranked up .net reflector (I love it) and had a look at my code (optimization set to .NET 1.0 - then translated by me):

private static void Main(string[] args)
{
    AutoDelegateClass tmpVar = new AutoDelegateClass();
    tmpVar.i = 1;
    while (tmpVar.i < 9)
    {
        DoLater(new Action(tmpVar.Method));
        tmpVar.i++;
    }
}

This is much more obvious...  .net optimizes the lambda expression out of the loop!

Well that was unexpected, but guess what happens when I change main to this:

static void Main(string[] args)
{
    for (int i = 1; i < 9; i++)
    {
        int j = i;
        DoLater(() => Console.Write(j));
    }
}

This works! ie. it does not optimize the lambda expression out of the loop.


Although confusing at first, it actually makes a lot of sense from the compiler point of view - the initial expression doesn't actually change, since we are referencing a mutable variable... so why not optimize it?

It might be convenient to turn off this optimization programmatically - kind of like a volatile keyword or something, but it's no big deal.

I guess it goes to show that functional code works best with immutable data... and that reflector is way cool.


BTW: The exact same thing occurs in foreach over a collection; and also using classes rather than structs.

UI - You don't know how much you suck

I made a small mistake in my last post - instead of 'do what you want' what I meant to say was:

Hire a damn graphic designer and buy some icons!

This is a subtle difference, but the delusion of competency is unfortunately common.

Like your home handyman disasters, the DYI attitude for user interfaces can cost you big time.

However IT departments are usually too cheap, and some developer ego's are too big to realize that they can't do everything... thus UI abominations are created everyday.


UI development is hard, harder than most people think. 

The UI layer is where everything comes together, and as a result it's here you find all the issues with the underlying architecture.


If you're a UI developer - you suck, so make sure you check out the UI guidelines for your dev system (Microsoft, Apple, Gnome or KDE).

If you're not a UI developer; shut up, fix your code and give the UI guys a break.

Oh, and good luck!

Disclaimer: I've done both jobs, so I suck twice as much.

UI - You can't please everyone

Although it's difficult to create good UI, it doesn't mean that bad UI is easy - in fact some people go to great lengths to produce bad UI.

That said, good UI simply boils down to:

Intuitive, Pretty and Performant - in a combination that targets the intended user.

eg. A programmer is going to prefer performance over pretty, and a hairdresser might prefer pretty over intuitive... but you still need all three for a good user experience.

If you don't have access to your user (and that's generally the case), just pretend that you are the user.

If you wouldn't use your own app; that's a good sign of bad UI.

In the end, you can't please everyone - so please yourself, then at least one person is happy.

Six Percent Genius

Project Euler (pronounced "Oiler") provides a number (~182) of challenging math/programming problems... which is a fantastic environment to play with F#!

I've only completed 6% of the questions but I've already learnt a lot. (F# is huge)

If you have some free time (or get bored during your compiles), I'd definitely recommend having a go - in your own language of choice of course!

Deriving Miss Daisy

As I've mentioned previously, I'm slowly reading through SICP - it's an amazing book and reading it has been a humbling experience.

'Geeking' Out

I completely geeked out reading one example on functional derivatives.  (Take a math expression as a parameter and return the derivative as an expression)

It's in the first chapter, and they use this to implement Newton's Method -> to then implement a Sqrt function!

Approximate Derivative

After I picked my brain off the floor, I decided to have a crack at implementing derivatives in C# using lambda expressions:



Using the formula of a derivative (above), we can simply code an approximate derivative (below):

Func<double, double> Approximate(Func<double, double> f)
{
    var h = 0.0001;
    return a => ((f(a + h) - f(a)) / h);
}

Symbolic Derivation

Easy huh!  So for fun I decided to use linq expression trees to derive algebraically.

It's easy to get an expression tree in C# - simply use the Expression<> type.

Func<double, double> Derive(Expression<Func<double, double>> e)
{
    var lambda = new Derivative().Eval(e) as LambdaExpression;
    return (Func<double, double>) lambda.Compile();
}

I parse the expression tree using Matt Warren's ExpressionVisitor class - and using the basic rules of Derivation (Constant, Product and Quotient rules) we get:

class Derivative : ExpressionVisitor
{
    public Expression Eval(Expression exp)
    {
        return this.Visit(Evaluator.PartialEval(exp));
    }
    
    protected override Expression VisitBinary(BinaryExpression b)
    {
        // Product Rule: (fg)' = f'g + fg' 
        if (b.NodeType == ExpressionType.Multiply)
        {
            return Expression.Add(
                Expression.Multiply(Eval(b.Left), b.Right),
                Expression.Multiply(b.Left, Eval(b.Right)));
        }
    
        // Quotient Rule: (f/g)' = (f'g - fg') / (g*g)
        if (b.NodeType == ExpressionType.Divide)
        {
            return Expression.Divide(
                Expression.Subtract(
                    Expression.Multiply(Eval(b.Left), b.Right),
                    Expression.Multiply(b.Left, Eval(b.Right))),
                Expression.Multiply(b.Right, b.Right));
        }
    
        return base.VisitBinary(b);
    }
    
    // Parameter Derivation: f(x)' = 1
    protected override Expression VisitParameter(ParameterExpression p)
    {
        return Expression.Constant(1.0);
    }
    
    // Constant Rule f(a)' = 0
    protected override Expression VisitConstant(ConstantExpression c)
    {
        return Expression.Constant(0.0);
    }
}

Wrapping it up

To quickly finish this long blog post, here's the usage code:

// f(x * x * x)' = 3 * x * x
var approx = Approximate(x => x * x * x);
var derive = Derive(x => x * x * x);
 
var y1 = approx(1); // 3.0000300001109532
var y2 = derive(1); // 3.0

And there it is!  Useful? maybe not.  Fun? Definitely!

The 'Y' in Blog

It's quite demoralizing when you consider how much you don't know. I am continually reminded that I underestimate it's sheer quantity... it's a sad realization.

Since there is so much already written by people more knowledgeable and intelligent than myself, why do I write?

Well, I started this blog to show my appreciation, and hopefully to contribute something back to the development community.

So kudos to all of you... keep sharing the love (of maths & programming)!

Associate with people who are searching for answers, avoid those who have found them.

Happy New Year!