Friday, August 1, 2008
When working with value types (structs) the JIT currently hits the stack each time you access a field - which is quite inefficient.
In SP1 they've added an algorithm called "value type scalar replacement"; which intuitively does exactly what it sounds like!
Take the following code:
The JIT will replace the value type (Point is a struct) with scalars:
After the JIT inlines the structure, it can optimise the routine by placing the integers on the register rather than the stack.
What's the difference?
We are roughly looking at a 10x improvement! (Using a quick and dodgy test app)
This is fantastic, especially when dealing with lots of structs like points and rectangles etc.
What's the catch?
This isn't magically going to work on all your value types; there are strict guidelines when this replacement can occur:
- The value type must have no more than 4 fields
- These fields must be value types, or object references
- You must use [StructLayout(LayoutKind.Sequential)], which is the default anyhow.
There some other catches, but if you are creating complicated structures, you really should think about making them a class.
This is just a snippet of the algorithm, whereas the original article explains it in much more detail.
BTW: This is in the CLR, so all .Net languages benefit! And of course this includes F# and FParsec.
Cheers to SP1!
Wednesday, July 23, 2008
Parameter Support (Contrived Example)
Continuing the Library example, how would you query the books that will be overdue next week?
Well you could write the following:
How it works
The CheckOverdue Property simply returns a Lambda Expression as it's result.
I've modified the expression parsing routines to support parameters (including multiple) and even nested lambdas!
I can't really see the need for it, but hey... why not! Curry anyone? :)
So how would you call such a monstrosity?
But check out the SQL! Great work Linq to Sql guys!
If you're particularly astute, you'll have noticed that my syntax for declaring a QueryProperty has changed slightly since the last post.
Here's the class declaration of the CheckOverdue Property:
Although this is a method, you can still define it as a property (notice that I return a lambda expression).
So What's Changed?
Well, QueryProperties no longer derive from an Attribute class. Although it worked, I felt dirty about it.
I borrowed from Fredrik's syntax and pass the property type to the Attribute - much cleaner.
Since I'm no longer defining the QueryProperty as static, I had to change the way it is referenced by the class property.
The code above shows my attempt at making this easy, yet efficient - but it's up to you how to implement it. E.g. this would work well too:
If you can think of better ways I'd love to hear it!
Since I posted the parser code in my last post, here it is again - and my has it grown!
As usual, here is the sample code for you to experiment with yourself.
I hope you've enjoyed this series, I certainly have.
If I get the chance I might write a post explaining the expression parsing in detail, or the compiled queries - but till then...
Thursday, July 17, 2008
So what's changed?
Not much. Defining your query properties has changed (and obviously some implementation details). All the linq syntax stuff stays the same.
Here's the same Loan class, rewritten:
And here's our query property definition:
What's the difference?
Well, depending on your programming tastes, it's cleaner. We've reduced some of the tight coupling, and we now have the ability to spread the logic (which is more suitable in certain situations).
And if you're wondering why I seem to have an obsession with static members, it's mainly due to performance - and truth be told, I'm pre-optimizing here. Generally I'm not such a fan... honest!
Speaking of performance, using Attributes has a penalty. According to my tests, it's roughly 30% slower to parse the expression.
This might sound like a lot, but we're talking about 0.37 vs 0.28 milliseconds per query here - and once you consider the database retrieval time... it's really nothing.
I don't even parse the expression tree with linq to objects - so no impact at all!
Bonus code (Expression Parser)
Here's my expression parsing code, I know you want to see it:
Beautiful isn't it!
If you've been following this little series, I'd love to hear what you think of the new syntax.
I'm still hesitant, since this new syntax results in more code - but I think it's a cleaner approach.
As usual, you can find the sample app here - enjoy!
Wednesday, July 16, 2008
See the changes
I took my rewrite as an opportunity to support the full query syntax:
and added easy support for compiled queries (currently only supporting linq to objects and linq to sql):
I've also optimized the expression parsing (speed difference is negligible to normal queries), and I tweaked the QueryProperty construction:
The big change here is that I've added an extra parameter; a reference back to the mapping property.
I'd really like a memberof(Loan.IsOverdue) keyword style syntax in C#, but since there's not, I use this expression to grab the MemberInfo - which is used in the expression parser.
It's a slight DRY drawback, but it's a faster way than using reflection (and avoids an ugly string lookup) - and it's much faster than using attributes.
Get the code
Most of the big changes are all implementation specific, so be sure to check out the sample app.
Please let me know what you think!
BTW: Joseph, I made some architectural changes to the LinqKit.ExpandableQuery stuff. Externally it behaves exactly the same, but it now lets me reuse a lot more code.
It's light on the comments, but I'm hoping my next post will explain myself.
ed: Continue to Part III
Tuesday, July 15, 2008
I'm a huge keyboard shortcut fan (for a windows developer)... for common tasks it's just annoying to have to grab the mouse.
For me, one such task was the rename / resolve support in Visual Studio:
Using the mouse to bring up this dialog has got to be the most frustrating task in Visual Studio.
So much so, I refused to use it; instead opting for the right-click 'resolve' approach, or even manually typing in namespaces.
A few months ago, I stumbled on "Ctrl ." and to my delight ended my coding frustration.
I'm now more productive, less frustrated... do yourself a favour and give "Ctrl ." a go!
Monday, July 7, 2008
With the following class, I can query on IsOverdue using linq... but not using linq to sql (or linq to nhibernate). How frustrating!
This is a fundamental issue in relational to object mapping systems, but we can fix it using some linq trickery.
I think my solution is very cool, but I'm not sure how acceptable it is for a production environment.
Please leave feedback if you like it / makes you want to vomit.
Let's modify the Loan class implementation of IsOverdue:
Here we move the logic for IsOverdue into a separate QueryProperty member that takes care of the nasty details.
Hopefully you'll agree that the code is still fairly readable, and encapsulated with all the logic still bundled inside the Loan class.
By moving this logic into a lambda expression, we can now write queries for both database and object collections! e.g
When coding these query properties it can be very handy to reference other query properties. e.g.
This allows us to create very complicated queries in a very simple manner.
Too good to be true? You're right; there's a devil in the details.
You might have picked it up already - how does linq interpret my query property correctly?
By modifying the expression tree!
My Repository class implements the Where() and translates the expression; replacing any QueryProperty references with a call to the actual lambda expression.
The really ugly part is that it's a string based lookup ie. "<name>Property". Though I also check the type to be extra safe.
It's really not a bad trade off, considering it's very similar to how WPF DependencyProperty's work.
This is a really easy and seamless way to implement custom queryable properties in your domain objects.
I've only touched on a fraction of the possibilities with this technique, but it shows the power and flexibility of linq.
Make sure you check out the sample application for the full code.
ed: Continue to Part II
Friday, June 27, 2008
What do you think is written to the console?
If you initially guessed (like me) that it displays the numbers 1-9 in some random order (eg. '14385726') - you'd be wrong... it actually produces '999999999' every time.
So I cranked up .net reflector (I love it) and had a look at my code (optimization set to .NET 1.0 - then translated by me):
This is much more obvious... .net optimizes the lambda expression out of the loop!
Well that was unexpected, but guess what happens when I change main to this:
This works! ie. it does not optimize the lambda expression out of the loop.
Although confusing at first, it actually makes a lot of sense from the compiler point of view - the initial expression doesn't actually change, since we are referencing a mutable variable... so why not optimize it?
It might be convenient to turn off this optimization programmatically - kind of like a volatile keyword or something, but it's no big deal.
I guess it goes to show that functional code works best with immutable data... and that reflector is way cool.
BTW: The exact same thing occurs in foreach over a collection; and also using classes rather than structs.
Monday, June 23, 2008
I made a small mistake in my last post - instead of 'do what you want' what I meant to say was:
Hire a damn graphic designer and buy some icons!
This is a subtle difference, but the delusion of competency is unfortunately common.
Like your home handyman disasters, the DYI attitude for user interfaces can cost you big time.
However IT departments are usually too cheap, and some developer ego's are too big to realize that they can't do everything... thus UI abominations are created everyday.
UI development is hard, harder than most people think.
The UI layer is where everything comes together, and as a result it's here you find all the issues with the underlying architecture.
If you're a UI developer - you suck, so make sure you check out the UI guidelines for your dev system (Microsoft, Apple, Gnome or KDE).
If you're not a UI developer; shut up, fix your code and give the UI guys a break.
Oh, and good luck!
Disclaimer: I've done both jobs, so I suck twice as much.
Tuesday, April 22, 2008
That said, good UI simply boils down to:
Intuitive, Pretty and Performant - in a combination that targets the intended user.
eg. A programmer is going to prefer performance over pretty, and a hairdresser might prefer pretty over intuitive... but you still need all three for a good user experience.
If you don't have access to your user (and that's generally the case), just pretend that you are the user.
If you wouldn't use your own app; that's a good sign of bad UI.
In the end, you can't please everyone - so please yourself, then at least one person is happy.
Wednesday, February 20, 2008
I've only completed 6% of the questions but I've already learnt a lot. (F# is huge)
If you have some free time (or get bored during your compiles), I'd definitely recommend having a go - in your own language of choice of course!
Tuesday, January 15, 2008
I completely geeked out reading one example on functional derivatives. (Take a math expression as a parameter and return the derivative as an expression)
It's in the first chapter, and they use this to implement Newton's Method -> to then implement a Sqrt function!
After I picked my brain off the floor, I decided to have a crack at implementing derivatives in C# using lambda expressions:
Using the formula of a derivative (above), we can simply code an approximate derivative (below):
Easy huh! So for fun I decided to use linq expression trees to derive algebraically.
It's easy to get an expression tree in C# - simply use the Expression<> type.
I parse the expression tree using Matt Warren's ExpressionVisitor class - and using the basic rules of Derivation (Constant, Product and Quotient rules) we get:
Wrapping it up
To quickly finish this long blog post, here's the usage code:
And there it is! Useful? maybe not. Fun? Definitely!
Monday, January 7, 2008
Since there is so much already written by people more knowledgeable and intelligent than myself, why do I write?
Well, I started this blog to show my appreciation, and hopefully to contribute something back to the development community.
So kudos to all of you... keep sharing the love (of maths & programming)!
Associate with people who are searching for answers, avoid those who have found them.
Happy New Year!