Of Course NHibernate Is Slow When You Use It Incorrectly
Posted by Davy Brion on August 19th, 2009
Just saw the following post where the performance of NHibernate and Entity Framework is compared for a couple of different operations. Spoiler alert: NHibernate loses. As it typically does in these kinds of ‘benchmarks’ or ‘comparisons’ that seem to pop up frequently lately.
For some reason, a lot of people seem to think that opening an NHibernate session and performing thousands of operations is a valid use case. It’s not. Far from it actually. And with all of the features that NHibernate offers, it can’t possibly perform well in such a scenario. See, an NHibernate session is a unit of work. A unit of work is a business transaction which is typically short and small, but it should never be something huge. Your DBA probably won’t appreciate huge database transactions on the database either.
Whenever you load an object through NHibernate, it will be tracked by the session that loaded it. That means that the NHibernate session keeps a reference to it, and performs a series of checks on it periodically, depending on what you’re doing and some configuration settings such as the FlushMode. For instance, suppose you’ve loaded a hundred different entities in one session. If the FlushMode is set to automatic, it means that NHibernate will perform a dirty check for each entity associated with the session before each query is executed. The more entity instances you’ve loaded, the longer this takes (obviously). If you take this to an extreme level, like loading thousands of entities like most of these ‘benchmarks’ do, performance will naturally be horrible.
Each entity instance is also stored in the first level (or session level) cache. That means that whenever you retrieve a row from the database, NHibernate will check if an instance of that row already exists in the first level cache. Again, the more instances you’ve loaded, the larger the overhead of this will be. There are also a lot of possible extension points where you can plug in custom logic. Again, there is a very minor cost that comes with this extensibility and as you can expect, that minor cost can add up to something much more noticable once you start dealing with an unreasonably large number of instances in your session.
Always keep in mind that an ORM (and this goes for every ORM) is most suitable for OLTP. Using an ORM for batch processing jobs or large data processing operations in general is simply put a bad idea. And they will never perform as well as other solutions in these scenarios. So please don’t bother even benchmarking ORM performance in non OLTP usage because it quite simply doesn’t make sense to do so, and the results will be completely untrustworthy anyway.
An ORM can offer you nice performance gains in OLTP scenarios simply by trying to minimize database connectivity, minimizing the number of database operations, and relatively sane caching usage. Unfortunately, these are aspects that are never tested in these ‘benchmarks’ or ‘comparisons’.
August 20th, 2009 at 11:14 am
Davy,
I totally agree with you; also, I followed the (large) discussion between Oren and one of the authors of ORMBattle.NET, and I also tend to agree with him. However, I believe that is not the point on this case: both EF and NH implement the Unit of Work pattern, so the comparison between the two is valid, since they have the same purpose.
Don’t you agree?
August 20th, 2009 at 11:15 am
if there is still a performance difference between NHibernate and Entity Framework once Entity Framework has all of the same features in their Unit Of Work implementation, then i would agree
August 20th, 2009 at 3:31 pm
Hi Davy,
I’m new to NHibernate, have just played a little bit at home with it.
From what you’re saying I deduce NHibernate is not to be used on a back office system, where most of the work we do consists of reading files up, transform some data and store every line in the DB.
Am I right?
Thanks.
Alberto.
August 20th, 2009 at 3:39 pm
It’s certainly not ideal for that kind of work. If you use a stateless session instead of a regular session the performance will be acceptable, but you could always do better.
August 20th, 2009 at 3:44 pm
Thanks, it’s just I’m trying to come up with reasons to use NHibernate at work, but it seems to be hard to convince some people to move on to an ORM and forget about complex stored procedures. Have you ever used an ORM for a back office system in a bank?
August 20th, 2009 at 3:45 pm
no, because it doesn’t really make sense to do so
August 20th, 2009 at 4:47 pm
Ok, I guess I’ll have to leave NHibernate for internal tools then
August 20th, 2009 at 4:55 pm
I would disagree with two things in this post:
1. NHibernate did not lose in the comparison. I’ve found that it’s far slower when it comes to storage than Entity Framework however is much faster when deleting objects. In the other cases the speed of the two were somewhat similar. And in the conclusion I’ve said that I wouldn’t choose between these tools based on their performance as there was no clear ranking.
2. The test was not a typical real world usage indeed, it pushed the limits of the tools and tried to do this in a fair way, the source is available for you to see. Session cache is probably not used in this test at all however this was meant to be a performance comparison test, not a load test.
August 20th, 2009 at 5:58 pm
[...] Of Course NHibernate Is Slow When You Use It Incorrectly – Davy Brion [...]
August 20th, 2009 at 10:13 pm
@Gergely
as i mentioned in the post, the whole problem with benchmarks like yours (and the ones we’ve seen lately) is that they only test performance in non-realistic usage scenarios, it doesn’t really matter if it is fair to all of the compared ORM’s… if it’s not tested in typical ORM usage, then it simply isn’t relevant. Performance of ORM’s can only be measured in a meaningful way when they are compared in real-world situations. Performance differences in other scenarios are just not trustworthy.
And just for the record, if NHibernate would’ve beaten Entity Framework in most or all of the tests, i would still feel the same way about the results. They simply aren’t valid.
August 20th, 2009 at 11:41 pm
@Davy
I chatted with Gergely and fixed the issue (other than not being a real world case).
We changed FlushMode to FlushMode.Never and removed a bunch of .Flush() in record data test and enabled batching. .SaveChanges() is doing batching, so should NHibernate.
Results? Nh wins
August 21st, 2009 at 12:48 am
[...] custom data layer to Nhibernate, choices that have to be made when choosing NH for your data layer.NHibernate Slow? Of course when used wrong. Published Thursday, August 20, 2009 4:30 PM by mxmissile Filed under: [...]
August 21st, 2009 at 5:02 am
@Davy,
Not sure you can speak in absolutes that a “unit of work” should be small. For example, in a Silverlight RIA application when there is a series of changes done to the objects on the client the changes are “committed” in large transaction batch involving many entities. I found that ADO.NET Data Services with EF performs better in these scenarios than nHibernate.
Depending on your application this could easily involve a ton of entities that require multiple changes. With EF even hundreds of changes batched performs really fast. If you think about hundreds of changed entities is not hard to achieve (15-20 related tables with 5 new “somethings” fully created).
August 21st, 2009 at 9:49 am
Bart, I have enabled batching in NH too, and ensured the ID gen is not one of the bad ones. NH performed better. I am interested in your code.