Of Course NHibernate Is Slow When You Use It Incorrectly

16 commentsWritten on August 19th, 2009 by
Categories: NHibernate, Performance

Just saw the following post where the performance of NHibernate and Entity Framework is compared for a couple of different operations. Spoiler alert: NHibernate loses. As it typically does in these kinds of 'benchmarks' or 'comparisons' that seem to pop up frequently lately.

For some reason, a lot of people seem to think that opening an NHibernate session and performing thousands of operations is a valid use case. It's not. Far from it actually. And with all of the features that NHibernate offers, it can't possibly perform well in such a scenario. See, an NHibernate session is a unit of work. A unit of work is a business transaction which is typically short and small, but it should never be something huge. Your DBA probably won't appreciate huge database transactions on the database either.

Whenever you load an object through NHibernate, it will be tracked by the session that loaded it. That means that the NHibernate session keeps a reference to it, and performs a series of checks on it periodically, depending on what you're doing and some configuration settings such as the FlushMode. For instance, suppose you've loaded a hundred different entities in one session. If the FlushMode is set to automatic, it means that NHibernate will perform a dirty check for each entity associated with the session before each query is executed. The more entity instances you've loaded, the longer this takes (obviously). If you take this to an extreme level, like loading thousands of entities like most of these 'benchmarks' do, performance will naturally be horrible.

Each entity instance is also stored in the first level (or session level) cache. That means that whenever you retrieve a row from the database, NHibernate will check if an instance of that row already exists in the first level cache. Again, the more instances you've loaded, the larger the overhead of this will be. There are also a lot of possible extension points where you can plug in custom logic. Again, there is a very minor cost that comes with this extensibility and as you can expect, that minor cost can add up to something much more noticable once you start dealing with an unreasonably large number of instances in your session.

Always keep in mind that an ORM (and this goes for every ORM) is most suitable for OLTP. Using an ORM for batch processing jobs or large data processing operations in general is simply put a bad idea. And they will never perform as well as other solutions in these scenarios. So please don't bother even benchmarking ORM performance in non OLTP usage because it quite simply doesn't make sense to do so, and the results will be completely untrustworthy anyway.

An ORM can offer you nice performance gains in OLTP scenarios simply by trying to minimize database connectivity, minimizing the number of database operations, and relatively sane caching usage. Unfortunately, these are aspects that are never tested in these 'benchmarks' or 'comparisons'.

  • http://weblogs.asp.net/ricardoperes Ricardo Peres

    Davy,

    I totally agree with you; also, I followed the (large) discussion between Oren and one of the authors of ORMBattle.NET, and I also tend to agree with him. However, I believe that is not the point on this case: both EF and NH implement the Unit of Work pattern, so the comparison between the two is valid, since they have the same purpose.

    Don’t you agree?

  • http://davybrion.com Davy Brion

    if there is still a performance difference between NHibernate and Entity Framework once Entity Framework has all of the same features in their Unit Of Work implementation, then i would agree ;)

  • Alberto

    Hi Davy,

    I’m new to NHibernate, have just played a little bit at home with it.
    From what you’re saying I deduce NHibernate is not to be used on a back office system, where most of the work we do consists of reading files up, transform some data and store every line in the DB.

    Am I right?

    Thanks.
    Alberto.

  • http://davybrion.com Davy Brion

    It’s certainly not ideal for that kind of work. If you use a stateless session instead of a regular session the performance will be acceptable, but you could always do better.

  • Alberto

    Thanks, it’s just I’m trying to come up with reasons to use NHibernate at work, but it seems to be hard to convince some people to move on to an ORM and forget about complex stored procedures. Have you ever used an ORM for a back office system in a bank?

  • http://davybrion.com Davy Brion

    no, because it doesn’t really make sense to do so :)

  • Alberto

    Ok, I guess I’ll have to leave NHibernate for internal tools then :-(

  • http://gregdoesit.com Gergely Orosz

    I would disagree with two things in this post:
    1. NHibernate did not lose in the comparison. I’ve found that it’s far slower when it comes to storage than Entity Framework however is much faster when deleting objects. In the other cases the speed of the two were somewhat similar. And in the conclusion I’ve said that I wouldn’t choose between these tools based on their performance as there was no clear ranking.

    2. The test was not a typical real world usage indeed, it pushed the limits of the tools and tried to do this in a fair way, the source is available for you to see. Session cache is probably not used in this test at all however this was meant to be a performance comparison test, not a load test.

  • Pingback: Arjan’s World » LINKBLOG for August 20, 2009

  • http://davybrion.com Davy Brion

    @Gergely

    as i mentioned in the post, the whole problem with benchmarks like yours (and the ones we’ve seen lately) is that they only test performance in non-realistic usage scenarios, it doesn’t really matter if it is fair to all of the compared ORM’s… if it’s not tested in typical ORM usage, then it simply isn’t relevant. Performance of ORM’s can only be measured in a meaningful way when they are compared in real-world situations. Performance differences in other scenarios are just not trustworthy.

    And just for the record, if NHibernate would’ve beaten Entity Framework in most or all of the tests, i would still feel the same way about the results. They simply aren’t valid.

  • http://tunatoksoz.com Tuna toksoz

    @Davy
    I chatted with Gergely and fixed the issue (other than not being a real world case).
    We changed FlushMode to FlushMode.Never and removed a bunch of .Flush() in record data test and enabled batching. .SaveChanges() is doing batching, so should NHibernate.

    Results? Nh wins :)

  • Pingback: NHibernate Talk 8/20/09 - Travis.Net.Blog

  • http://www.silverlighthack.com Bart Czernicki

    @Davy,

    Not sure you can speak in absolutes that a “unit of work” should be small. For example, in a Silverlight RIA application when there is a series of changes done to the objects on the client the changes are “committed” in large transaction batch involving many entities. I found that ADO.NET Data Services with EF performs better in these scenarios than nHibernate.

    Depending on your application this could easily involve a ton of entities that require multiple changes. With EF even hundreds of changes batched performs really fast. If you think about hundreds of changed entities is not hard to achieve (15-20 related tables with 5 new “somethings” fully created).

  • http://tunatoksoz.com Tuna toksoz

    Bart, I have enabled batching in NH too, and ensured the ID gen is not one of the bad ones. NH performed better. I am interested in your code.

  • Larry Liston

    I would have to agree withe Bart regarding the “unit of work” statement. The application should determine this based on requirements and environment. It would seem that if performance was the #1 concern, an ORM (and it associated overhead) might not be advisable anyway. Regardless, loading a collection, dirtying them up, and shoving them back in the database is a real world scenario and a useful metric as long as the playing field is level.

  • Dato1100

    Great post, I just made a little write “benchmark” and was curious why NHibernate was hundred times slower than EF. After simulating a more OLTP scenario (that is, disposing ISession everytime a transaction commits) I’ve got better performance than EF :)