Bulk Data Operations With NHibernate’s Stateless Sessions
Posted by Davy Brion on October 30th, 2008
In my previous post, i showed how you can configure NHibernate to batch create/update/delete statements and what kind of performance benefits you can get from it. In this post, we’re going to take this a bit further so we can actually use NHibernate in bulk data operations, an area where ORM’s traditionally perform pretty badly.
First of all, let’s get back to our test code from the last post:
var testObjects = CreateTestObjects(500000);
var stopwatch = new Stopwatch();
stopwatch.Start();
using (ITransaction transaction = Session.BeginTransaction())
{
foreach (var testObject in testObjects)
{
Session.Save(testObject);
}
transaction.Commit();
}
stopwatch.Stop();
var time = stopwatch.Elapsed;
The only thing that changed since the previous post is the amount of objects that are created. In the previous post we only created 10000 objects, whereas now we’ll be creating 500000 objects.
The batch size is configured like this:
<property name="adonet.batch_size">100</property>
This means that NHibernate will send its DML statements in batches of 100 statements instead of sending all of them one by one. The above code runs in 2 minutes and 24 seconds with a batch size of 100.
However, if we use NHibernate’s IStatelessionSession instead of a regular ISession, we can get some nice improvements. First of all, here’s the code to use the IStatelessSession:
var testObjects = CreateTestObjects(500000);
var stopwatch = new Stopwatch();
stopwatch.Start();
using (IStatelessSession statelessSession = sessionFactory.OpenStatelessSession())
using (ITransaction transaction = statelessSession.BeginTransaction())
{
foreach (var testObject in testObjects)
{
statelessSession.Insert(testObject);
}
transaction.Commit();
}
stopwatch.Stop();
var time = stopwatch.Elapsed;
As you can see, apart from the usage of the IStatelessSession instead of the regular ISession, this is pretty much the same code.
With a batch-size of 100, this code creates and inserts the 500000 records in 1 minute and 26 seconds. While not a spectacular improvement, it’s definitely a nice improvement in duration.
The biggest difference however is in memory usage while the code is running. A regular NHibernate ISession keeps a lot of data in its first-level cache (this enables a lot of the NHibernate magical goodies). The IStatelessSession however, does no such thing. It does no caching whatsoever and it also doesn’t fire all of the events that you could usually plug into. This is strictly meant to be used for bulk data operations.
To give you an idea on the difference in memory usage, here are the memory statistics (captured by Process Explorer) after running the original code (with the ISession instance):
And here are the memory statistics after running the modified code (with the IStatelessSession instance):
Quite a difference for what is essentially the same operation. We could even improve on this because the code in its current form keeps all of the object instances in its own collection, preventing them from being garbage collected after they have been inserted in the database. But i think this already demonstrates the value in using the IStatelessSession if you need to perform bulk operations.
Obviously, this will never perform as well as a bulk data operation that directly uses low-level ADO.NET code. But if you already have the NHibernate mappings and infrastructure set up, implementing those bulk operations could be cheaper while still being ‘fast enough’ for most situations.
October 30th, 2008 at 9:32 pm
[...] Note: this was originally posted on my own blog [...]
October 31st, 2008 at 10:32 am
[...] Bulk Data Operations With NHibernate’s Stateless Sessions – Davy Brion looks at how Stateless Sessions in NHibernate afford memory use benefits when performing bulk data operations. [...]
November 14th, 2008 at 6:59 pm
Hi Davy,
I am investigating NH batch operations on Oracle and would like to speed it up compared to ODP.NET, but when I looked at NH 2.0 sources I seems like when stateless session is used, batching has no effect, inserts are still done one by one and in my case it performs just like normal session with no batching.
Which NHibernate version did you use and which database you were working on?
Regards,
Andrzej
November 15th, 2008 at 2:19 pm
@Andrzej
I use NHibernate 2.0.x but i do use it on SQL Server… The last time i used it on Oracle was about a year ago, with NHibernate 1.2. But i actually think most of the batching functionality of NHibernate simply doesn’t work with Oracle, not sure though…
March 16th, 2010 at 3:04 pm
Something I noticed when using the stateless session is that dependent objects get not cascade as they do using the “normal” session even though cascade=”all” is set. To overcome this problem you need to loop through the dependent objects yourself, which is not a biggy. Just add the additional loops to your repository.
This is from the documentation:
“A stateless session does not implement a first-level cache nor interact with any second-level cache, nor does it implement transactional write-behind or automatic dirty checking, nor do operations cascade to associated instances. Collections are ignored by a stateless session. Operations performed via a stateless session bypass Hibernate’s event model and interceptors. Stateless sessions are vulnerable to data aliasing effects, due to the lack of a first-level cache.”