The Inquisitive Coder - Davy Brion’s Blog

Thinking outside of the typical .NET box

Archive for the 'NHibernate' Category


Batching++

Posted by Davy Brion on 30th July 2008

The usual example:

                dispatcher.Add(new GetProductCategoriesRequest(), new GetSuppliersRequest());

                View.ProductCategories = dispatcher.Get<GetProductCategoriesResponse>().ProductCategories;

                View.Suppliers = dispatcher.Get<GetSuppliersResponse>().Suppliers;

                View.DataBind();

Thanks to my request/response service layer, those are two service requests that will be performed in one service call. On the server side, these two requests are dealt with separately. But they merely retrieve data from the database. Wouldn’t it be cool if those 2 queries were performed in one database roundtrip instead of two? Wouldn’t it be cool if we could batch the queries that will be performed by read-only service requests into a single database roundtrip? It sure as hell would be. Implementing this was one of those irresistible challenges you just can’t say no to, so from now on we can actually do this. I’m not gonna get into the actual implementation of how I got it working (check the code in the library for that if you’re interested), but i will show you how you can do this in your application.

First of all, if you have requests that merely return data, inherit from ReadOnlyRequest instead of the usual Request type:

    [Serializable]

    public class GetProductCategoriesRequest : ReadOnlyRequest {}

 

    [Serializable]

    public class GetSuppliersRequest : ReadOnlyRequest {}

The handlers for these requests should inherit from the ReadOnlyRequestHandler base class instead of the usual RequestHandler class. Btw, my ReadOnlyRequestHandler class inherits from my UoWRequestHandler class, which requires a IUnitOfWork instance to be passed to the constructor. So we need to put an IUnitOfWork parameter into the constructor along with our other dependencies if we want the IOC container (or the request processor) to pass us a valid IUnitOfWork instance.

    public class GetProductCategoriesHandler : ReadOnlyRequestHandler<GetProductCategoriesRequest>

    {

        private readonly IRepository<ProductCategory> repository;

 

        public GetProductCategoriesHandler(IUnitOfWork unitOfWork, IRepository<ProductCategory> repository)

            : base(unitOfWork)

        {

            this.repository = repository;

        }

 

        public override void AddQueries(IQueryBatcher queryBatcher, GetProductCategoriesRequest request)

        {

            queryBatcher.AddCriteria(“categories”, repository.CriteriaForAll());

        }

 

        public override Response GetResults(IQueryBatcher queryBatcher, GetProductCategoriesRequest request)

        {

            var categories = queryBatcher.GetEnumerableResult<ProductCategory>(“categories”);

 

            return new GetProductCategoriesResponse { ProductCategories = categories.ToDTOs() };

        }

    }

The ReadOnlyRequestHandler class defines two abstract methods: the AddQueries and GetResults method. Both are passed an instance of IQueryBatcher and the actual request object so you can use the request parameters if there are any.

In the AddQueries method, you simply add the queries this request needs to perform to the batcher (with a string key value of course for later retrieval), and in the GetResults method you can get the results (with the key values used in AddQueries) back from the batcher and you can construct your Response object. The GetResults method will be called after each ReadOnlyRequestHandler for the current request batch has added the queries to be performed to the batcher. The queries are then all executed, and each ReadOnlyRequestHandler will have its GetResults method called so each handler can construct the proper response.

And that’s it basically… The calling code doesn’t even change.

Now there are obviously some limitations. If you mix ReadOnlyRequests with other Requests, then each ReadOnlyRequest will be handled as if it were a regular Request and they will each get their very own IQueryBatcher instance. I don’t think there’s any strategy to automagically determine which ReadOnlyRequests can be batched together while still correctly executing the other Requests since those requests might influence the results of ReadOnlyRequests that are present further in the batch.

Anyways, you can find an updated version of the library here. Check it out, it’s some pretty slick shit, if i do say so myself ;)

Note: i only added this stuff today, so the previous build (080729) doesn’t have this yet, in case you only downloaded that today, and the stats show that some of you actually did download it ;)

Posted in NHibernate, Performance, Software Development, WCF | 1 Comment »

IMultiQuery/IMultiCriteria usability improvement

Posted by Davy Brion on 9th July 2008

A while ago i posted my query batcher which allowed you to use IMultiCriteria/IMultiQuery with key values instead of relying on the index position of the results.

I was asked to add those capabilities to NHibernate, so i submitted a patch for IMultiCriteria first, which was applied today. So i also submitted the patch for IMultiQuery.

So now you can basically do this:

            using (ISession session = OpenSession())

            {

                IMultiCriteria multiCriteria = session.CreateMultiCriteria();

 

                DetachedCriteria firstCriteria = DetachedCriteria.For(typeof(Item))

                    .Add(Expression.Lt(“id”, 50));

 

                DetachedCriteria secondCriteria = DetachedCriteria.For(typeof(Item));

 

                multiCriteria.Add(“firstCriteria”, firstCriteria);

                multiCriteria.Add(“secondCriteria”, secondCriteria);

 

                IList secondResult = (IList)multiCriteria.GetResult(“secondCriteria”);

                IList firstResult = (IList)multiCriteria.GetResult(“firstCriteria”);

 

                Assert.Greater(secondResult.Count, firstResult.Count);

            }

Same thing for IMultiQuery, but with hql queries instead of criteria obviously.

This has been added to the trunk, so i’m not sure if this will be ported to the NH2.0 branch, but at least it should be available for NH2.1 :)

update: the second patch has also been applied

Posted in NHibernate | No Comments »

Data Access With NHibernate

Posted by Davy Brion on 23rd June 2008

One thing that keeps amazing me is how many smart developers still feel the urge to write their own data access layer (DAL). They either do it all manually, or they generate parts of it, or they generate the whole thing. Whichever way you go, there are still various alternative paths you can choose from. Some people like to use stored procedures for everything, some people generate sql statements and provide a semi-OO API in front of it, some people still spend time constructing Command objects. I think i’ve seen most approaches by now, and my personal opinion is that they pretty much all suck.

These DAL’s usually have at least one big downside to them. Generated DAL’s are usually pretty good for productivity but most of the times they don’t really offer you that much control over how queries should be executed, whether or not statements should be batched, specify fetching-strategies, etc. Hand-written DAL’s are terrible for productivity but you can fine tune each action to an optimal implementation. These downsides are pretty big IMHO, and are often underestimated. Sometimes due to semi-religious stances and sometimes it’s just due to ignorance.

As you already know, i’m a fan of NHibernate. Well, consider me a fan of decent ORM’s in general. NHibernate just happens to be the one i know best and am very happy with. In this post i’d like to take a look at how you can easily offer the most common data access requirements without generating code, while still allowing high developer productivity. We’ll go over the implementation of a generic Repository class, and hopefully you’ll see how much NHibernate can improve your way of working.

We’re going to look at the implementation piece by piece, so lets just start with the declaration of the class so we can get that out of the way:

    public class Repository<T> : IRepository<T>

As you can see, this is just a generic class that takes a type parameter. The type parameter represents the type of the Entity you want this repository to handle. If you have an Entity base class or interface, you probably want to restrict the type of T to inherit from Entity or implement IEntity or whatever.

This class needs to be able to access the current NHibernate session, which i discussed yesterday:

        private readonly IActiveSessionManager activeSessionManager;

 

        public Repository(IActiveSessionManager activeSessionManager)

        {

            this.activeSessionManager = activeSessionManager;

        }

 

        protected ISession Session

        {

            get { return activeSessionManager.GetActiveSession(); }

        }

Right, now we can actually get to the interesting Data Access parts. Obviously, each DAL needs a way to retrieve a specific entity based on its Primary Key value:

        /// <summary>

        /// Retrieves the entity with the given id

        /// </summary>

        /// <param name=”id”></param>

        /// <returns>the entity or null if it doesn’t exist</returns>

        public T Get(object id)

        {

            return Session.Get<T>(id);

        }

Very straightforward stuff… this simply uses the current NHibernate session to retrieve an entity of the requested type, with the given primary key value.

Another thing we need is a way to create or update entity instances:

        /// <summary>

        /// Saves or updates the given entity

        /// </summary>

        /// <param name=”entity”></param>

        public void SaveOrUpdate(T entity)

        {

            Session.SaveOrUpdate(entity);

        }

This method simply uses the session’s SaveOrUpdate method, which will either perform an Insert (in case of a new entity) or an Update (in case of an existing entity). NHibernate will perform a check for a new instance depending on how you’ve configured this in your mapping files.

So far this has been really straightforward, so it’s time to get to a more interesting part. Retrieving entities based on criteria (basically a query):

        /// <summary>

        /// Returns each entity that matches the given criteria

        /// </summary>

        /// <param name=”criteria”></param>

        /// <returns></returns>

        public IEnumerable<T> FindAll(DetachedCriteria criteria)

        {

            return criteria.GetExecutableCriteria(Session).List<T>();

        }

This probably needs a bit of explaining. The parameter is an instance of the DetachedCriteria type. A DetachedCriteria is just like a Criteria instance, except that it hasn’t been associated with a session yet. So you can create the DetachedCriteria without being connected to a session. This basically represents a query. I’ll show a concrete example of this later on. The thing to remember is that this is the only code you really need to perform any query you want. You just have to write the query using the Criteria API. This does have a bit of a learning curve, but most people pick it up pretty fast.

You usually also want a way to determine the ordering of the result of the query:

        /// <summary>

        /// Returns each entity that maches the given criteria, and orders the results

        /// according to the given Orders

        /// </summary>

        /// <param name=”criteria”></param>

        /// <param name=”orders”></param>

        /// <returns></returns>

        public IEnumerable<T> FindAll(DetachedCriteria criteria, params Order[] orders)

        {

            if (orders != null)

            {

                foreach (var order in orders)

                {

                    criteria.AddOrder(order);

                }

            }

 

            return FindAll(criteria);

        }

Each order you provide is applied to the query and then the query is executed. Again, pretty simple stuff right?

You want to know how you can get paging working? Here it is:

        /// <summary>

        /// Returns each entity that matches the given criteria, with support for paging,

        /// and orders the results according to the given Orders

        /// </summary>

        /// <param name=”criteria”></param>

        /// <param name=”firstResult”></param>

        /// <param name=”numberOfResults”></param>

        /// <param name=”orders”></param>

        /// <returns></returns>

        public IEnumerable<T> FindAll(DetachedCriteria criteria, int firstResult, int numberOfResults, params Order[] orders)

        {

            criteria.SetFirstResult(firstResult).SetMaxResults(numberOfResults);

            return FindAll(criteria, orders);

        }

Keep in mind that the executed query only retrieves the results within the paging range. It does not retrieve everything to perform the paging client-side, this happens in the db where it’s supposed to happen.

What if you have a query that should only return one instance? That’s easy to do as well:

        /// <summary>

        /// Returns the one entity that matches the given criteria. Throws an exception if

        /// more than one entity matches the criteria

        /// </summary>

        /// <param name=”criteria”></param>

        /// <returns></returns>

        public T FindOne(DetachedCriteria criteria)

        {

            return criteria.GetExecutableCriteria(Session).UniqueResult<T>();

        }

What if you have a query and you just want the very first result instead of the entire resultset? Again, pretty easy to do:

        /// <summary>

        /// Returns the first entity to match the given criteria

        /// </summary>

        /// <param name=”criteria”></param>

        /// <returns></returns>

        public T FindFirst(DetachedCriteria criteria)

        {

            var results = criteria.SetFirstResult(0).SetMaxResults(1)

                .GetExecutableCriteria(Session).List<T>();

 

            if (results.Count > 0)

            {

                return results[0];

            }

 

            return default(T);

        }

Again, NHibernate will issue a smart sql statement… that is, it only retrieves the first result instead of the entire resultset.

Obviously, this is also useful if you can define the order of the results to pick the first result:

        /// <summary>

        /// Returns the first entity to match the given criteria, ordered by the given order

        /// </summary>

        /// <param name=”criteria”></param>

        /// <param name=”order”></param>

        /// <returns></returns>

        public T FindFirst(DetachedCriteria criteria, Order order)

        {

            return FindFirst(criteria.AddOrder(order));

        }

How often have you seen developers execute a query in code, only to use the count of the records without actually needing the returned records? We no longer need to beat the shit out of these developers:

        /// <summary>

        /// Returns the total number of entities that match the given criteria

        /// </summary>

        /// <param name=”criteria”></param>

        /// <returns></returns>

        public long Count(DetachedCriteria criteria)

        {

            return Convert.ToInt64(criteria.GetExecutableCriteria(Session)

                .SetProjection(Projections.RowCountInt64()).UniqueResult());

        }

In this case, NHibernates issues a simple select count… query based on the criteria you’ve provided. Nice huh?

We might as well throw in this one as well:

        /// <summary>

        /// Returns true if at least one entity exists that matches the given criteria

        /// </summary>

        /// <param name=”criteria”></param>

        /// <returns></returns>

        public bool Exists(DetachedCriteria criteria)

        {

            return Count(criteria) > 0;

        }

And last, but certainly not least, you’ll also want a way to delete entities from the database. How about this:

        /// <summary>

        /// Deletes the given entity

        /// </summary>

        /// <param name=”entity”></param>

        public void Delete(T entity)

        {

            Session.Delete(entity);

        }

 

        /// <summary>

        /// Deletes every entity that matches the given criteria

        /// </summary>

        /// <param name=”criteria”></param>

        public void Delete(DetachedCriteria criteria)

        {

            // a simple DELETE FROM … WHERE … would be much better, but i haven’t found

            // a way to do this yet with Criteria. So now it does two roundtrips… one for

            // the query, and one with all the batched delete statements (that is, if you’ve

            // enabled CUD statement batching

            foreach (T entity in FindAll(criteria))

            {

                Delete(entity);

            }

        }

The first Delete method simply deletes the given entity. The second method probably needs to be explained a bit more. This method receives a query, and it deletes all the items that the query returns. As you can see from the comment, it would be better if it would perform a DELETE FROM … WHERE instead of fetching the results of the query but i didn’t find a way to do that with the criteria API. In a more advanced scenario, this might actually be better than simply issuing a large DELETE statement because you could offer a Delete method which also receives a block of code to execute before or after each delete is executed. Which opens the door to a lot of interesting opportunities.

And that’s it basically… I really haven’t shown you that much code right? And what does this code offer us? We get create/update/delete functionality, and we also have some nice options for querying our data. And since the Criteria API of NHibernate allows you to create powerful and complex queries, you can use it to create your queries and then you simply pass these criteria to the repository to fetch the data. Just so we’re clear on this, you’re not just limited to specifying which data you want to retrieve, but you can also tell NHibernate how you want to retrieve it because you can define fetching strategies for each association. This is a tremendously powerful feature which offers you a lot of flexibility in choosing the most optimal data fetching approaches, without being limited to what your DAL supports or having to spend a lot of code on it.

Let’s wrap up this post with a small example of how you could use this repository class to execute a query you wrote yourself:

            var criteria = DetachedCriteria.For<ProductCategory>()

                .Add(Expression.Like(“Name”, “Test%”));

 

            var categories = repository.FindAll(criteria);

As you can see, this is really easy. The only effort basically lies within building the query, so as you’re queries become more complex, this effort obviously increases.

If you combine this repository approach with query batching, you end up with an easy-to-use data layer which offers you all the flexibility you could want, while still allowing you to implement specifically tuned solutions to achieve excellent performance. Also, keep in mind that this is only a very basic repository implementation. The Rhino Commons repository implementation offers even more functionality with a couple of extra options to boost performance. I really can’t think of a single good reason not to use this approach anymore.

Posted in NHibernate | 12 Comments »

Managing your NHibernate Sessions

Posted by Davy Brion on 22nd June 2008

I was working on my northwind sample application (which i’ll post about as soon as i get something that’s worth showing) and i was struggling to find a clean way to manage my NHibernate sessions. I wanted a way to create a session once you enter the service layer, and that session should be available transparently to the repository implementations without actually having to constantly pass it around. But i didn’t just want to store it somewhere and have all the classes that needed the current session get it directly from that place. Also, i didn’t want the current session to be available to everyone. Another important requirement was to have maximum flexibility for writing tests. I could just use Rhino-Commons’ implementations of the UnitOfWork and Repository patterns and be done with it, but where’s the fun in that? And since this application is mostly a learning experience i figured i should try to write this myself. After a bit of searching and experimenting, i finally came up with a way that i’m happy with (for now anyways). Let’s have a look.

To illustrate what i want, take a look at this made-up method in my service layer:

        public ProductCategoryDTO[] GetAllProductCategories()

        {

            using (ISession session = GetNewSession())

            {

                var repository = GetProductCategoryRepository();

                var categories = repository.GetAllProductCategories();

                return categories.ToDTOs();

            }

        }

This is just some example code, it doesn’t even work. But it’s kinda what i want… The thing is, i don’t want to deal directly with the ISession type in the service layer, but i do want the ProductCategoryRepository to use the ISession that is created within the context of this method call.

NHibernate’s ISession type is an implementation of the Unit Of Work pattern. I don’t want to use the ISession directly in my service layer because i want something that is a bit more high-level. Basically just something that would allow me to work within an NHibernate session, and provide me with the ability to flush changes to the database whenever i want to. And obviously, i want it to allow me to create a transaction as well. So i came up with the following unit of work interface:

    public interface IUnitOfWork : IDisposable

    {

        /// <summary>

        /// Flushes any changes that haven’t been executed yet

        /// </summary>

        void Flush();

 

        /// <summary>

        /// Creates an ITransaction instance with the ReadCommitted isolation level

        /// </summary>

        /// <returns></returns>

        ITransaction CreateTransaction();

 

        /// <summary>

        /// Creates an ITransaction instance with the given isolation level

        /// </summary>

        /// <param name=”isolationLevel”></param>

        /// <returns></returns>

        ITransaction CreateTransaction(IsolationLevel isolationLevel);

    }

This interface pretty much offers me anything i’m concerned with in my service layer. The real implementation would do a bit more though… It has to create an NHibernate session and make it available to the repositories in a clean way. Here’s the thing though… the repositories have a completely different lifetime than the NHibernate sessions. The NHibernate session has to be created when we enter a service method, and it has to be valid within the scope and context of the execution of that service method. The repositories however stay alive for the lifetime of the application so they can’t just hold a reference to an NHibernate session. Every time you call a method of a repository, it has to find out which session it should use, which is the session that is being used in the context of the current service method call. Now, we could always pass around the session but that really doesn’t look good and is cumbersome. So i basically need something that gives me access to the active nhibernate session:

    public interface IActiveSessionManager

    {

        /// <summary>

        /// Returns the active ISession for the current thread. Throws exception if there’s

        /// no active ISession instance

        /// </summary>

        /// <returns></returns>

        ISession GetActiveSession();

 

        /// <summary>

        /// Sets the active ISession for the current thread. Throws exception if there’s

        /// already an active ISession instance

        /// </summary>

        /// <param name=”session”></param>

        void SetActiveSession(ISession session);

 

        /// <summary>

        /// Clears the active ISession for the current thread.

        /// </summary>

        void ClearActiveSession();

    }

That gives me the ability to retrieve and set the active session on a per-thread basis. After all, a service method call will be handled by one thread, so setting the active session for that current thread is a convenient way to store the session.

Now i still need something that takes care of creating the NHibernate sessions:

    public interface ISessionFactory

    {

        /// <summary>

        /// Creates a new ISession instance

        /// </summary>

        /// <returns></returns>

        ISession Create();

    }

Ok, so what do we have so far? An interface to define the functionality that a UnitOfWork should offer at the service level. An interface to store/retrieve the active session on the current thread, and finally, an interface to actually create the NHibernate session. Let’s start looking at the real implementations. Here’s the code to the SessionFactory class:

    public class SessionFactory : ISessionFactory

    {

        private readonly NHibernate.ISessionFactory sessionFactory;

 

        public SessionFactory()

        {

            Configuration configuration = new Configuration()

                .Configure()

                .AddAssembly(“Northwind”);

 

            sessionFactory = configuration.BuildSessionFactory();

        }

 

        public ISession Create()

        {

            return sessionFactory.OpenSession();

        }

    }

Pretty straightforward… it initializes NHibernate and gives us the ability to ask for new sessions. So how are we going to make these sessions available to our repositories? Through the ActiveSessionManager class of course:

    public class ActiveSessionManager : IActiveSessionManager

    {

        [ThreadStatic]

        private static ISession current;

 

        public ISession GetActiveSession()

        {

            if (current == null)

            {

                throw new InvalidOperationException(“There is no active ISession instance for this thread”);

            }

 

            return current;

        }

 

        public void SetActiveSession(ISession session)

        {

            if (current != null)

            {

                throw new InvalidOperationException(“There is already an active ISession instance for this thread”);

            }

 

            current = session;

        }

 

        public void ClearActiveSession()

        {

            current = null;

        }

    }

It basically just stores the session in a ThreadStatic field, which means that each thread will have a different static reference for this field. So if we set the active session through the SetActiveSession method in thread X, and thread Y also sets an active session, the GetActiveSession method will return the correct session instances for each thread.

So now we have everything we need to create our UnitOfWork class:

    public class UnitOfWork : Disposable, IUnitOfWork

    {

        private readonly IActiveSessionManager activeSessionManager;

        private readonly ISession session;

 

        public UnitOfWork(ISessionFactory sessionFactory, IActiveSessionManager activeSessionManager)

        {

            this.activeSessionManager = activeSessionManager;

            session = sessionFactory.Create();

            activeSessionManager.SetActiveSession(session);

        }

 

        protected override void DisposeObjects()

        {

            if (session != null)

            {

                session.Close();

                session.Dispose();

            }

        }

 

        protected override void ClearReferences()

        {

            activeSessionManager.ClearActiveSession();

        }

 

        public void Flush()

        {

            session.Flush();

        }

 

        public ITransaction CreateTransaction()

        {

            return CreateTransaction(IsolationLevel.ReadCommitted);

        }

 

        public ITransaction CreateTransaction(IsolationLevel isolationLevel)

        {

            return session.BeginTransaction(isolationLevel);

        }

    }

When the UnitOfWork is created, it receives an ISessionFactory instance, and an IActiveSessionManager instance. It then creates a new session through the ISessionFactory and uses the IActiveSessionManager to make sure that session is the active session for the current thread. When the UnitOfWork is disposed, it closes and cleans up the session and it also uses the IActiveSessionManager to clear the active session for the current thread. Oh and it obviously also provides implementations for what it is we actually need in our service layer: flushing the changes whenever we want and creating transactions.

The ISessionFactory and IActiveSessionManager instances should stay alive as long as the application is alive. But as you could see in the implementations of those types, we didn’t write any code to deal with their lifetimes. I’m actually relying on my IoC container for that. In the class where my container is set up, you’ll find the following code:

        private static void RegisterUnitOfWorkComponents()

        {

            Register(Component.For<ISessionFactory>()

                .ImplementedBy<SessionFactory>().LifeStyle.Singleton);

            Register(Component.For<IActiveSessionManager>()

                .ImplementedBy<ActiveSessionManager>().LifeStyle.Singleton);

            Register(Component.For<IUnitOfWork>()

                .ImplementedBy<UnitOfWork>().LifeStyle.Transient);

        }

ISessionFactory and IActiveSessionManager are registered as singleton instances, so whenever these types are requested, the same instances will be returned. The IUnitOfWork type is registered with a transient lifetime, so whenever it is requested, the container will create a new UnitOfWork class and pass the ISessionFactory and IActiveSessionManager instances to the constructor.

Right, we’ve taken care of creating the session, associating it with the current thread and making it available in a nice and clean way. Now we actually have to make sure our repositories can use it. In the base repository implementation, you can find the following code:

        private readonly IActiveSessionManager activeSessionManager;

 

        public Repository(IActiveSessionManager activeSessionManager)

        {

            this.activeSessionManager = activeSessionManager;

        }

 

        protected ISession Session

        {

            get { return activeSessionManager.GetActiveSession(); }

        }

Whenever a repository needs a session, it just needs to use the protected Session property and it will get the session that is associated with the current thread.

So now we can rewrite our made-up service method from earlier to the following:

        public ProductCategoryDTO[] GetAllProductCategories()

        {

            using (Container.Resolve<IUnitOfWork>())

            {

                var repository = Container.Resolve<ProductCategoryRepository>();

                var categories = repository.GetAllProductCategories();

                return categories.ToDTOs();

            }

        }

We know that the NHibernate session will be created, and more importantly, that it is not just globally accessible to everyone. It can only be accessed when you have a reference to an IActiveSessionManager instance. We also don’t need to pass around the session all the time so our code is a bit more concise, showing only the intent of what we’re trying to do without distracting us with details that are not relevant to that intent. We also have a lot of flexibility to write tests… i can write tests for my repositories without having to create a UnitOfWork… i can simply pass a fake IActiveSessionManager to my repositories when i’m testing them and have it return the session that i’m using for my test. All in all, this approach offers me with a lot of advantages, without ugly disadvantages. Well, at this moment i don’t really see any disadvantages so if you do see some, please let me know :)

Posted in Dependency Injection, Inversion Of Control, NHibernate, Patterns | 10 Comments »

The Query Batcher

Posted by Davy Brion on 14th June 2008

One of my favorite NHibernate features must be the MultiCriteria/MultiQuery support. This basically allows you to execute a set of queries in one database call. In case you don’t know, an ICriteria instance is basically a programmatic query. If you use MultiCriteria, you can batch either ICriteria or DetachedCriteria instances. And if you use MultiQuery, you can batch regular HQL strings, IQuery instances (which are in fact also HQL queries) or references to named queries.

That gives you quite a few options to batch your queries. The only downside to using MultiCriteria and/or MultiQuery is that you have to retrieve the results with an index, based on the order in which you added the criteria or queries. Using index values in your code often reduces readability, so if i can i try to avoid using them.

So i wrote a little QueryBatcher class which allows me to retrieve the results based on a key value. The class supports MultiCriteria as well as MultiQuery so you can use it to batch all kinds of queries. I didn’t find a way to combine the batched criteria and the batched HQL queries in one database call, so if you mix criteria with hql queries, it will use one database call to execute all the criteria, and one database call to execute the hql queries.

And now i can write code like this:

        [Test]

        public void ReturnsProperResultsWhenUsingMultipleQueries()

        {

            var queryBatcher = new QueryBatcher(Session);

 

            queryBatcher.AddCriteria(“Products”, Session.CreateCriteria(typeof(Product)));

            queryBatcher.AddCriteria(“Customers”, DetachedCriteria.For<Customer>());

            queryBatcher.AddHqlQuery(“Orders”, “from Order”);

            queryBatcher.AddHqlQuery(“Employees”, Session.CreateQuery(“from Employee”));

 

            queryBatcher.ExecuteQueries();

 

            Assert.That(queryBatcher.GetResult<IList>(“Orders”).Cast<Order>().Count() > 0);

            Assert.That(queryBatcher.GetResult<IList>(“Customers”).Cast<Customer>().Count() > 0);

            Assert.That(queryBatcher.GetResult<IList>(“Products”).Cast<Product>().Count() > 0);

            Assert.That(queryBatcher.GetResult<IList>(“Employees”).Cast<Employee>().Count() > 0);

        }

Obviously, this is a very simple example with extremely simple queries… but you can of course batch very complex queries with this as well. You can also add queries that return scalar values so you could for instance do do something like queryBatcher.GetResult<long>(”TotalOutstandingAmount”) and it would return that scalar value.

This is the code of the QueryBatcher class:

    public class QueryBatcher

    {

        private readonly Dictionary<string, int> criteriaResultPositions;

        private readonly IMultiCriteria multiCriteria;

        private readonly IMultiQuery multiQuery;

        private readonly Dictionary<string, int> queryResultPositions;

        private readonly ISession session;

 

        private int criteriaIndex;

        private IList criteriaResults;

        private int queryIndex;

        private IList queryResults;

 

        public QueryBatcher(ISession session)

        {

            this.session = session;

            multiCriteria = session.CreateMultiCriteria();

            multiQuery = session.CreateMultiQuery();

            criteriaResultPositions = new Dictionary<string, int>();

            queryResultPositions = new Dictionary<string, int>();

            criteriaIndex = 0;

            queryIndex = 0;

        }

 

        public void ExecuteQueries()

        {

            if (criteriaIndex > 0) criteriaResults = multiCriteria.List();

            if (queryIndex > 0) queryResults = multiQuery.List();

        }

 

        public T GetResult<T>(string key)

        {

            object result = GetResultFromList(key, criteriaResults, criteriaResultPositions);

            if (result != null) return (T)result;

            result = GetResultFromList(key, queryResults, queryResultPositions);

            if (result != null) return (T)result;

 

            return default(T);

        }

 

        public void AddCriteria(string key, ICriteria criteria)

        {

            multiCriteria.Add(criteria);

            criteriaResultPositions.Add(key, criteriaIndex);

            criteriaIndex++;

        }

 

        public void AddCriteria(string key, DetachedCriteria detachedCriteria)

        {

            AddCriteria(key, detachedCriteria.GetExecutableCriteria(session));

        }

 

        public void AddHqlQuery(string key, string hqlQuery)

        {

            multiQuery.Add(hqlQuery);

            queryResultPositions.Add(key, queryIndex);

            queryIndex++;

        }

 

        public void AddHqlQuery(string key, IQuery query)

        {

            multiQuery.Add(query);

            queryResultPositions.Add(key, queryIndex);

            queryIndex++;

        }

 

        private static object GetResultFromList(string key, IList list, IDictionary<string, int> positions)

        {

            if (positions.ContainsKey(key)) return list[positions[key]];

            return null;

        }

    }

Posted in NHibernate, Performance | No Comments »

Efficient data fetching

Posted by Davy Brion on 8th June 2008

Suppose you have the following table structure:

And we have the following classes:

What is the most efficient way to retrieve all Product instances, fully populated (with complete Supplier and ProductCategory references)? There are a couple of ways to retrieve this data, depending on what kind of data access techniques you’re using. These are the goals we should aim to achieve:

  • We only want one roundtrip to the database
  • No joins… Depending on the amount of data in each of the tables, a 3-table join could easily lead to a query which is way too expensive for what we’re trying to achieve
  • We don’t want to write boring code to construct the object graph. The object graph should be built up automagically

Ok… let’s see. Since we only want one trip to the database, that rules out using lazy loading (which would be a terrible idea in this case anyway). And we don’t want to join either, so our best bet is probably to fetch all the categories, all the suppliers, and all the products with 3 simple queries. Obviously, we’d have to batch those 3 queries so the 3 results are retrieved with only one roundtrip. And then we need to figure out a way to join the results together in an easy to use object graph.

Luckily for me i’m using NHibernate which makes all of this incredibly easy. First, we create the 3 queries:

            IMultiCriteria multiCriteria = Session.CreateMultiCriteria();

 

            multiCriteria.Add(Session.CreateCriteria(typeof(ProductCategory)));

            multiCriteria.Add(Session.CreateCriteria(typeof(Supplier)));

            multiCriteria.Add(Session.CreateCriteria(typeof(Product)));

A Criteria instance is kinda like a programmatic query. If you create a Criteria that is only based on the type of an object, you basically create a query which returns each record from the table that is associated with that type without any other conditions. In the code above, we create 3 queries this way to retrieve all the instances of the ProductCategory, Supplier and Product types. At this point, the queries have only been created, they haven’t been executed yet.

To retrieve the results of these 3 queries with one database call, we simply do this:

            IList results = multiCriteria.List();

When this line is executed, we can see the following happening on the database (through SQL Server Profiler):

Alright, so we’ve accomplished the goal of using only one database roundtrip without using joins. Now, we want to use the data we retrieved without having to write boring code to make sure each product refers to the correct ProductCategory and Supplier references. Actually, we don’t really have to do anything for that. The list of products was the third Criteria instance that we added, so it is also the third item in the result list.

            IEnumerable<Product> products = ((IList)results[2]).Cast<Product>();

Ok, so now we have a list of Products… Now how do we make sure that each product points to the correct ProductCategory and Supplier instances? Fortunately, we don’t even have to worry about that. When we access a Product’s Category or Supplier properties, NHibernate first checks its current session’s identity map to see if those objects are already present in the session’s first-level cache. Because we retrieved the ProductCategories and the Suppliers, all of them are already present. So if we would write the following code, no more extra queries would be executed:

            foreach (Product product in products)

            {

                string dummy = product.Name + ” - “ +

                    product.Supplier.CompanyName + ” - “ +

                    product.Category.Name;

            }

This code is pretty much pointless, but it does show that we can simply access whatever data we need, without having to write boring code, and do it in an efficient way as well.

Now, you might be thinking "who cares if you retrieve the data in one roundtrip instead of three?". Well, you should care about that stuff... what if you used this technique throughout your application (where applicable of course)? Even if the data you need to retrieve is not related, this is still a potentially large performance improvement if you simply try to batch statements wherever you can. The more you reduce the number of network roundtrips, the fewer time your application spends waiting on data to cross the wire. And as your number of concurrent users increases, you really want to minimize the places in your code base where the application is just waiting on something.

Posted in NHibernate, Performance | No Comments »

NHibernate and virtual methods/properties

Posted by Davy Brion on 24th May 2008

I love NHibernate but one of the things that bothers the hell out of me is that i keep forgetting to add the virtual keyword to each method or property in my entities. And since NHibernate needs your classes’ properties and methods to be virtual, this causes run-time errors when i run my tests. Since i’m already using custom compile time checks, i figured i might as well add another one… from now on, i want my compilation to fail if any of my NHibernate entities have public methods/properties that aren’t marked virtual.

Once again, it’s PostSharp to the rescue:

using System;

using System.Reflection;

 

using PostSharp.Extensibility;

using PostSharp.Laos;

 

namespace Northwind.Aspects

{

    [Serializable]

    [AttributeUsage(AttributeTargets.Assembly | AttributeTargets.Method | AttributeTargets.Property)]

    [MulticastAttributeUsage(MulticastTargets.Method, TargetMemberAttributes = MulticastAttributes.Managed |

        MulticastAttributes.NonAbstract | MulticastAttributes.Instance |

        MulticastAttributes.Protected | MulticastAttributes.Public)]

    public class RequireVirtualMethodsAndProperties : OnMethodBoundaryAspect

    {

        public override bool CompileTimeValidate(MethodBase method)

        {

            if (!method.IsVirtual)

            {

                string methodName = method.DeclaringType.FullName + “.” + method.Name;

 

                var message = new Message