Must Everything Be Virtual With NHibernate?
Posted by Davy Brion on March 30th, 2009
If you’ve ever used NHibernate 2.0 or later, you will have undoubtedly run into the following runtime exception a couple of times:
NHibernate.InvalidProxyTypeException: The following types may not be used as proxies:
NHibernateExamples.Entities.OrderLine: method get_UnitPrice should be ‘public/protected virtual’ or ‘protected internal virtual’
NHibernateExamples.Entities.OrderLine: method set_UnitPrice should be ‘public/protected virtual’ or ‘protected internal virtual’
Oops… we forgot to make the UnitPrice property on the OrderLine entity virtual. But why does it need to be virtual in the first place? That’s a question that many people who are new to NHibernate have.
The quick answer to that question is: because we need members to be virtual in order to do our lazy loading magic/voodoo.
The longer answer is more interesting though. An important feature that any real ORM must have is transparent Lazy Loading. If you retrieve an object through an ORM, you don’t want it to automatically pull in an entire object graph (not by default anyway), yet you don’t want to litter your code with checks to see if certain associations have been loaded yet, and then loading them if necessary. This is the ORM’s responsibility. Ideally, you want to be able to access properties and have the ORM load the necessary data upon first access of those properties if the data hasn’t been retrieved yet.
NHibernate has this ability, yet it doesn’t require you to inherit from some kind of NHibernate base class or implement any interfaces or anything like that. So how does it work? Well, NHibernate uses proxies of your classes at runtime whenever lazy loading is required. Ok, so what exactly is a proxy? In this case, an NHibernate proxy is a type which is generated dynamically when NHibernate is initialized for your application (this only happens once upon application startup). A proxy type will be generated for each of your entities that hasn’t explicitly been mapped to avoid lazy loading (more on this later). A proxy type for one of your entities will actually inherit from your entity, and will then intercept each possible call you can perform on that type.
Let’s discuss a small example that might make things clearer. Suppose you have an Order class. The Order class has properties such as Employee and Customer, among others. But when you load Order instances, you might not always want the Employee property to already contain the real Employee entity instance. Same thing goes for the Customer property. By default, NHibernate considers each entity type as eligible for lazy loading unless it’s been explicitly configured not to (again, more on this later). So when NHibernate is initialized, it will know that it needs to dynamically generate proxy types for Customer and Employee. Let’s just assume these types will be named CustomerProxyType and EmployeeProxyType (they wouldn’t be called like that btw, but it doesn’t matter). Now suppose that you are retrieving an Order instance (or a bunch of them, doesn’t really matter) and you don’t instruct NHibernate to already fetch the Customer or Employee data. You haven’t requested the Customer or Employee data, so it shouldn’t be there, right? But it shouldn’t be null either, right? So NHibernate assigns an instance of the CustomerProxyType class to the Customer property, and an instance of EmployeeProxyType and initializes both proxies so that they contain their identifier value, which you already have in memory anyway after selecting the order record.
You can safely use the Order instance(s) and you can even access the Employee and Customer instances and nothing will happen. But, whenever you access any of the non-identifier members (that means properties _and_ methods) of a proxy instance, NHibernate needs to make sure that the data of either the Customer or the Employee (depending on which one you’re using) needs to be fetched from the database. So how does NHibernate do that? The proxies will override all of your properties and methods and when one of them is accessed, NHibernate will either fetch the data of the entity if it’s not present yet and then proceed with the original implementation of the property or the method, or it will immediately call the original implementation if the data was already present.
This is basic OO… your entities are base classes to NHibernate’s proxies, and those proxies need to add a little bit of behavior to your entities’ behavior. In order to do that, NHibernate needs to override every public member to make sure that this extra behavior is triggered at the appropriate time. Now, there are quite a few people who dislike this requirement. First of all, there is a minor performance cost to calling virtual members as opposed to calling non virtual members. However, this performance cost is really extremely small and in practically every situation it’s completely negligible. This extra cost certainly doesn’t even compare to some real world performance costs, like hitting the database more often than you should or retrieving more data than you really need. Another reason why some people don’t like this is because they don’t like to enable derived classes to override whatever member they want to. In some cases, this is a valid objection. In most cases however, it’s pure Intellectual Masturbation which offers no real value at all. There are other ORM’s that don’t require you to make your members virtual and they are still able to offer lazy loading features. But those ORM’s usually require you to either inherit from a specified base class, or to implement one or more interfaces that the ORM will use. In both cases, i’d argue that this pollutes your entities far more than virtual members do, but that’s just my opinion.
But for those cases where you really do not want to make members virtual, and don’t mind forgoing on the lazy-loading features of NHibernate, you can simply map those entities to not enable lazy loading at all. You could just map an entity like this:
<class name="OrderLine" table="OrderLine" lazy="false" >
Setting the lazy attribute to false will ensure that NHibernate will not create a proxy type of your entity type, and that you will always be dealing with instances of the actual type of your entity instead of a possible proxy type. It also means that you will never be able to use any kind of lazy loading when it comes to retrieving instances of these entity types.
March 30th, 2009 at 9:22 pm
AFAIK, having an interface for entity (Post:IPost) is something that NH supports.
Morever, there are some static proxy factories but i don’t know what their statuses are.
March 30th, 2009 at 9:25 pm
yeah but those aren’t the more common usage scenarios
March 30th, 2009 at 10:12 pm
But why is it necessary to make _every_ public member virtual? Rather than just the properties that map to the database.
It is not uncommon to want other methods that take care of the responsibility of the class that do not directly map to the data.
Order may have an “AddOrderLine” method. Why does this also need to be virtual?
Surely this requirement violates the open-closed principle. ie I want the AddOrderLine behavior to be fixed by design (it is core to the the behavior of the domain. But I’m open to the properties behavior to be modified/extended.
March 31st, 2009 at 6:05 am
@Andy,
What should happen if a public method uses a private field which is supposed to contain data that is mapped through public properties?
If that public method can not be intercepted by the proxy, then it could reference private fields which haven’t been initialized yet to contain the data you are expecting it to contain. This could result in extremely flakey behaving code.
March 31st, 2009 at 8:22 am
[...] Must Everything Be Virtual With NHibernate? – Davy Brion looks at the reason NHibernate wants all methods and properties to be virtual, and talks about hiw dynamic proxies are used to provide usefuly functionality such as Lazy Load. Davy also shows how you can turn off this functionality and therefore no longer require virtual properties [...]
March 31st, 2009 at 9:34 am
Nice article which explains this dynamic proxy stuff very well.
(But, there’s a slight glitch; the mapping example where you show how to turn off dynamic proxies for classes, should specify lazy=false instead of lazy=true.
).
March 31st, 2009 at 9:41 am
@Frederik
hah… thanks for pointing that out. i’ve updated it now
March 31st, 2009 at 10:22 am
I understand why items which need lazy loading should be marked virtual, but why must simple value types (like User.Name) be virtual too? Even if I had a non-virtual property which depended on a lazy-loaded property it wouldn’t be a problem because the overridden property code would be used for the lazy-load association.
For me NH needs to change at least 2 things before I’d consider using it.
1: Not have to mark ALL members Virtual, only ones I want to.
2: When lazy-fetching a polymorphic association I’d want to see see a proxy of each item in the association being a descendant of the type actually created and not just a descendant of the base type. For me this one is a complete show-stopper. As soon as I learned about it I simply stopped learning about NH because I consider it useless.
March 31st, 2009 at 10:43 am
@Peter
i guess you either didn’t read the post carefully or that i didn’t explain it as well as i’d hoped i would
it really doesn’t matter whether you are talking about value types or reference types or properties that need to be lazy-loaded vs properties that don’t need it.
If one of the non virtual public properties of a lazy-loaded instance were the very first thing that would be accessed, say User.Name, then NHibernate would have no way of actually loading the data for your User instance, which would result in your User instance returning null for the Name property. Not exactly the behavior you’d want, right?
The requirement for virtual members is not for properties that require lazy-loading… it’s a requirement for _all public members of a type which should be eligible for lazy loading_. Without that requirement, there is no way to guarantee that your instances have their data loaded at the appropriate time when you access them _in any way_
And the behavior you would want for the polymorphic association is impossible to achieve with _lazy_ loading… after all, we can’t possibly determine the actual type _without hitting the database_ and thus, preventing the whole point of wanting to use lazy loading on that specific polymorphic association in the first place. You could however, simply prevent lazy-loading for that specific association and your ‘problem’ would be solved. If you want to know the actual type, you have to hit the database somehow.
I really wouldn’t get my hopes up on NH changing these things for you… your requested changes would break a bunch of features that other people want and/or depend on
I guess you’d be better off creating your own ORM which satisfies your requirements to the fullest
March 31st, 2009 at 11:06 am
“What should happen if a public method uses a private field which is supposed to contain data that is mapped through public properties?”
Then you’ve got a bug.
This type of thing would be a source of flakey behaviour regardless of NH.
To avoid flakey behaviour… write tests.
This type of thing is part of the contract/expectations that a developer creates as best practice for the project. ie you can only expect the value to be populated, or persisted, correctly if you access via the property (which is proxied via NH).
I really don’t want some other framework thinking it knows best and overriding my stuff without my explicit permission. (That’s what government is for!!)
I would much prefer an option (similar to lazy=false) that says only proxy virtual properties/fields. I would expect this to be a simple switch around the exception that is currently throw. (but what do I know
I’ll avoid flakey behaviour by writing tests which I should be doing anyway, right?
March 31st, 2009 at 11:12 am
@Andy
so you’d never use the private backing field of a public property directly within the same class?
If that were a requirement that NHibernate would enforce, then i’d imagine that would get even more complaints than the requirement of virtual public members
March 31st, 2009 at 11:46 am
Of course I might do that. That would be my choice and there may also be perfectly good reasons for access it _without_ some AOP style proxy doing magic for me.
I’m saying that I want it to be up to me to decide.
You talk about enforcement. I’m talking about choice. I want NH to do less not more. I want a simple option (like “lazy”) so that _only_ virtual properties/fields are proxied. Other members are just inherited as normal. There is nothing for NH to “enforce”. It is _my_ responsibility to “enforce” the right practices and write the tests.
I’m obviously missing some important point here.
I also take your point to Peter. I hate complaining about things that obvious work for (many) others and are free. Maybe I should just write a patch!
March 31st, 2009 at 12:01 pm
@Andy
about only proxying virtual members and ignoring the non-virtual ones: i think that still would be problematic. Suppose you have a certain type which can be proxied… if code from another class uses a possibly proxied instance of this type, it might fail even though it would work when it’s working with a real instance. Some pieces of code might work perfectly in some cases if associations weren’t lazily loaded, while it could fail when that code is accessed in some other way when the associations are lazily loaded.
All in all, i think you’d quickly end up with a brittle code base, no matter how many tests you may write. And those tests would probably only be really valid if all of them are using your domain in the exact same way as it would at runtime (ie, with NHibernate running the show). I don’t want all of my domain tests to require a database and NHibernate.
March 31st, 2009 at 1:49 pm
First… Thanks for being persistent with me.
But I don’t see how we can have it both ways.
Either persistence is orthogonal to the behaviour or it isn’t.
You seem to be saying that NH can change the behaviour of the domain object. If this is so then you _must_ always test with NH.
You seem to be saying that in order to detect all the places where I’m access possibly persisted values NH has to inspect all my code so it can fix my sloppy coding.
If a class is written in such a way that it needs its hand holding, then I’d suggest that it’s broken regardless of how you do persistence.
I believe that there is no such thing as complete persistence ignorance. You still have to be aware of the concerns and design for it.
I guess I’m struggling to see the scenarios that you are concerned about.
Accessing backing fields (avoiding property code) will be potentially problematic anyway. My answer would be “don’t do that then”.
Again, thanks for the help to “get it”.
March 31st, 2009 at 2:04 pm
well no, NH doesn’t really change the behavior of a domain object, outside of lazily loading its data if necessary
the only thing i’m saying is that ignoring potential access to private backing fields (and there certainly are valid reasons why people want to do that) could lead to a host of problems, which could potentially be increased when using NH because you will sometimes end up with proxies and sometimes with real instances. Simply saying “don’t do that then” doesn’t really cut it IMO.
I do think that persistence ignorance is possible, but only with a very specific custom made ORM. NHibernate doesn’t get you true persistence ignorance (otherwise we wouldn’t require virtual public members or default constructors), but i do think it gets you far closer to it than any of the alternatives (outside of rolling your own)
April 1st, 2009 at 9:47 am
@Davy
So the virtual is for lazy loading ALL object state then, not for identifying which parts of the state should be lazy loaded? What if your class has a large byte[] property which is only used 50% of the time? I already use an ORM which lets me mark individual elements as lazy-load, and it loads polymorphic associations properly too (http://www.capableobjects.com)
Considering its popularity I am really very surprised that NH works this way.
April 1st, 2009 at 9:50 am
@Peter
right, lazy loading individual properties is not possible by default, though i do remember seeing a workaround for it once to enable it… that was far from trivial though
in case of the byte[] array, i’d either put the byte[] array in a separate table which links to the original record and map it as an association, or use a separate mapped class which uses the same table.
April 1st, 2009 at 9:55 am
@Davy
“we can’t possibly determine the actual type _without hitting the database_”
But when you access the lazy-load object you *do* go to the database to read its content, so you do know the exact types.
April 1st, 2009 at 10:04 am
@Peter
the situation you are complaining about is a situation where we need to create a proxy _before_ hitting the database. If you can come up with a way that enables us to figure out which of the derived proxy type we should use without hitting the database, and without breaking anything, i would be more than happy to implement it for you. Though i can’t help but wonder why more people don’t complain about this if it really is that big of an issue.
April 1st, 2009 at 9:31 pm
@Peter
http://mijalko.blogspot.com/2009/03/hibernate-nhibernate-and-polymorphism.html
that has a workaround to the issue you mention, though i haven’t tried it yet
having said that (and i can’t believe this didn’t cross my mind sooner), if you feel the need to check for the actual type when all you have is a base type (whether it’s a proxy or not), you’re kinda missing out on the whole point of polymorphism, no?
April 3rd, 2009 at 2:31 pm
Hi Davy
Maybe I am misunderstanding the problem. Let’s take this model
JobBatch 1—* JobAction
Where JobAction is abstract and has an abstract Execute method. Now let’s say that my lazy-load JobBatch object is used like this
foreach(JobAction action in job.Actions)
action.Execute();
Every instance in job.Actions is a dynamically created sub-class of JobAction isn’t it? Can you confirm this is the case? We can work from there
April 3rd, 2009 at 10:02 pm
since i don’t like repeating myself i’m just gonna link to a comment i left earlier as a reply to the last time you asked that question
http://davybrion.com/blog/2009/03/entities-required-properties-and-properties-that-shouldnt-be-modified/#comment-10632
but for a many-to-one association, the proxy will indeed be derived from the base type for reasons i’ve already explained earlier
April 4th, 2009 at 11:12 pm
No, that’s fine. It’s just that something seems so obvious to me that it cannot possibly be right and I wanted to be 100% sure I didn’t have a basic fact wrong.
Now that I know I am not making a stupid mistake I would like to discuss it with you. Do you think you might have 5 mins spare to chat on skype (mrpmorris) or msn (mrpmorris@live.co.uk)? I’m not after support or anything because I am not currently using NH
I just wanted to discuss some ideas etc.
Cheers
Pete
April 5th, 2009 at 4:31 pm
sure, we’ll talk about this on msn
April 7th, 2009 at 8:37 am
Thanks for adding me. I hope you log into MSN now and again
April 7th, 2009 at 10:19 pm
Thanks for your time today. The scenario we came to certainly explained what the limitation actually is, and I am fairly convinced it is irrelevant.
I do however still disagree that everything needs to be virtual
I think that only the association properties need to be virtual. I can’t think of a scenario where you will be able to get a proxy without going via an association. For example when you load an object you get the object itself, all of its single links are instances of proxies but you can’t get to that proxy unless you do LoadedObject.SomeAssociation so all you’d really need is LoadedObject to be a proxy which loads the associated object when that property is accessed.
In the case of a multi-association again it’s the property which must be accessed in order to obtain a reference to any item in its list. How is it possible to access the proxy any other way? I don’t think it is.
So I am still utterly convinced that you only need to actually make association properties virtual. If someone reading this could give me a step by step scenario of when not having virtual properties/methods on a class could cause problems (except for not making associations virtual) I would appreciate it
Pete
April 8th, 2009 at 8:01 am
if only the association properties were virtual, then merely accessing the proxy property would trigger the lazy load, which is not good either
Say you have order.Customer and the Customer property holds a reference to a proxy. If only association properties should be virtual as you claim then merely accessing the Customer property would trigger the lazy load. But what if i only want the Customer property’s Id value? That one is already there in the proxy and can be retrieved without having to hit the database. But when i access one of the proxy’s other properties, say order.Customer.Name it needs to hit the database.
April 8th, 2009 at 8:03 am
btw, you can just as easily get a proxy without going through an association by calling session.Load instead of session.Get.
April 8th, 2009 at 10:11 pm
Why would Session.Load fetch a proxy and not the true type?
April 9th, 2009 at 8:05 am
That’s the main difference between Session.Load and Session.Get (bad naming, i know). A lot of people use Session.Load for various reasons. You can find one example here:
http://davybrion.com/blog/2009/04/assigning-foreign-keys-in-nhibernate/
April 9th, 2009 at 2:08 pm
I’m quite persistent, aren’t I?
So it’s a way of loading a referenced object without fetching any state? In your example I would fetch the category and then set product.Category = categoryProxyThatIsntInitialized right?
August 18th, 2009 at 2:03 am
[...] but it’s not as bad as it looks. What this is doing is declaring what library to use to generate dynamic proxies for our data classes. It is a good thing that NHibernate is open to multiple providers here, and the good news is this [...]
September 23rd, 2009 at 6:56 am
[...] already tried to explain this before, but here’s a simple example from a presentation i recently did on [...]
February 8th, 2010 at 8:21 pm
[...] proxies to achieve its Lazy Loading feature. This is a bit beyond the scope of this post, but Davy Brion offers a pretty detailed explination, The quick answer to that question is: because we need members to be virtual in order to do our [...]