If you’re not very experienced with NHibernate, it’s not always clear what NHibernate is doing behind the scenes. What kind of queries is it generating? How many entities are we loading? How often does it go the database? Even experienced NHibernate users can make mistakes which may result in inefficient data access, high memory usage, slow response times, etc. When used properly however, you can get excellent performance out of NHibernate.

According to the website, NHibernate Profiler delivers the following advantages:

• Cognitive application awareness
• Visual insight into the interaction between your database and application code
• Analysis and detection of common pitfalls when using NHibernate
• Analysis is delivered via perfectly styled SQL and linkable code execution

Sounds pretty good, right? Let’s take a closer look.

What is NHibernate Doing?

To get a good view on what NHibernate is doing, you should get answers to questions like:

• How many statements are we sending to the database?
• What statements are we sending to the database?
• What data are we loading into memory?

NHibernate Profiler makes it easy to anser those questions. For an NHibernate session where we send a simple query to the database, the profiler already gives us some useful information:

In the Statements tab, you get an overview of all of the statements in this session. When you select one of the statements, you can view more details in the tabs on the bottom. In this screenshot, you see a properly formatted SQL query which is the query that was generated by NHibernate. The overview also shows how many rows were retrieved by the query.

So we know how many entities were loaded, but which ones were they? Again, the profiler gives us some useful information on that:

The Entities tab shows you the identifiers of each entity instance (grouped by type) that was retrieved in the session. There’s also a useful general overview of your session usage:

Looks like the profiler is already alerting us of a possible problem. Let’s go the Alerts tab:

You can see the alert right there, with a link to a help page on the profiler’s website with a more detailed explanation. The help pages are very helpful and always suggest a way to avoid the problem. In this case, the problem is that our query returns the rows of a table without limiting the number of rows to be returned. In some cases, this can really be a problem but if you’re really sure that the table will always have just a few records then it’s not really a problem. A helpful extra feature here would be to mark the alert with some sort of “i know about this, and i don’t consider it a problem” action so it is removed from the profiler’s output. This specific alert can be disabled altogether as well, although i think it would be more useful to remove these alerts on a case-by-case basis.

Now, we know what NHibernate is doing which is pretty useful, but we also need an easy way to figure out which piece of code caused the SQL statements to be sent to the database. The profiler allows you to look at a stacktrace when you select a statement:

You can double click at each entry in the stacktrace and Visual Studio will show you that line of source code.

I think it’s clear that the profiler does a good job of showing you what NHibernate is doing behind the scenes. The output that the profiler is showing you is very useful as well:

This is the output of a batch of queries. Notice how the queries are nicely separated, and that the parameter values are already filled in. You can easily copy these select statements to execute the queries in whatever querying tool you’d like to use so you can study the actual output of a query. Another nice extra feature would be the ability to select a statement and have the profiler execute it against the database and showing you the results in a separate window.

I hope to see more information regarding the usage of NHibernate’s 2nd Level Cache in a future version of this profiler. It would be great to see which entity instances are stored in the cache, when they are stored and through which query, when they are retrieved from the cache instead of the database, when they expire from the cache, when they get refreshed, etc…

What are YOU doing?

Alright, we already covered how the profiler shows what NHibernate is doing. That’s nice. But do you know what you’re doing? Are you sure you’re using NHibernate properly? The profiler has a couple of nice features to warn you from mistakes you might make. As you saw earlier in this review, the profiler alerts you when you do something that might be problematic.

The profiler currently has the ability to alert you of the following problems:

• Select N + 1
• Too many database calls in a session
• Unbounded result set
• Excessive number of rows
• Usage of implicit transactions
• Large number of writes

The profiler offers guidance for each type of alert, making it clear how you can avoid the problem by suggesting other approaches. This guidance is offered through the website of the profiler (the application provides links), but it might be a nice idea to show that guidance from within the tool as well.

Here’s an example of the Select N + 1 problem:

As you can see here, the same query is executed 4 times with an ID parameter that differs each time. Naturally, it would be much better if this data was fetched in one query instead of 4. The profiler clearly alerts you to the problem, and through the stacktrace tab you can very easily navigate to the part of code that is causing the problem.

It’s also worth pointing out that you can modify the settings which trigger these alerts:

Overview Features

Looking at what each individual session is doing is useful, but sometimes you kind of want an overview of the overall NHibernate usage of your application. NHibernate Profiler offers 4 extra views on what’s going on in your application.

The first is an overview of all of the queries that were executed during the run:

This is useful, but i think it would be better if the grid included the average duration for each query. Also, a way to navigate from one of the queries to where it is triggered in code would be great here.

There’s also an overview of the queries that were triggered by class and method:

In this view it would also be very useful to show more information about average durations of statements.

The third one is a general overview of your overall NHibernate usage:

This one is very nice… the aggregated alerts pane is great, but when you click on the links next to the alerts a browser window will open with relevant alert information from the profiler’s website. I think it might be easier if you could just navigate from these aggregated alerts to the specific sessions which are causing these. For instance, if i have a SELECT N + 1 alert in there, i would like to have a way to go to the data from that specific session (or the sessions) with as little clicks and searching as possible.

The aggregated entities and the general statistics are also a very nice touch.

And finally, you can also keep track of the stats that NHibernate itself provides:

Conclusions

I’ve been playing with this profiler for a couple of weeks now, and i definitely like it a lot. It not only does a great job of quickly pinpointing problems, it also does a good job of teaching users how to properly use NHibernate. It not only saves you time when you need to fix a problem, it enables you to fix the problem in an informed manner. It’s got some great features already, and there will probably be many more interesting additions to the profiler in the future. Is this profiler worth the cost of the license? I’d say it definitely is. Not just for the time you’ll save because of it, but also for the stuff you’ll learn from it.

Oh, and Ayende has a 30% discount on licenses at the moment… i’d take advantage of that if i were you

My birthday. I am not sure yet what events led to my existence but here i am anyway. I have no idea what i am, what i’m supposed to do or what my purpose in life is. But there are a few more of my kind running around here. I guess i better go find out.

October 23, 2008

Apparently, i’m a bug. A software bug if you want to be specific. My brothers and sisters told me that we are a byproduct of an activity that occurs between code and developers. That activity being screwing around without proper protection. It means that a developer made a mistake somewhere, and then we come in. Some of us remain dormant, never to be discovered while most of us are spotted sooner or later. Some of us can wreak havoc when we are triggered, while others just cause minor inconveniences. I don’t quite understand everything yet, but it appears that we are no good. We cause problems and nobody seems to like us. So why are we here?

October 30, 2008

It’s been another pretty boring week… i haven’t been triggered yet, so i’ve had plenty of time to seek answers to the questions that i still have. Most people deride us, because they claim we are only costing people money. In some way, that’s true. But we are a necessary evil in the field of software development. Our purpose is to force software developers to step their game up. If they mess up, we punish them with everything we’ve got in the hope that they at least learn something from the mistakes they make. If the developers are smart, they make sure to put protective countermeasures in place so fixed bugs can’t reappear, or that similar bugs can’t be created. We also sometimes manage to keep big headed developers grounded. After all, most of them think they’re some kind of gods because they’re oh so smart and clever and because they are ‘creating’ things. If it weren’t for us, there would be nothing to keep them from feeling superior to everyone else.

December 3, 2008

One of my brothers got ‘fixed’ today. That’s how the developers call it. Fixed. That’s quite a euphemism considering the fact that it actually means one of us was killed. I know that this is a part of our job, but that doesn’t mean it’s an easy thing to deal with. My resentment towards developers is growing every day. You know, we’re doing our part… but developers don’t really seem to learn too much from the mistakes they make, considering our prominence in the history of software development. Bringing us in this world just to kill us is cruel enough, but do they really need to create more and more of us just so we can be killed? It all seems so pointless.

December 5, 2008

Uh-oh… i’ve been triggered. This is the first time i was triggered in this code base and i’m not sure how to feel about it. A part of me is anxious to complete my task, but i’m starting to like it here… i don’t want to go so soon… i’d much rather stick around for a while so i can hopefully make it to one of the maintenance branches.

December 6, 2008

Hmm, i haven’t been triggered anymore so i guess nobody wrote a test case to truly expose me yet. I feel good about that, but i am getting more nervous with each passing hour.

December 16, 2008

My worst fears have been confirmed. I’ve been discovered and someone is triggering me over and over again. I looked into the subversion commit logs (yes, we can actually do that) and some guy with username ‘davybrion’ committed a test case. Great… the rookie of the team. Probably has a chip on his shoulder too… feels like he has to prove something. I hate these guys. I’ve been given a name too… NH-1609. Gee, thanks. Now i really feel like a number in a system. These developers… arrogant pricks!

December 22, 2008

I’ve got a headache… i’ve been running for a while now, with a debugger attached. That’s a lot harder than running in a normal environment, trust me. No code has been changed yet, but i think this ‘davybrion’ guy found my exact location. He appears to be exploring his options for killing me. Oh sorry, i mean ‘fixing’ me. I can hear him swearing and cursing too… i even heard him say he’s definitely gonna ‘fix’ me. What an over-confident, foul-mouthed prick.

December 24, 2008

It’s night time, and i haven’t been triggered for a couple of days. Time to enjoy Christmas Eve with my fellow bugs! We’re going to enjoy a nice meal, have some wine, and debate on which of the developers we despise most. My vote is for ‘davybrion’.

December 27, 2008

It’s been pretty quiet around here… no modifications to the code that i can interfere with. Which is too bad because i’m eager to spawn some kids of my own. Everybody wants to leave a legacy when they go.

December 29, 2008…. 10PM

Well, he’s at it again. I’m constantly being triggered and i’ve got a bad feeling about this. ‘davybrion’ is changing code left and right. He seems to have found the root cause of my existence, and he’s changing a lot of code… he’s taking a lot of risks… procreation, here i come!

December 29, 2008… 11:30PM

Hah, this rookie screwed around… i’m a proud father now!

December 29, 2008… 11:55PM

Damn it! He must have realized the risk he was taking and he reverted all of his changes. He killed my babies before they were ever committed!

December 30, 2008… 7PM

Oh great, he’s back for more… I can’t wait for him to screw this up again.

December 30, 2008… 8:28PM

Oh crap… i think he’s got a fix ready… this is gonna be the end of me… i can feel it

December 30, 2008… 8:30PM

He ran all of the tests and i didn’t get activated… I guess that’s it for me. I can’t believe this rookie got the best of me. He claims he’s not happy with how he had to fix me (yes, i can spy on his IM messages and his email) and he wants to refactor an important part of this code base. At least he realizes it’s too risky to do this with their next release in mind. Perhaps this prick did learn something from all of this. Oh well, i can only hope so. I still don’t like developers though… we can’t live with them, but we can’t really live without them either. It’s a complicated relationship really.

• The same kind of topics as in 2008
• More NHibernate related posts
• Less NHibernate related posts
• More introductory posts to various topics
• More real world examples, problems and situations
• More opinion posts
• Less opinion posts
• More information about NHibernate’s development

You can select up to 4 items if you want. I won’t promise anything, but i will try to keep the outcome of the vote into account

Any other suggestions or comments about what kind of stuff i should be writing about are always welcome of course.

Let’s start with ASP.NET WebForms. This is the option that has been available since the .NET platform was introduced, so there are already a lot of people experienced with it. There is also a ton of information available about the ins and outs of WebForms. And when it comes to commercial tool vendor support, it’s clearly miles ahead of the other options since there are lots of commercially supported controls available for it. Unfortunately, WebForms was originally created to offer a similar development model to WinForms developers. The code-behind files and the event-driven way of working is very similar to how you would write code in WinForms applications. Obviously, web applications and windows applications are completely different things, so you can’t reasonably expect the same way of working to be suitable for both.

Particularly, the Page and Control lifecycles are rather complex (at least much more complex than i would think they’d need to be) and often cause weird issues on complex pages. The event-driven model of both the Page class and the Control class (and its derivatives) seem to be the major cause of this. I think most WebForms developers have on more than one occasion spent long times debugging weird situations which were ultimately related to certain events triggering unexpected behavior in other controls on the same page. Granted, people with an in-depth knowledge of how these lifecycles really work don’t run into this as much, or are capable of avoiding these problems altogether.

You could indeed make the argument that most of the problems that people are experiencing with WebForms are caused by those people’s lack of understanding how it really works and how it really should be done. On the other hand, that is a pretty good sign that WebForms development isn’t intuitive or clear enough, and that while Microsoft has tried hard to make it easy to use, most people seem to solve their WebForms problems by hacking around their problems, or playing around with settings until it seems to work.

Another major problem with WebForms is that it doesn’t easily enable Test Driven Development. True, there are various patterns you can use to make sure you can write testable UI logic (up to a certain point anyway) but all in all, these approaches require more effort than should be necessary and you often end up wrapping a lot of stuff just to be able to test it. WebForms in general was never designed with testability in mind, and you will most definitely be confronted with that if you try to write testable UI code with WebForms.

As you can probably tell, i’m not a huge fan of WebForms. I think it’s fine for simple applications, but for anything beyond that i’d like to avoid it as much as possible.

Microsoft’s upcoming ASP.NET MVC framework aims to fix much of the issues i’ve mentioned above. It was designed with testability in mind, and although it’s not perfect either (depending on who you ask), it is certainly a huge improvement over WebForms when it comes to writing testable UI code. You also have a lot more options when it comes to having the framework behave the way you’d like it to. Another important benefit is the fact that ASP.NET MVC kinda forces you to structure your code in a much more sane manner. You put as little logic as possible in the views, and you put most of it in the Controllers where it belongs. Obviously, that doesn’t mean you should put business logic in the Controllers!

One of the downsides of ASP.NET MVC is that, due to its completely different way of working compared to WebForms, it comes with a much higher learning curve. Experienced WebForms developers might struggle with it at first, and might even be frustrated because most of their hard-earned WebForms experience no longer gives them a benefit. Some will probably enjoy it much more than WebForms, and some will probably dislike it strongly because it’s so different. Another downside is that there are far less commercial tool vendors that are offering ASP.NET MVC controls, at least compared to WebForms. Although that might not be that big of an issue, since i suspect that it’s easier to develop nice looking and reusable controls yourself when using ASP.NET MVC than it is to do so when using WebForms.

If you’re starting out with ASP.NET MVC, i think it’s safe to assume that you won’t proceed as quickly as you’re used to at first. But once you’re used to the new way of working, i’m pretty sure that it enables you to rapidly implement new pages and new functionality in a very clean way. For true web applications, i would probably pick ASP.NET MVC everytime, unless you don’t have the room to get over the learning curve. Also, when combined with a client-side javascript library like JQuery, this approach seems very compelling. Yes, i know you can use JQuery with WebForms as well, but it just seems to lend itself better to the MVC approach.

And then there’s Silverlight. There are already a couple of reasons why people would not want to use Silverlight for web applications. For starters, even though the application is running in a browser, it’s not really a true web application is it? Your users can’t bookmark pages, using the back button leads to unexpected behavior, there aren’t that many options for Search Engine Optimization, etc… However, if your only web-related requirement is that your application needs to run in a browser, without having to worry about any of the typical expected requirements for web applications, then Silverlight is a pretty interesting choice as well. You can very easily create very rich applications, with possibilities that are either impossible, or extremely difficult to do with typical web development platforms.

Our Genesis web front-end is developed in Silverlight. The UI not only looks great, but you can navigate between all of the available data in a manner that is simply much easier to develop than it would be for typical web apps. It’s very easy to create that ‘wow’-effect with your users in Silverlight. Obviously, that ‘wow’-effect isn’t the most important part of your application, but it does count for something.

Another interesting aspect of Silverlight development is that you can go back to a more statefull development model compared to the typically stateless nature of web aplications. After all, the cost of that state is no longer something your web server has to bear. It’s now the client who has to keep that state around, so you can avoid a few extra roundtrips here and there as well. Then again, you really don’t want your Silverlight application using huge amounts of RAM in your user’s browser either so you do need to take care not to go overboard with it. The development model is completely different than both WebForms and ASP.NET MVC, but if you already have people with WPF experience the learning curve is probably not that high.

However, testability isn’t great (yet) when it comes to Silverlight. It’s possible to write testable code, and you can execute tests in the Silverlight runtime, but it doesn’t really lend itself to a true TDD approach yet although i hope that will improve in the future.

So there you have it… the three options for .NET web development. I’d stay away from WebForms altogether from now on, and i’d decide between ASP.NET MVC and Silverlight on a case by case basis. What, you weren’t hoping for a definitive answer were you?

I would be interested in hearing your thoughts about pro’s and con’s of any of the options… particularly things that i haven’t mentioned, or if you just plain disagree with my statements. Which option would you prefer to use, or not to use?

First of all, what’s the difference between a senior developer and a junior developer? Is it merely the number of years of experience? In previous client engagements, i’ve seen more than my share of bad developers who’ve had years and years of experience. Would i trust those developers more simply because they have the experience required to be called ’seniors’? Hell no. Trust has to be earned, i don’t care if you just graduated or if you’ve been writing code for 5 years or more.

When i have to work with someone i’ve never worked before, i assess this person’s qualities and capabilities on two things: how he thinks about writing code (in general), and how easy he can pick up new concepts/practices/principles. That’s it. A junior developer with little to no experience can often be a lot more valuable than a developer who has 5 years of experience under his belt and just assumes that he knows it all.

With a senior developer, you have to be lucky that he’s learned from his previous mistakes (and every developer makes mistakes, no matter how good he is or how much experience he has), that he hasn’t picked up too many bad habits and that he is open minded. If you can get a senior developer like that, consider yourself very, very lucky because there really aren’t that many of them.

With a junior developer, you can easily mold them into the kind of developer you want them to be. They haven’t really had a lot of time to pick up bad habits, and they are eager to prove that they belong at your company so they will be very eager to learn and improve. All you need is a couple of people who are willing and capable of teaching these young developers.

Of course, with junior developers you do have to live with the fact that they will make rookie mistakes. You have to review their work a bit more, and make sure that they learn from their mistakes. If you do this from the beginning, you’ll quickly notice that the extra reviewing tasks will soon take up less and less work.

At my company, we don’t really differentiate between juniors and seniors. The last couple of years, we’ve pretty much only hired young developers who just graduated. And so far, it’s worked out great. They never get assigned easier tasks or anything like that, and they have to do the same kind of stuff that people with more experience need to do. The result is that we have a bunch of young developers (i think the average age of our developers is 24 or something) who already do a great job, and they’re constantly getting better.

Consider the following solution:

Now, i’m not going to get into the specifics of each project in this solution… most of these projects were created before i ever got involved with this project, and i’m not really happy with the entire structure. It’s a pretty big application, and a while ago we decided to move to a new architecture. But since we can’t just rewrite the whole thing, we put the new stuff (using the new architecture) in the same application and we’re going to gradually rewrite the old parts using the new architecture.

I wanted to keep the new stuff completely separated from the old stuff, so i added more projects to it (the EMS.* projects). Before i added the new projects to the solution, it was already painfully slow to use this solution with Visual Studio. After adding the new projects, it obviously only got worse. Since we’re spending most of our development work in the new projects, i wanted to see if i could simply create a new solution which would contain only the projects we usually need. That new solution looks like this:

Much better…. but now you’re probably thinking: doesn’t CMS.WebApplication reference any of the other projects? It does reference a few of them actually:

Visual Studio indicates that it can’t resolve these references. So this new solution isn’t usable, right? Well, it is actually. You just have to make sure that you’ve done a full build of the entire solution (the one that has all of the projects in it) before you build the small one. If you use the small solution after you’ve built the big one, Visual Studio is smart enough to remember where it got those compiled dependencies from in the first place.

So is this really usable? It sure is… we do most of our work in the smaller solution, and we can modify and recompile as much as we want without problems and without wasting huge amounts of time just waiting for Visual Studio and the compiler. The only issue we have with this approach is when we need to make changes in some of the older projects that aren’t in the small solution. Whenever someone makes a change there that requires a recompile of the CMS.WebApplication project, every teammember needs to recompile the entire big solution. But to avoid having to load the entire solution in Visual Studio, you can just run the following command in a Visual Studio 2008 Command Prompt:


msbuild ems.sln


and it builds the entire solution without using Visual Studio. After that, your smaller solution will work again.

If you’re working with big Visual Studio solutions and the slowness of this bothers you, be sure to give this a shot. You can create as many of these small solutions as you like, depending on which parts of the codebase you typically need to work with. It can easily save you a lot of time, and avoid unnecessary frustration as well

For new solutions, i think it’s better to just keep the number of projects to a minimum which i’ve explained previously here

I’m also pretty happy with the way Microsoft is starting to open up the development of some of their projects, with ASP.NET MVC being a very nice example. They’re not doing true open source development yet, but at least they are listening to the community more and more. I’m especially happy with how people like Scott Hanselman, Phil Haack, and Glenn Block not only deal with the community, but are active (and positive) participants of it. Then again, there are definitely things that should’ve been handled better (Entity Framework, Oxite, LINQ To SQL’s future, …). Hopefully, we’ll get to see a lot less of those kind of mistakes in 2009. But all in all i think it’s been a rather impressive year from Microsoft’s Developer Division.

2008 has been pretty good for myself as well. I escaped from working in Enterprise Hell and now get to do a lot of interesting things at work with motivated and talented co-workers. I’ve also seen my subscriber count go from 40 to around 700 (on average) this year, which certainly provides a lot of motivation to keep going. I also joined some great bloggers at ElegantCode.com, and joined the NHibernate developers. Should keep me pretty busy in 2009

I’m taking a few days off from blogging, so happy newyear everyone, and i’ll see you all early January

• Windows XP Professional, with Service Pack 3: i don’t have Windows 2008 and i still don’t consider Vista to be a valid option
• SQL Server 2005 Developer Edition: i don’t have the 2008 version, plus the 2005 version more than suits my needs
• Visual Studio 2008 with Service Pack 1: duh….
• Resharper 4.1: duh… won’t use Visual Studio without it
• TortoiseSVN + VisualSVN: TortoiseSVN is great already, but the Visual Studio integration that VisualSVN provides is so nice that i can’t go back to just using TortoiseSVN.
• NHibernate Profiler: it’s not even out of private beta and i already consider this a must-have tool… i hope Ayende is gonna go easy on the licensing cost for this
• dotTrace: you gotta have a profiler, right?

Anything else i should consider installing? I only use this system for development so i don’t want any suggestions that aren’t related to that.

On a side note, despite being a Parallels customer i set this system up in VMWare Fusion. According to some reviews i read, Fusion beats Parallels when it comes to IO, making the system feel more ’snappy’. I’ve been very impressed with the Fusion performance already so i just bought a license and am planning to use Fusion exclusively from now on instead of Parallels.

I quote:

A reasonable deadline for the project was approximately 3 months but it had to be completed in a month. At this time an agile book will indicate that the developer should talk to the owner and come to a reasonable deadline. Unfortunately, in this case the owner was fixed with the deadline and was not willing to move it further.

The author further concludes that following the agile principles would have killed the project. He dropped some of the agile principles and practices, and finished the project in 29 days.

Yes, the resultant code was messy but at the end I get to keep my job.

I have no problem with Mohammad dropping agile practices/principles in order to complete this on time and to help him keep his job (especially in the current economy). I do have a problem with the idea that ‘Agile’ can kill a project. The only thing that killed this project is the unrealistic deadline which was set by the owner. The fact that the project was delivered within the requested time frame, and even assuming that it is completely satisfactory to the owner, does not mean that it was a good idea to hold this project to a deadline which only allowed for one third of the estimated workload.

Now some of you are probably thinking “the deadline couldn’t have killed the project since it was delivered within that deadline!”. True, to a point. We all know what happens when developers face tight deadlines: they take technical shortcuts and they accumulate technical debt. Mohammad already stated that the resulting code was messy so i guess it’s safe to assume there’s quite a bit of technical debt there. Being in technical debt is like owing money to Tony Soprano… you better get out of debt fast or things are gonna get much worse, very soon even. The longer you wait with paying off the debt, the more you’ll lose certain abilities and possibilities.

Getting out of technical debt can be pretty expensive, but there’s not really an alternative. If you don’t get out of debt, further maintenance of the code base and adding new features will only become more expensive over time. Failure to pay off the debt will sooner or later kill the project entirely because the gains that the project brings are no longer greater than the costs that come with it. At that point, the decision is often made to build a new system to replace the old one. The new one is gonna fix all of the problems of the old one. And if you’re lucky, the managers will have figured out by then that unrealistic deadlines only cost more money in the long term than a reasonable deadline would have.

I have to admit that i still prefer Windows XP, probably because it’s the lightest one of them all at the moment, and since i run XP in a Parallels VM on a cheap Macbook, that kinda matters

At work, i’m using Windows Vista… not really happy with it though…

If you’re using an IOC container it’s important to not use it all over the place. You basically use it in as few places as possible to resolve a component and you let the container sort out all of the dependencies. So in the few places where you use the container directly, you need to resolve the component, and in case of transient components you also need to release them through the container. Releasing it is very easy to forget, so i wanted something that would guarantee that the component would be properly released. Enter the Resolvable class:

The Resolvable class inherits from my Disposable class, so the Disposable pattern is correctly implemented.

From now on, instead of calling the container directly, i just instantiate a new Resolvable in a using block. Let’s try it out.

I’m reusing my test component with a dependency from one of the previous posts:

Now, instead of resolving an IController directly through the container and having to dispose of it, i just do this:

The container doesn’t hold the reference to the instance, and both the instance and its dependency is properly disposed.

Multi-tenancy to the rescue! Not sure if this counts as the official definition of multi-tenancy, but wikipedia defines it likes this:

Multitenancy refers to a principle in software architecture where a single instance of the software runs on a software-as-a-service (SaaS) vendor’s servers, serving multiple client organizations (tenants). Multitenancy is contrasted with a multi-instance architecture where separate software instances (or hardware systems) are set up for different client organizations. With a multitenant architecture, a software application is designed to virtually partition its data and configuration so that each client organization works with a customized virtual application instance.

Sounds like this is exactly what we’re looking for. So i’ve recently been working on changing the application to support this, and i came up with the approach i will outline in the rest of this post. The approach does not strictly comply with the definition above, but it does seem to comply with Ayende’s definition of it. I’d like to get some feedback from you guys as to whether you believe this approach is good or not, what could be better, what we need to keep in mind, etc…

First of all, i would like to point out that this application already exists. We already have a database with about 200 tables in it and there are about 150 pages (it’s a web app obviously). So obviously, we can’t just change everything to make it fully compliant with ‘the definition’.

Let’s start with the database. Instead of trying to use one huge database to keep all of the tenants’ data, which would require modifications in a large amount of the existing tables, we’ve decided to go with a separate database for each tenant. Each database will have an identical structure, regardless of whether some functional modules are used or not by a particular tenant. There is also one ‘master’ database which contains tenant-specific data. Basically it contains all of the configuration settings for each tenant, including connection strings to each tenants’ specific database. The connection strings do not contain user names and passwords as we will use Windows’ Integrated Security to connect to the specific databases (more on that in a bit).

Now for the application itself. Our first idea was to have one actual instance of the application, and the application would be able to determine the current tenant for each request based on the URL of the incoming request. Tenant A would have an URL like tenanta.ourproduct.com, tenant B would have tenantb.ourproduct.com etc. The application would basically use the URL to get the correct tenant-info from the master database (note that the connection string to the master database would be the only connection string we’d have in the application’s configuration file) and it would then use the tenant-specific database for each request with the tenant’s application URL.

The idea was to use an NHibernate SessionFactory for each specific tenant. You obviously can’t use just one because you’re using multiple databases. But we also use NHibernate’s 2nd Level Cache, which is problematic when you’re using multiple SessionFactories. The 2nd Level Cache is great, but it doesn’t differentiate between multiple SessionFactories. So if you have the result of a specific query cached, you could get that data back for a different tenant than the one the data actually belongs to if the query’s parameters happen to be identical. Btw, if i’m wrong about this please let me know.

So then we figured we could still use one physical deployment (as in: one physical folder where the application is located), and then we’d use multiple virtual directories in IIS which all point to the same physical folder. We’d basically have one virtual directory (and one instance of the application) per tenant. We still have the benefit of one physical deployment, and because each tenant’s ‘virtual’ application runs in its own AppDomain, the caching problem is no longer an issue. Each virtual directory is configured for the URL of its tenant and the running instance of the application can still retrieve the actual tenant’s data from the master database based on the URL. Each virtual directory can run in its own Application Pool which can be set up to run under a Windows account which is specific to the current tenant. This allows us to use Integrated Security when connecting to the tenant-specific database. So each tenant would only have access to the master database and its own specific database.

Obviously, this approach would require a bit more effort in our ‘management module’ (which is yet to be written). Whenever we need to add a tenant, we not only have to create a specific database for the tenant, we’d also have to create a new windows account for the tenant, set up a new virtual directory, and an application pool if each tenant indeed runs in its own application pool under it’s own Windows account.

Apart from the management module, i’ve modified the application to work with this approach in just a few days work. But, nothing is final just yet… so now i’d like to hear from you guys whether this is a good approach or not. What are the possible problems we need to take into account? Is this really still multi-tenancy? I guess the opinions on this will be divided, but it does largely solve the issue of multiple deployment and multiple configuration files. True, the configuration is now mostly in the master database and those settings need to be maintained as well. However, they could now be maintained without having to redeploy the application which is a plus. Some settings could even be modified by the tenant itself (at least, the users who have the proper privileges to do so).

So anyways, i’m awaiting your feedback

UPDATE: i’ve been told that i can simply use a different cache region for each tenant in NHibernate’s 2nd Level Cache… so i could avoid having to use multiple instances of the application. Still not sure on whether one instance for all tenants would be better than one instance for each tenant though.

This prompted the following comment by Bryan Watts:

So Controller is responsible for disposing of something it did not explicitly create? That’s a little presumptuous isn’t it?

> if you own a reference to an IDisposable instance, you are responsible for properly disposing of that instance.

I agree (and I’ve read your other article). However, in this case, Controller clearly does not own its dependency.

And Bryan is right. For a lot of languages, memory management rules can be summarized (somewhat simplified) like this:

• If you create an instance of a class, you are responsible for releasing/freeing/disposing it.
• If a factory creates an instance of a class, the factory is not responsible for releasing/freeing/disposing it. This burden falls upon the object which requested the object from the factory
• If you receive an instance of an object, you should not release/free/dispose it, unless you received the instance from a factory

So where does this leave us with the example code shown above? Clearly, we did not create the instance of the disposable dependency, so in theory, we are not allowed to dispose it. Then again, if we weren’t using Dependency Injection we would have probably instantiated the disposable dependency in our constructor (or when we first need it). In that case, we would have been responsible for disposing the disposable dependency.

You could argue that the IoC container is in some way a factory which creates and supplies the dependency to our class. So according to the rules stated above, it would be allowed to dispose the dependency. The downside to this is that this means that our class (the Controller) has knowledge of the lifecycle of the dependency. We know that the dependency has been configured with a Transient lifecycle (which means the container creates a new instance whenever an instance of its class is needed) so we can safely dispose it. The problem is that this knowledge is implicit. It is nowhere visible in the code of the Controller class. The only place where this lifecycle is configured explicitly, is in the configuration of the IoC container so it would be wrong for the Controller to assume anything about the lifecycle of its dependencies.

After all, suppose someone makes a change to the code of the dependency which makes it possible to be used with a different lifecycle. Suppose someone makes the change to the dependency and then configures the IoC container to always return a singleton instance of this dependency. If nobody thinks about the fact that the Controller still assumes a Transient lifecycle of the dependency, the disposal of the dependency will not be removed from the Controller class. Once a Controller instance is created, and then disposed, it will try to dispose the instance of its dependency, which will cause problems because that instance is supposed to be used by other objects as well!

So what do we do? Do we dispose of the dependency in the Controller class because we know it’s safe (until someone changes the dependency and/or its configuration with the container)? Or do we simply not dispose it (since we’re not really allowed to do so without some kind of implicit knowledge of the ‘outside world’)?

Neither option sounds very appealing. The first option could lead to problems depending on possible future changes outside of the Controller class. The second option is already problematic because a disposable dependency is not being disposed of as fast as it should be. The disposal would basically be postponed until the instance is collected by the garbage collector.

Another alternative is to have the container inject a factory which can create transient instances of the dependency, instead of having the container inject the transient instance directly. This would fix all of the discussed problems: the controller would not need to dispose the injected dependency, and can safely dispose the instantiated disposable instances. However, having to create factories simply to avoid this problem seems a bit cumbersome. The implementation of the factory could lead to other discussions as well because of how the factory should create the actual instance and its dependencies, which is exactly the reason why we wanted to use an IoC container in the first place. True, the factory could resolve the instance directly through the IoC container, but then you’re just moving the problem to a different class without actually fixing it properly.

This entire problem has been described as the Component Burden by Hammet, the original creator of the Windsor IoC container. Obviously, this is something that the container should solve for us instead of us having to worry about it. Ideally, the container would keep track of all of the disposable dependencies it has created to satisfy the creation of requested components. Then when the requested component would be released, the container could safely dispose of each transient disposable dependency it created to create the requested component. Luckily, the Castle developers have implemented this behavior recently, so if you’re using a recent build of the Windsor IoC container, this problem should no longer occur and you shouldn’t be forced to dispose of dependencies that were injected into your objects.

The reason why i’m not using it yet is because there were many attempts at implementing this, and most attempts had considerable downsides to them. This one has been committed to the trunk, so it’s probably a good solution, but i’m not going to use this solution in production systems until it has had some more time to prove itself. So for now, i’m sticking with my approach where i manually dispose the injected dependencies. I know it’s not the best approach, but until the real solution in the container is somewhat more proven to work without downsides, manually disposing the dependencies seems to be the lesser of multiple evils.

Keep in mind that i’m only talking about the Windsor IoC container. I have no experience with StructureMap, NInject or Unity, so i have no idea how these containers deal with this problems. If anyone with knowledge about these container could shed some light on how they deal with this, i’d be very interested in reading it

My mistake was that after i was done with the components that i asked Windsor to resolve, i simply disposed them (which in turn would dispose their dependencies) and i figured that would be enough, since my components are registered as Transient. That means that each time you request a Transient component, you get a new instance. This led me to believe that Windsor wouldn’t need to hold a reference to the constructed components so i figured that simply disposing them would be enough, since they are IDisposables. Disposing them is a good thing obviously, but because the container was still holding references to the requested components, that’s still a lot of memory that is being wasted because even though you’ve disposed them, they aren’t eligible for Garbage Collection until they are no longer accessible. And because the container kept references to them, they remained accessible and were never collected. And there was my memory leak. Oops.

In order to prevent this problem in the future for myself and anyone else who reads this, let’s go over a few examples which should make it clear how you should make sure that your components are properly released so they are eligible for garbage collection.

Let’s start really simple. We have an IController interface and a simple Controller class which implements the IController interface:

In our tests, we’ll use the following method to create and configure the container:

The IController interface is registered with the container, and the container will return a new Controller instance (because of the Transient lifestyle) whenever someone requests an IController instance.

The following test highlights the memory leak that i was experiencing:

When you request a component from the Container, it keeps a reference to that instance in its Kernel’s ReleasePolicy object. If you merely dispose of your requested component, the ReleasePolicy still holds the reference to the component. This is what caused my memory leak.

So how do we avoid this problem? It’s pretty easy actually:

Instead of just disposing our controller, we tell the container to release it. The container in turn knows that because IController inherits from IDisposable, it should dispose the Controller instance. It also removes the instance from its Kernel’s ReleasePolicy object and once your own reference to the Controller instance goes out of scope, it’s eligible to be collected by the Garbage Collector.

As you can see, it’s very easy to make sure your components are properly released and eligible for garbage collection. But what about possible dependencies of your components? Let’s take a look.

Suppose we define the following interface and implementation:

The dependency doesn’t actually do anything, but bear with me

We modify the IController interface and Controller implementation like this:

And then we modify the configuration of the container like this:

Whenever we request an IController instance, the container will construct a Controller instance and will pass a MyDependency instance to the Controller’s instance constructor. The question now is: does the container also track the instances of a requested component’s dependencies? The answer is: no

We request an IController instance, which is tracked by the container. The IController’s Dependency property contains an IDependency instance, which was also created by the container. But as the last line of the test shows: the container does not track instances of the requested IController’s dependencies.

So what does that mean? If the dependencies don’t require any cleanup, then this is great. We simply need to release the requested component, and the component and its dependencies will all be eligible for Garbage Collection. But what happens when the dependencies need to be disposed? Let’s take another look.

We modify the IDependency interface and MyDependency class so it looks like this:

Now let’s see what happens with the Controller’s dependencies when we release the Controller:

The container holds no references, but the Controller’s Dependency instance is not disposed! Notice however that the Controller has been disposed by the container. As i’ve mentioned earlier, if you own a reference to an IDisposable instance, you are responsible for properly disposing of that instance. So we modify the Controller’s Dispose method so that it looks like this:

The previous test will now fail, because the Dependency will be properly disposed.

NOTE: i certainly don’t recommend to implement your Dispose methods like i just did. This is just a simplified example. The proper way to implement the Disposable Pattern is discussed here.

Anyways, i hope it’s clear now how you can make sure your IoC usage does not cause memory leaks, and that everything is properly disposed of.

Those of you who’ve been reading this blog for a while know that i’ve often stressed the importance of reducing the number of remote calls. Yeah that’s right, i just linked to 7 of my own posts in a single sentence :p

Anyways, there were a few people who thought that my preference for batching queries/service calls was actually a case of premature optimization, and that it was therefor evil and not something you should be doing until it was actually necessary to do so. The speaker of the presentation explained that there is a difference between premature optimization and pro-active performance management. Performance and scalability simply do not come for free, and you have to keep certain things in mind if you want your system to have those qualities.

Now, before i go further, i would like to state that i do believe that clean, simple and reusable code is something that developers should always strive for. I also believe that you should try to limit the number of times you hit the database, or the number of remote service calls you make in a single business transaction. Those goals often seem to contradict each other. There aren’t too many data access layers that allow you to easily perform multiple queries in a single roundtrip while still making sure that each query is reusable in a different context. It gets even worse when it comes to remote services. As you undoubtedly know, a lot of industry experts will r