The Inquisitive Coder - Davy Brion’s Blog

Most teams have some kind of technical lead in place. They’re often referred to as the ‘dev lead’, or the ‘lead developer’ or the ‘lead programmer’ or whatever. Some people are great fits for this role, and some simply aren’t. Below is my list of what i think makes for a great technical lead. Note: i’m not claiming that i live up to all of this, although i do try to.

• You obviously need strong technical skills. You are responsible for the final result, so you better make sure that there is a solid technical foundation for the team to build upon. This doesn’t mean that you should build this foundation entirely yourself. Preferably, you involve your teammates into this as much as possible. You’re also responsible for fixing technical issues that your teammates can’t solve. You either fix it, or when that’s not possible you should figure out an acceptable workaround. Be sure that your teammates are fully aware of the details of the solution or the workaround.

• You have to be able to teach your teammates. As a technical lead it is your duty to improve the skills of your teammates. No matter how good you may be, if you can’t transfer that knowledge and those skills, you’re not doing a good job as a technical lead. If there are some core principles or practices that you want them to apply to their work, you need to make sure that each and every one of them really ‘gets’ it. You need to be willing to invest the time and effort it takes to achieve that.

• You need to trust your teammates. This isn’t always easy, especially when it’s about someone who hasn’t progressed as far or as fast as you would’ve liked him to. But it is definitely necessary. If a teammate realizes that you trust him, he will usually respond with improved output. It might not always be everything you hoped for… but either his effort, or the quality of the work will improve. Keep trusting the teammate, and eventually both will improve. Which actually happens a lot sooner than most people would think.

• Stimulate self-organization. I know a lot of people don’t believe in self organizing teams. And to an extent, that disbelief is somewhat valid. There always has to be one or a couple of people who have some kind of leadership role (be it officially or not). Those leadership roles do not prevent self organizing teams though, in fact, i’d say they enable them. Do not simply assign tasks to your teammates all the time. Organize planning meetings where members can choose the tasks they will do. Make sure there is always some kind of balance though. You really need to prevent that person A always gets the shitty tasks or that person B always get the funnest tasks. Mix it around it a little and try to make sure that the fun/shit ratio is never out of whack.

• Don’t keep the coolest technical tasks for yourself. If the project requires some kind of technical research to use a new library or to figure out an approach to a new problem or whatever, make sure all of the teammates get the chance to do these kind of things once in a while. It’s not always possible to do so, but when you can, it really pays off. Morale goes up, trust goes up in both directions, and everybody improves.

• Try to prevent overtime as much as possible. If you’ve been given an impossible deadline, try to convince management that it’s simply not doable. Fight for it if you must. But realistically speaking, you can’t extend every deadline they throw at you, and sooner or later you’re gonna be in a situation where you and your teammates are going to have to put in some overtime to get everything done on time. Don’t tell your teammates they have to do it. Ask them if they want to do it. For some people, this doesn’t always make a difference but for others, it makes a huge difference. And this one is very important when you’re doing overtime: do not go home early while your teammates are still working, because it will come back to bite you in the ass eventually.

• Remember that you are responsible for the final result. If something goes wrong, it really is your fault. Never put the blame on one of your teammates. Not within the team, and definitely not when managers are present. If a teammate screwed up and the problem made it into the production version, then that is your responsibility and your fault.

• Give credit where credit is due. If a teammate did a great job with something, give them that credit and make sure everyone else knows about it too, especially management. Never take credit for the work of a teammate because that is simply one of the lowest possible things you could do.

• Finally, you need to realize that your teammates are not your developers. I’ve often heard technical leads say things like “yeah, my developers…” or “my guys … “. No. They are your teammates, no matter what your role in the team is. Having a leadership role does not mean you are the boss of your teammates or that you can boss them around. If anything, it destroys morale and the results of your team will suffer tremendously from it.

And that’s my list… did i miss anything?

Just read the following post on the performance of try/catch and throwing exceptions. In the author’s small (and completely unrepresentative of real-world scenarios) performance tests, it showed that throwing exceptions is a quite costly operation.

Ah… if only it were that simple. Throwing an exception is indeed a costly operation, compared to most other statements. But let’s get real… the performance ‘hit’ of throwing exceptions in real-world scenarios is completely negligible unless you’re doing it in a tight loop or a piece of code that is being executed pretty much all of the time under heavy user load. In practically every other case, you really won’t feel the difference performance-wise.

Anyways, because of the perceived performance hit, the author concludes the following:

Code defensively rather than catching exceptions wherever possible.

I’m not entirely sure what the author really means with this, after all there are probably plenty of developers who consider throwing exceptions when something is wrong instead of letting things blow up as ‘coding defensively’. But if he’s talking about the ‘defensive coding’ variant which often entails returning error codes or null references to avoid exceptions, and then checking for those return values all over your consuming code, then i could not disagree more.

I’m probably biased because i once had to maintain a codebase where the authors (to this day i believe they were relics from a C++ museum… and no, i don’t dislike C++ but ‘typical’ C++ programming in .NET is just not ideal) tried to hide exceptions from everyone who used their code because ‘application developers don’t understand exceptions anyway’ (sidenote: can you imagine that? not only did they suck tremendously, they were so arrogant to think they were so much better than ‘typical’ developers). So they had a few try/catch blocks in low-level pieces of their code, and when they caught an exception, they would just return a null reference or some kind of old school error code.

My first problem with this, is that it leads to code which is littered with null checks and the likes. It hurts readability, and i’d even go as far as to say that it hurts maintainability because it really is easy to forget a null check if you’re writing a piece of code where a certain reference really should never be null. And you just know that the null checks will be missing in some places. Which actually makes debugging harder than it needs to be. Had the original exception not been caught and replaced with a faulty return value, then we would’ve gotten a valuable exception object with an informative stacktrace. What does a missing null check give you? A NullReferenceException in a piece of code that makes you go: wtf? this reference should never be null!

So then you need to start figuring out how the code was called, and why the reference could possibly be null. This can take a lot of time and really hurts the efficiency of your debugging efforts. Eventually you’ll figure out the real problem, but odds are that it was hidden pretty deeply and you spent a lot more time searching for it than you should have.

Another problem with null checks all over the place is that some developers don’t really take the time to properly think about the real problem when they get a NullReferenceException. I’ve actually seen quite a few developers just add a null check above the code where the NullReferenceException occurred. Great, the exception no longer occurs! Hurray! We can go home now! Umm… how bout ‘no you can’t and you deserve an elbow to the face’? The real bug might not cause an exception anymore, but now you probably have missing or faulty behavior… and no exception! Which is even more painful and takes even more time to debug. That original exception with the stacktrace sure sounds appealing now, doesn’t it?

Btw, why is it that the kind of developers who avoid certain language features because of ‘performance problems’ (usually based on faulty information) are often very likely to make mistakes such as remote calls in loops, memory leaks due to lousy event handling schemes, or my personal favorite: writing a HighPerformanceTimer class which uses Windows API’s because ‘it times much more accurately’ and then in the same class, walking down the stackframe at runtime to figure out where the HighPerformanceTimer instance was called from for ‘easy logging purposes, dude’

Well like i said, maybe i’m just biased…

Here’s the situation: a couple of months ago we started developing according to a new architecture. Obviously, you need infrastructure code for this. For the first project, we just put the infrastructure code into the project’s solution and everything was easy. We could make changes as we needed them, and it enabled us to ‘grow’ the infrastructure into what we really needed. Then came the second project. I was reluctant to extract the infrastructure code in a separate reusable assembly because i felt it would lead to less flexibility to make changes. So i copied the infrastructure classes into the new project. Obviously, some changes were made in the classes of the second project, which weren’t ported to the classes of the first project. Add another project or two, and you can see the problem

So now we’re trying to figure out how best to move forward. I’ve got 3 options in mind:

1. Infrastructure code as a separate project, binary ‘framework’ dependency per ‘client’ project
2. Infrastructure code as a separate project, ‘framework’ dependency (in source form) per project (as in: copying the code of a specific version of the ‘framework’ into the project’s own repository)
3. Each project just contains the infrastructure code in their own project and there is no specific ‘framework’

The way i see it, each approach has its pro’s and con’s:

1. Infrastructure code as a separate project, binary ‘framework’ dependency per ‘client’ project
• Pro’s
• The code only has to be maintained in one place
• Everybody can benefit from changes
• Con’s
• Can make debugging harder because you can’t step into the framework code
• Requires a lot of discipline for versioning and distributing updates to ‘client’ projects
• The infrastructure code has to have a lot of extensibility points so each application can add extra functionality




2. Infrastructure code as a separate project, ‘framework’ dependency (in source form) per project (as in: copying the code of a specific version of the ‘framework’ into the project’s own repository)
• Pro’s
• Does not have the debugging issue
• Code only has to be maintained in one place (in theory)
• Everybody can benefit from changes
• Con’s
• The infrastructure code has to have a lot of extensibility points so each application can add extra functionality
• If people change the infrastructure code in their project, all changes should be sent upstream to the ‘real’ infrastructure repository, or extension points need to be provided in the original infrastructure code so upgrades of the infrastructure library still offer the same possibilities for the specific project
• Still requires versioning discipline, although it probably wouldn’t need to be as strict as with Option 1




3. Each project just contains the infrastructure code in their own project and there is no specific ‘framework’
• Pro’s
• Highly flexible… each project can freely make changes to make the infrastructure behave exactly as it needs to for the project
• Con’s
• Leads to multiple ‘versions’ of many of the classes… when a new project starts, which versions of each class should be used?
• Starting a new project contains boring set-up work which is basically just copy/pasting existing classes from previous projects

The reason i’m posting this, is because i’d love to get your feedback on this… what other pros/cons can you think of for each approach? Which approach would you recommend? Is there another approach we haven’t thought of?

Ayende has released his NHibernate Profiler into private beta and i was lucky enough to be able to play around with it.

There’s no installer, so it’s just xcopy deployment. I don’t really like installers so the xcopy deployment is actually a plus in my book. I just put the files into an NH-Prof folder and that was that. Using it in a project currently requires adding a reference to one of the profiler’s assemblies and then adding the following line of code in your application’s startup routine:

After that, you simply start the client and run your application. As soon as NHibernate does something, you can see the results immediately in the profiler:

My test application immediately sends 2 queries in a MultiCriteria batch which is correctly picked up by the profiler as one statement. It also immediately alerts me that i executed those queries outside of a transaction. Pretty nice.

The profiler also can give you more information on the number of entities that were loaded in a session, and it can also show you which entities where loaded. But it looks like there’s still a bug there:

As you can see from the SessionFactory statistics, we’ve loaded 37 entities. Now, the SessionFactory statistics keeps statistics for all Sessions it created, but it also clearly shows that there has only been one used Session so far. The profiler however says that there were no entities loaded in this session:

In my second Session in the application, i use one query to retrieve the entities of 3 different classes. You can easily get NHibernate to print the SQL statements it generates, but they are not formatted so they’re not always very readable. This profiler makes the generated statements a lot easier to read:

But now that i’ve executed my second Session, i can suddenly see the entity details of the first Session:

The SessionFactory’s statistics show that we’ve already loaded 151 entities, and 37 of those were actually loaded in the first session. Those 37 entities are shown in the details of the second session however and i have no idea which entities where loaded in the second Session. Hopefully it’s an easy bug to fix because this is a piece of functionality that looks very useful for troubleshooting.

Another thing i like a lot is how the profiler properly replaces parameter values in the formatted queries:

Now you can easily just copy/paste the formatted query and execute it in whatever tool you prefer to see the actual output of queries.

Now, as you could see, there was a query where i used joins to combine the result of 3 different tables. If i would rewrite the code so i don’t use the joins and simply rely on NHibernate’s lazy loading to fetch the related data, the profiler would show me something like this:

As you can see, the profiler detects the notorious SELECT N+1 problem which is a very common mistake that a lot of people make with ORMs, or generated DALs for that matter. But wait, it gets better:

It also notifies you when you send too many queries in a Session! Very neat stuff.

I’ve only used the NHibernate Profiler on a small test application and i’m already pretty impressed. In the next couple of days i’m going to experiment with it on a real application and then i’ll post some more feedback on it.

Update: the bug i mentioned is already fixed in a newer build

I just read the following post by Robert C. Martin. In it, he says the following:

If you can enumerate the states, and events, then you know the number of paths through the system. So if Given/When/Then statements are truly nothing more than state transitions, all we need to do is enumerate the number of GIVENs and the number of WHENs. The number of scenarios will simply be the product of the two.

To my knowledge, (which is clearly inadequate) this is not something we’ve ever tried before at the level of a business requirements document. But even if we have, the BDD mindset may make it easier to apply. Indeed, if we can formally enumerate all the Givens and Whens, then a tool could determine whether our requirements document has executed every path, and could find those paths that we had missed.

The company i work for (Item Solutions) actually has a tool for this: Genesis. After reading Robert’s post and discussing it here at the office, we decided to introduce this tool to a wider audience.

Genesis is more than just a tool. It is a methodology that we’ve been using for our internal projects for over two years now and that we are starting to commercialize. Before i start explaining how it all works, i’d like to show you how we can monitor the coverage of the requirements:

This is the global overview of one of our projects where we use the Genesis methodology. Each circle represents a functional module of the application we’re working on. Some of these circles are entirely green, which means that the requirements of that functional module are completely covered. Some of the circles still have a red part in them. This shows us that some of the requirements for that part have not been implemented yet.

The Genesis web application allows you to click on each circle to drill down into that functional module. In the screenshot, you can see that the circle for the Project Management module still has a red part in it. As we drill down into the module and the parts it consists of, we can get to the following view:

As we can see here, we have a couple of use cases within this part of the application that are entirely green. We also have two that are entirely red, which means we haven’t done anything for those use cases yet. And then we have a couple which are mainly green, but still with a bit of red in it. If i click on one of those use cases, this is what shows up on my screen:

As you can see, there are a couple of business requirements which are green, and below them we list the automated tests which cover this specific requirement. You can also see a red requirement, and the tool tells us that are no tests that cover that specific requirement.

That’s just a small overview of how we can monitor the development progress of our projects (there’s actually a lot more that we can see, but i might cover that in future posts about Genesis).

By now, you’re probably wondering: so how does it all work? Obviously, i can’t give away too many implementation details, but i can tell you how we use the tool and the methodology.

It all starts with a requirements document obviously. The requirements of the use case i showed earlier are written down like this:

Every time our continuous integration build runs, Genesis analyzes all of the test results and all of the data in our requirements document. In the Genesis tool, we can click on each business requirement to generate a specific code attribute for that requirement. When we write our tests, we put those attributes on top of the test methods. When Genesis analyzes the results and requirements, it processes the links between the requirements and the tests. If all of the tests that are linked to a requirement succeed, then the requirement shows up in green. If there are no tests for a requirement, or if existing tests for a requirement fail, the requirement shows up in red with some information about the problem. Genesis will either tell us that the tests for that requirement are no longer working, or that they are missing. Not only that, it will also notify us if the requirement changed since the test was written.

All of this gives us a tremendous amount of information and feedback, which is easily consulted by developers, analysts, project managers, clients, and pretty much everyone who’s been given access to the data. It’s also possible for a developer to ‘comment’ on the requirements… those comments will also be shown by the tool so the analyst and/or the project lead immediately know that something might be wrong with the requirements. This actually enables an effective line of communication when team members aren’t sitting together in the same physical location. We actually use this methodology with remote team members as well, and we’ve had less communication-related delays ever since we started doing this.

Finally, i would like to stress that this methodology can be a supplement to your existing process, instead of an “all-or-nothing” approach. It imposes no restrictions on the way you write your automated tests. You can place the requirement attributes on your unit tests, on integration tests, or whatever other kind of tests you write. All we need is data from the test results, which we then match to the data in the requirements document.

Also, i’d like to note that we currently only support .NET and Java but the system is open for extension so we can add support for other development environments as well.

Finally, you can get some more information about Genesis here.

There were some comments on a previous post about how the problem described in that post is somehow typical for applications that use ORMs. I really could not disagree more.

It seems that even today, in 2008 mind you, we still have a lot of people who are convinced that ORM usage can never be as efficient as the more classic data access approaches. Now, i don’t even want to get into the whole debate about where business logic belongs (but if you think it belongs in the database you no doubt have better things to do than reading this blog), but one thing that does bother me tremendously is that a lot of people discard ORMs because they simply don’t know how to use it properly.

An ORM is a tool. Nothing more, nothing less. Well, it is a pretty powerful tool and, as with any other powerful tool, improper usage of said tool can really cause a lot of problems. Should we discard the tool because a lot of people never took the time to figure out how to use it properly? That would be kinda stupid, no?

It seems to me that a lot of people seem to have this misconception that using an ORM essentially leads to data-fetching in tremendously inefficient manners. They see the tutorials where only the ‘get-by-id’ functionality and the lazy loading features are shown and they somehow think that’s all there is to it. They hear all the horror stories about projects that performed terribly because the developers used an ORM and they blame the tool, not the developers. Nevermind the fact that there are plenty of projects that use more classic data access approaches who perform like shit as well.

So let’s try to get a few of these misconceptions out of the way, shall we?

• You can create highly efficient queries with an ORM tool, and you can actually do so in a manner which enables high developer productivity
• ORM’s are not slow by definition. Using them wrong (just like with any other data access technology) can be tremendously slow however. Who’s at fault?
• ORM’s do not use a shitload of memory. Improper usage of them however can lead to excessive memory usage. Blame the developer, not the tool.
• ORM’s do not lead to lazy developers, who are doing lazy coding by relying on lazy loading. Bad developers lead to lazy coding by relying on lazy loading.

So, for those who think that ORM’s can never work ‘right’, i have only one question: are you absolutely sure you know what you’re talking about?

In response to my last post where i showed how you could fill entities with the resultset of a stored procedure, i was asked if it was also possible to fill entities and their associations if the stored procedure returned all of the necessary data. I looked into it, and it’s possible, although it did take me some time to figure out how to actually do it.

First of all, here’s the modified stored procedure:

As you can see, this returns all of the columns of the Products table, as well as the columns of the Categories table. The goal is to let NHibernate execute this stored procedure, and use the returning data to give us a list of Product entities with a Category reference which is already set up with the proper data.

The mapping of the named query now looks like this:

We map each column of the Product table to its correct property of the Product class. Notice that we defined the ‘Product’ alias for this part of the data. Then we use the return-join element to map the joined properties to the ‘Product.Category’ property. This might look a bit weird at first. You have to specify the alias of the owning object (which in our case is the ‘Product’ alias), and then you need to specify the name of the property of the owning object upon which the other part of the data should be mapped (in our case, the ‘Category’ property of the ‘Product’ object).

Now we can retrieve the data like this:

I first tried to use the IQuery’s generic List of T method which i had hoped would give me a generic list of Product entities. But i couldn’t get that working. So i tried the regular List method, and it turns out that NHibernate doesn’t just give me a list of Product entities… it gives me a list where each item in the list is an object array where the first item in the array is the Product entity, and the second item is the Category. Each Product entity’s Category property references the correct Category instance though. So you can get the product instances like this:

There’s probably an easier way to just get the list of Product entities from the named query, but i haven’t found it yet

As i had mentioned earlier, one of the NHibernate related posts i had on my TODO list was “Getting Up To Speed With NHibernate’s Second Level Cache”.

Luckily, Gabriel Schenker beat me to it and wrote a fantastic post about it. It is by far the best documentation on the 2nd level cache that you’ll find for now, so be sure to check that out if you’re looking into using some of NHibernate’s more advanced caching features.

A short while ago we needed to fetch the data for some entities through a stored procedure for performance reasons. We already use NHibernate in the typical way to fetch and modify the data of this entity type, but we just wanted something so we could also use the resultset of the stored procedure to populate the entities. One of my team members spent some time figuring out how to get the data returned by the stored procedure into the entities without actually having to write the code ourselves. Turns out this was pretty easy to do. Let’s go over the solution with a very simple example.

The stored procedure i’ll use for the example is extremely simple, and you’d never need to use this technique for such a stupid procedure. But in the situation we faced at work, the stored procedure was obviously a lot more complicated. So the stored procedure for this example is just this:

This just returns the product rows for the given CategoryId parameter. Again, you’d never do this in real life but this simple procedure is just used as an example.

Now, the structure of the resultset that this procedure returns is identical to the structure that the Product entity is mapped to. This makes it really easy to get this data into the Product entities. Just add a named query to your mapping like this:

And this is all you need to do in code to get your list of entities from this stored procedure:

Is that easy or what?

Now, suppose that the stored procedure returns more columns than you’ve got mapped to the entity. You can still use this approach as well, but then you’ll need to specify which return values map to which properties in the entity like this:

Notice how the CategoryID and SupplierID columns are mapped to Category and Supplier properties, which in Product’s mapping are mapped as Category and Supplier many-to-one types, so basically references of type Category and Supplier respectively. NHibernate basically just takes care of all of the dirty work.

Big weekend so far… Fabio Maulo invited me to join the NHibernate development team, which i gladly accepted

It’s a great project and a great team as well. I’m very proud to be a part of that from now on. I’m so psyched about this that i don’t even know what to say