I just came across the Manifesto for Half-Arsed Agile Software Development thanks to the wonderful world of Twitter. I’m gonna shamefully copy the text of the manifesto from the original website:

As funny as this is, it’s really quite sad as well. I don’t think i’m going out on a limb here if i say that the large majority of us have seen and experienced this with a striking similarity. And i think i know why. I’ve only spent time in two ‘enterprise’ companies but from what i’ve heard from a variety of people, it’s pretty much the same deal in most of them. Simply put: the typical enterprise culture leads to widespread incompetence, the majority of which is hidden from higher levels of management which eventually results in an environment where it is virtually impossible to work in an efficient manner. Whoa, i kinda dropped a bomb there didn’t i? Allow me to explain why i believe the above statement to be true.

I believe the root cause of all of the typical problems in the enterprise world to be the Peter Principle. In the enterprise world, a large group of employees is typically interested in climbing the corporate ladder. Some of them will get promoted because they did a great job. If they do a great job in their new position, they might be promoted again. Eventually, most people will end up in positions at which they perform at a level that is no longer good enough to ever be promoted again. Quite (very?) frequently, those people produced more overall value for the company in their previous position than they do in their current position. Unfortunately, none of them (save the occasional exception) will ever go back to that previous position. As good as those people were in their previous positions, they are simply incompetent for the job they’re currently doing.

Not only is it bad enough that you’ve got a lot of people who are too incompetent to do their job to a satisfactory level, you have to keep their superiors in mind as well. After all, they are the ones who promoted them. And they have superiors too. And you can safely assume that very few of them will be willing to own up their mistake in judgment to their superiors. That is, if they even realize that they made a mistake. Either way, either the incompetence is not noticed (which only points to more incompetence one level up) or it is actively hidden. Brushed under the carpet for nobody to notice. Obviously, that can’t be good for the quality of the work that is supposed to be done at every level of the hierarchy below this one. Problems will show up in pretty much everything that gets done by the lower levels. Rules, policies and guidelines are put in place to combat these problems and to try to keep the situation under control.

This in turn leads to decreased motivation, which is especially deadly for those who do have the skills and talent to right a lot of wrongs. You’re either losing time and efficiency because of people who no longer care, or because of people who simply can’t do what they’re supposed to do. Some will persevere, only to be promoted until they too are part of the problem. And the others will leave and prefer to work for smaller companies where everything seems to go “a lot smoother and easier”.

With that in mind, go over the manifesto again and ask yourself the following question: how could it possibly work in such an environment?

I happened to come across some blog posts and tweets that once again mentioned how evil it is to use comments in code. Some people still like to advocate the “if you need comments, your code sucks!” sentiment. As is often the case with this kind of statement, the only correct (or dare i say realistic) point of view is: it depends!

I generally agree that you should strive to avoid comments in code. That is, you should write your code in a way that doesn’t require comments for the reader to easily understand and grasp what the code is doing. However… there are situations where comments are helpful (or even required), since you simply can’t write everything in such a manner that it doesn’t require any comments.

To back up my point, i’d like to point you to this challenge i posted over a year ago. That code is clean. But pretty much everyone could use some comments to understand why that certain approach was necessary since it’s just not that obvious when focusing solely on the code. That code applies a certain pattern which isn’t well known, but there is a very specific reason why the pattern is needed. And i still believe that most developers need comments to understand it properly. That’s not to say that the guy who wrote it (which happened to be me) is better than those who’re going to have to maintain it. Hell, i wrote the code, and i would certainly be glad to see some comments in that code if i had to make a change 6 months after the fact.

The key thing to remember is this: don’t blindly follow what the books and/or the ‘legends’ say. If you need to write some code in a very non-obvious way, then you could really make things easier for those who need to maintain it (which could very well be you, btw) by including some comments to explain why a certain solution/pattern was chosen. There are some details you simply can’t show through good naming practices or clean code in general… some things simply need to be explained and it’s quite possible that a WHY comment or two benefits not only you but whoever is going to read the code more than some kind of excessive and futile exercise in making it as easy to understand as possible without comments ever will be.

In short, strive to avoid the need to write comments by writing clean code. But don’t be afraid to use comments wisely either when clean code simply doesn’t cut it. There’s nothing wrong with that. After all, we don’t all have the luxury to restrict our efforts to contrived examples.

It’s been about a month since i caved and finally joined Twitter after having earlier said that i wouldn’t. When i joined Twitter, i said this:

If i like it, i’ll keep using it and i’ll gladly state that i was wrong about the whole thing.

So here it is: i was wrong about the whole thing! Overall, i actually like it a lot and i’d probably find it pretty weird if i had to go back to going without it. But it’s not all perfect though. So, this is a list of likes and dislikes that i’ve noticed after about a month of twitter.

Likes:

• You can get in touch with everyone. You can reply to a tweet by someone ‘famous’ or that you admire or respect a lot or whatever, and actually get a response back. Well, you’re never certain to get a response but it’s highly likely that they at least read your reply. Getting a response is like icing on the cake. Likewise, people you’ve never heard of before will send you tweets to ask you a question, give their opinion on a post you wrote, or just reply to something you said. I often check the person’s profile to see if they might be interesting to follow, and indeed, some people i enjoy following have been added to my list that way.
• You can get into really interesting discussions merely by replying to other people’s tweets or by asking a question or making a statement. It’s a pretty easy way to get some other perspectives on certain topics.
• You can sort of take the pulse of the community. When something happens (like say, a big company cutting the funding for a popular project or a big company releasing yet another train-wreck) you’ll immediately know how a lot of people feel about that.
• It’s fun! If you follow the right people, tweets can range from insightful, to thought-provoking, to funny, to downright hilarious.
• It’s asynchronous communication so you spend time on it when you want to, instead of being interrupted by it (though that might depend based on the notification settings of your particular client). I’m involved in a conversation with 2 people right now, but that’s not preventing me from doing something productive (though you could argue that writing posts like this is hardly productive, but hey, it could’ve been a productive post as well! :p).

Dislikes:

• People can, and will, talk about anything. Sometimes that’s good, sometimes it’s bad. I’ve unfollowed a couple of people merely because they talk about politics or religion (or worse: both) too much. Which is too bad because they do have interesting things to say about technical stuff. Obviously, what you want to read on Twitter varies from person to person, and i’m sure people have unfollowed me for not liking the things i talk about as well. I try to restrict my tweets to technical/development topics, but occasionally there will be something else in there. I generally follow (and keep following) people who stick to those same topics.
• The 140 character limit can be pretty frustrating. For regular tweets, it’s usually enough but once you get into a conversation it can quickly be a bother. Especially if multiple people are involved because repeating their account names takes up a lot of your 140 characters.
• I wish Twitter would store which Tweets you’ve already read, or which ones have already been retrieved by your Twitter client(s). I guess i’m in the minority here, but i actually like to go over all of them to make sure i didn’t miss anything good. I guess very few people use Twitter this way, but it would definitely make things easier for people who use it like i use it. For instance, i have a twitter client on my iPhone, one on my Mac at home and another on my Windows PC at work. Each of those clients will show tweets that i’ve read in at least one of the other clients already and i have to manually look for the ones i haven’t read yet. It’s not a big issue obviously, just a minor inconvenience i guess.
• As interesting as some discussions are, some of them aren’t. Even worse, some of them are simply annoying and as far as i know, there’s no way to tune out of a conversation if you’re following both (or more) of the people involved. This luckily doesn’t happen frequently, but when it does it would be nice to have some kind of “hide this conversation” button

It’s obviously quite possible that some of the dislikes i mentioned are due to my noobness so if anyone has any tips regarding those issues, i’d love to hear them

Most people in the software development industry will (hopefully) agree that estimates are important. You need estimates to predict the cost of development and how long it’s going to take. Without them, there is no way you could promise a delivery date to a customer or come up with a total cost estimate. Both of which are tremendously important when your core business is based on delivering software to clients. It is therefore very important to make sure your estimates are as good as they can be, and that you review your estimates with the final actual figures so you can figure out where you need to start improving (either your development process, or your estimation practices).

But there are some things you need to watch out for in any organization which takes estimates seriously. Most development shops have estimates at the task-level. These task-level estimates are often used during the planning of the next iteration, but were most likely also used in the initial total estimate of the project. I certainly don’t doubt the importance of those task-level estimates, but i do think they should be used cautiously.

If too much importance is placed on staying below the estimates, or at least not going over them, you run the risk of growing a culture where developers let the estimates have too much influence on the quality of the work they’re doing. Some developers will strive hard to routinely stay below the estimates. Some developers will work in fear of going over the estimate. In both cases, it leads to technical shortcuts that will be taken. Those shortcuts will consist of skipping some tests here and there, skipping a bunch of tests altogether, decreasing willingness to refactor code that could use it and worst of all, no longer keeping a clean design in mind. Granted, i’m painting a very bleak picture here but it does happen with some people and the negative results can not be underestimated.

Let’s consider the overall effect of these shortcuts. A shortcut here or there is not abnormal in some circumstances, but it should be exceptional. It should not become routine, it should be frowned upon and i’d even go as far as recommending a “no shortcuts allowed unless the team approves of it” policy. A shortcut here or there doesn’t take a lot of effort to fix and because of this, it can often be done as soon as the original reason of the existence of the shortcut (typically: a deadline) is no longer an issue. An accumulation of shortcuts however leads to not only an obviously bigger workload to fix all of the shortcuts, but it also impacts a lot of other code that shouldn’t have been impacted in the first place. Once this happens, you’ve got yourself an unhealthy code base and it’s only going to get worse until it eventually dies at a younger age than it could have reached.

Now, imagine a culture where estimates are given as much importance as quality. If you’re approaching the estimate and you’ve still got a lot of work left to fully complete the task (and with fully completed, i really don’t mean watching it work on your machine as you manually test it), then you consult with the rest of your team members. Estimate how much time it would take to fully complete the task, and see if the increased time over the original estimate is doable. If you’re already behind your overall schedule, it’s probably not doable unless there are many subsequent tasks that are dependent on this particular task. But if you’re ahead of schedule, you’re probably better off accepting the fact that you’re going to go over this task’s original estimate in order to do it right. If you do so, the extra hours you spent on this task should (in most cases at least) not lead to extra hours on future tasks. Conversely, if you’ve finished a task well below the estimate, it might actually be a good investment to refactor some ‘bad’ code that you’ve come across while working on your task. Or to add some missing tests here or there. That certainly doesn’t mean that you need to spend all the available remaining time you have but you shouldn’t run to start on a new task as soon as humanly possible either. Follow the boy scout rule here… you won’t spend that much extra time on it, but overall quality can improve greatly from this.

In fact, i’d even bet that the time it takes to do it right will in a large majority of cases still be less than the sum of the original time that was spent on a task hastily done and the time that will be spent on bugs that resulted from that hastily done task, not to mention the impact it could have on the development of future tasks simply because of the suboptimal implementation of the hastily done task.

Finally, it’s also important to think about why there was a need to go over an estimate. There can be a variety of reasons for this, and one of them is that the task was simply badly estimated. If that indeed turns out to be the case, make sure that you learn from it to prevent this from happening in the future. When we find bugs in our code, we try to prevent those bugs from ever occurring again. We should have a similar attitude to our estimating practices as well. If an estimate was simply too low, make sure that a similar task in the future won’t be estimated too low again or you will again introduce a possible problem in your (future) project.

And that, in my opinion, is also a good reason to decide to go over the estimate if you can. If you go over the estimate and all people involved learn from the bad estimate, then everyone basically benefits from it. Future estimates should become more accurate, and no code was harmed in the process. You might have lost some money on it, but at least you only lost it once instead of losing it over and over again in similar future circumstances.

One of the things that pretty much every developer has had to face at least once in his career is the issue of working overtime (typically, to meet a deadline). If you’re lucky (or smart), this is a rare occasion. For some however, this is either a habit, or happens frequently. There are actually quite a few developers who are proud of being able to do 60-80 hours a week. And they could not be more wrong.

There’s nothing wrong with putting in a few extra hours occasionally. Sometimes, it just needs to be done. But, as soon as it is required more than occasionally (and with occasionally, i mean every couple of months), something is wrong. First of all, you’re not gonna produce anything good if you’re doing a lot of overtime. At first, it starts with a couple of late nights. The first couple of nights, the work you do is still ‘good enough’. It’s not gonna be great, but hey, it’ll do. After a couple of nights though, if you’re still doing a lot of extra hours, there’s gonna be a noticeable negative effect on the work you’re doing. And you know what the worst part is? You, of all people, won’t notice it. You’re working hard, you’re trying to catch up with the schedule, and you’re doing the best you can. Or at least, you think you are.

In reality, you are hurting yourself, the project and your teammates more than you could imagine at that time. Here’s the deal: there’s a reason why most people are working 8 hours a day on average. If you routinely work more than 8 hours a day, your work is going to suffer from it (it depends on the job obviously, but for software developers, i’d argue that this is definitely the case). If you do a bit of overtime occasionally, then you, nor the project will suffer from it. It’ll just take a little bit more out of you than what you’re used to. But once it becomes a routine, you, as a human being, will start performing worse and worse as time goes on. Inevitably, that deterioration will become very noticeable in your work. And here’s the sad thing: you’re putting in the extra effort to catch up with the schedule. But by doing so, you’re actually performing worse and you’re hurting the schedule. Here’s what happens: you spend a few late nights at the office. At first, things go ok. You’re getting things done and it appears to be working. You keep spending late nights at the office. After a few days (really, it happens that fast), the quality of your work starts deteriorating. And you won’t notice it until you get back the next day. So then what happens? You start cleaning up the mess you made the night before. Which only means you’re getting even more behind schedule. So then you’ll need even more time to catch up with the schedule. But you’re already spending this much extra time at work, and the problem only seems to get worse! Well, yeah… that’s the whole point that i’m trying to make here. You know what’s the best thing you could possibly do in those situations? Go home at 3 or 4 in the afternoon, go have some fun, relax, and get a good night’s sleep. You won’t believe how much more productive you’ll be the next day.

But where is this need to do overtime coming from? I’d argue that, in the large majority of cases, this is due to management mistakes. Either the wrong things were promised to the customer, or your management doesn’t have a clue of what is really going on. So, if you find yourself in a position where you routinely need to do overtime, you really need to start asking yourself whether or not you’re working for the right company. If you routinely need to do overtime to reach a deadline, that tells you that the project management (and highly likely, every kind of management beyond that) is pretty messed up. I don’t know about you, but i really don’t want to work for people who don’t know what they’re doing. If you’re doing a lot of overtime, your management is either clueless or stupid. Probably a combination of both.

As for the developers who are proud of their 60-80 hour work weeks… what does that tell you about them? There truly are very few people who can keep on producing great results beyond the regular 40 hours a week. Something tells me that very few of those developers who brag about their extra hours really fit into that category. If they really did fit into that category, they’d let their results speak for themselves instead of needing to brag about the amount of effort they’re putting in.

I’ve never understood why IIS by default configures newly created application pools to recycle every 1740 minutes.  That means that by default, our ASP.NET applications are restarted every 29 hours.  And for what?  Are we, the .NET community, so bad that we really can’t write code that can keep running without problems for more than 29 hours?  Well, considering the overall lack of knowledge regarding memory management in .NET, i actually wouldn’t be surprised if that turned out to be the case for a very large portion of .NET development teams.

And indeed, if you browse the web to find the reason why application pools are configured to recycle automatically periodically, you’ll be hard pressed to find a reasonable answer that doesn’t pertain to memory issues.  It’s like the community in general has pretty much accepted the fact that our web applications (or service layers hosted in IIS) will need to be recycled to avoid memory problems.  To make things worse, i’ve frequently seen people discuss workarounds on how to keep things working properly after an application pool recycle in mailinglists for various projects.  And again i ask myself: why?

I’ve always been of the opinion that if your code requires periodic restarts to keep working correctly, then something is clearly wrong.  There is a bug in your code somewhere and you need to fix that, instead of restarting the process occasionally to make the problem ‘go away’.  Guess what, it’s not going away, and if the load on your application increases, the problem will only pop up more frequently.  So what are you going to do about that? Make the application pool recycle even more frequently?  Put in another server to share the load?  Perhaps we should focus more on fixing the actual code.  You’re leaking resources somewhere, and that is what you need to fix.

In my company, it’s just standard procedure to disable application pool recycling on all application pools.  Our applications simply have to keep working without application pool recycles.  It’s that simple.  If we run into a memory leak that causes problems, we focus on the actual problem and we fix it.  We’ve ran into plenty of memory leaks (though it hasn’t really happened a lot in the past year, guess why?) and we’ve fixed each and every single one of them that we’ve ran into.  Side note: i’m talking about .NET code that is running on the server.  While i’m pretty sure that we do better at memory management in our Silverlight clients than most other Silverlight-using-shops do, you just never know for sure with Silverlight, partly because there are so many memory leaks in Silveright itself.  But for .NET code running on a server, a memory leak is simply inexcusable in my opinion.  I mean, it can happen to the best of us (i’m pretty much responsible for most of ours to be honest), but it’s not about who introduced the memory leak.  It’s all about how you deal with it.  And recycling the application pool periodically is not dealing with it, it’s living in denial.

The only viable reason to recycle application pools that i can think of is in the situation of a shared hosting provider.  If you’re offering shared hosting, it only makes sense to recycle application pools occasionally since it’s quite realistic to assume that most of those application pools will not be in use constantly.  If a pool is recycled, it will not consume any resources until the next request for an application in that pool is received.  In the context of shared hosting, that actually makes sense.  In the context of real applications, that makes little sense unless you’re hosting a lot of them and plenty of them are hardly being used.  Still, is that really a good enough reason to enabling application pool recycling by default?  I obviously don’t think so.

In my opinion, we, as a community, really need to start focusing more on memory management in .NET and on making sure that our applications can keep running without problems.

One thing that frequently bothers me when reading technical blogs or when talking to people about software development in general is that people often get extremely enthusiastic about some new kind of technology, library, framework or tool that they recently started using or heard other people talking about.  In essence, there is nothing wrong with that.  We are after all geeks, and the people who like to write or read technical blogs or talk about software development with others are typically passionate geeks and it’s only normal that we’re enthusiastic about tools, frameworks, libraries or pieces of technology in general.  The thing is… we really ought to be a bit more careful in the way some things are often hyped in an uninformed manner.

I have a very simple rule when it comes to avoiding hype: if you can’t say something bad about it, you don’t know it well enough (yet).  And until you are able to say something bad about a library, framework, approach, technology, tool or whatever, i simply can’t believe all of the good things you’re saying about it without some form of skepticism.  In life, nothing is perfect.  In software development even less so.  Every single piece of technology or approach has its drawbacks, issues and problems.  And quite often, those things only become noticeable after having truly used it on a real project for a while.  I think most of us will be able to agree on that.  So really, what does that tell you about someone when that person is extremely positive about something, without being able to mention any downside?  That either tells me that you don’t know what you’re talking about or that you’re not even using whatever it is that you’re hyping on something real (hobby projects don’t really count).  So does it really make sense for one to listen to it?  Conversely, if you are very positive about something but are also able to mention the negative points that you’ve encountered (they don’t have to be huge, and can in fact even be minor) then i’m immediately much more inclined to believe what you’re saying. 

Granted, my approach frequently prevents me from being an early adopter.  But if/when i do become an adopter, i will be a pretty informed one.  In many cases i’ll know about the negatives before they bite me, and will either guard myself against them, work around them if i have no other alternative or simply choose to ignore the technology altogether if the positives don’t outweigh the negatives.  The only downside that i’ve experienced so far from this approach is that i’ll never be one of the cool kids on the internet and among the hype-hungry developers.  Then again, that’s probably more of an upside than a downside, come to think of it

Looking back over the past 5 years or so, i can say that i was late in adopting unit testing, agile development, Inversion Of Control, Dependency Injection, ORM, and probably a bunch of other things that i can’t really think of right now.  But i’m pretty sure that i know all of those things pretty well by now.  A large reason for that (i think) is because i took my time and made sure that i made informed decisions before adopting something.  I experimented, thought about drawbacks, tried it in real projects, learned from mistakes that i made, worked around problems that were inherent with the piece of technology or approach in question and only then did i consider myself able to make informed statements about it.  That doesn’t mean you can’t talk about something that you don’t know fully yet.  A lot of very valuable blog posts are written by a lot of people who are learning things or are just experimenting with things.  The smart ones among them though, will be very honest about how much they know about it, how much they don’t know about it, and how far they are in the learning progress.

So, as Public Enemy said: don’t believe the hype!

Get informed instead.

A lot of people are of the opinion that isolating yourself at work, as a developer, is a bad thing to do.  Communication with your teammates is highly important, and thus, isolating yourself from your surroundings by sitting in a different or separate office or listening to your iPod is often spoken of as an inherently bad thing to do. But is it really?  The way i see it, this can be a good thing, and this can be a bad thing.  It pretty much depends on the type of person you are, what kind of people you work with, or what kind of environment you’re working in.

I can only speak from my own experience, but i frequently isolate myself from my coworkers and my surroundings by turning on my iPod and listening to some music while i work.  I don’t do it all the time, and it definitely depends on what i’m working on, or who i’m working with.  Obviously, if you’re doing pair programming then listening to your iPod is pretty much out of the question.  But if you’re all sitting together, do we really need to avoid any kind of isolation?  When i’m working on something complex, and i don’t really need to communicate with my teammates at that time, then there really isn’t anything wrong with listening to some music and getting some work done.  In fact, it often helps me focus on my work without getting easily distracted, helps me get into a good zone, and it would certainly surprise me if i were the only one who thought that way.  If you or some of your teammembers actually produce better results while they tune into their music, then this can’t really be an inherently bad thing, no?

But obviously, there will be many situations where you actually want to avoid listening to music.  Sometimes you might be working on something that requires you to consult with teammates frequently, whether it is about design decisions or to ask some feedback on implications of certain implementation details.  In those cases, you can end up pausing and resuming your iPod way too frequently so it might just be better to refrain from turning it on altogether.  Or when you notice that teammates are discussing something and it’s taking them a bit too long.  It’s hard to tell when a discussion is too long if you’re not hearing what they’re saying but when i notice 2 developers talking for more then a couple of minutes, i’ll often pause the iPod to check out what they’re talking about.  Maybe you can help by offering a different perspective or maybe you know an immediate answer to the question that they’re discussing.  If you can’t help, go ahead and press Play again.

The thing is… even if you do tune into some music at work, it’s important not to tune out of your coworkers completely.  Go ahead and listen to your music, but keep looking around occasionally.  Turn it off when you notice your teammates are talking and follow the discussion to see if it’s interesting to you or if you can help them.  Conversely, if you’re participating in a discussion and you notice that one or more of your teammates is listening to music while they should probably be involved or at least hear what the conclusion or decision was, go ahead and notify them of what’s been discussed/decided/concluded.  Also don’t be afraid to involve people who weren’t involved yet if you think that it might be important to them.  If it turns out that it wasn’t important to them, then all you’ve lost is a few minutes.  But if it was important to them, you might have saved a lot more than a few minutes by pulling them into the discussion.

As for sitting in a separate office… i’m not a fan of that.  I’ve always noticed the best results are achieved when all of the developers are sitting together in the same office.  In some organizations, senior developers, lead developers or ‘architects’ might be sitting in different offices from the people they’re supposed to be working with.  I don’t know about you, but i’ve never actually seen that work properly.  The typical on-the-floor communication that is so vital to successful teamwork is just reduced a lot of if some people are sitting in a different office.  Obviously, when working with remote coworkers, this problem only increases but that’s something i haven’t really found an efficient solution for either.  In some cases, communication through IM works great.  And sometimes it doesn’t.  It depends on how effectively you can communicate through IM, and how effectively the others can. 

But anyway, when you hear people talk about how developers that isolate themselves at work is a bad thing, don’t necessarily buy into it.  Like i said, it can be good or it can be bad.  As with anything in our business, it all depends.

I’ve been using Visual Studio 2010 for about 2 weeks now, and i’ve upgraded several of our projects to .NET 4 as well. I’ve noticed that i’ve hardly used any of the new features of C# or any of the new classes in the .NET framework. Granted, i don’t have a very thorough knowledge of everything that’s been added in C# 4 and i don’t know all of the new classes in the .NET framework. But i do know something about the new stuff, and so far, i haven’t really noticed anything compelling. Nothing that made me think “this is awesome!”.

I looked at the new WCF 4 features so i could see how Agatha could benefit from it, but apart from the improved REST support (which Andrew Rea is working on), there’s nothing that Agatha could really benefit from. Perhaps maybe the fact that you can reduce XML configuration for your WCF services but then again, the amount of XML config that you need for Agatha is already very limited compared to typical WCF services and the parts that you do need are things that i actually prefer to see explicitly in a configuration file instead of hidden away in some infrastructure assembly that your project is using.

As for the new language features… i like the whole dynamic thing, though i haven’t found a need to use it yet. The next time i see some old reflection code i’ll probably modify it to use dynamic, but i don’t really need to use tricks like that very often. Optional parameters are something that i can’t really get excited about, perhaps because i hardly ever use interop and more likely because i still believe in meaningful method overloads. In fact, it sometimes makes me think that C# 4 is moving closer to Dysfunctional Programming as opposed to at least being influenced by Functional Programming in C# 3. Named Parameters is something that i can see myself using to increase readability when it comes to using multiple overloads with confusing method parameters but that as well is something that i don’t frequently run into.

The greatly improved support for parallelism looks very nice, though i haven’t really needed that either. The new thread-safe collections look nice too, though every time i want to modify some code to make use of it, i notice that i typically need my locking to be just a bit more coarse-grained than what i can get from the new collections. Unless i haven’t looked hard enough of course, which is certainly possible.

As for Visual Studio 2010… from a performance point of view it is somewhat better than i expected but i really can’t say it’s an improvement over Visual Studio 2008 (again, from a performance point of view). I occasionally notice about 25% CPU usage (on a quad core system, so that’s pretty bad IMO) coming from devenv.exe while it isn’t actually doing anything. I can’t really say that i’m happy about that. The default color theme looks butt ugly IMHO, but at least there’s a plugin to change that. Which i haven’t actually done yet though

Now the thing is… maybe it’s just because i haven’t looked hard enough, or that i’m missing some cool new stuff, but isn’t this very different from when .NET 3.5 came out? The Linq extension methods and the ability to use Expressions was highly welcome and i guess a lot of us kinda went overboard with it (and some probably still do). But at least that showed that those new features were not only highly welcome, but very popular and in general just great new additions to the C# language. But with .NET 4 i really find it pretty hard to find anything to get really excited about. And just to be clear on this: that’s not to be critical of Microsoft or anything like that. But i’m just wondering: “is it just me or is this really not that special?”

So what do you think? Are there great new additions to C#/.NET that i’m probably missing and that you are enjoying a lot? What are your favorite additions to the platform? Or are you, like me, not really all that excited about the new stuff?

I sometimes still get questions from people who want to expose their entities through their WCF Services.  Regardless of whether these are entities that are populated through NHibernate or any other ORM, this is just not a good thing to do.  Many people prefer to accept and return entities through their services because they believe this is an easier programming model.  They believe that it takes less work than mapping to DTO’s and that as a whole, this solution is much more manageable.  Rest assured that this is a fallacy.  Any perceived benefit that you’ll get from exposing entities outside of your service layer will only last a very short time and will quickly be dwarfed by added complexity, increased maintenance overhead and a performance overhead which must not be ignored. 

In this post, i’d like to take the chance to explain the downsides to exposing entities through services.  Though i’ll probably miss quite a few of the downsides (feel free to add to the list through comments), the ones i will mention are IMO important enough to take note of.

Exposing entities to clients means your clients are very tightly coupled to your service(s)

Entities are a part of your domain.  These entities in your domain can change for various reasons.  Sometimes because functional changes are required, but quite often also for optimizations (whether they are for performance reasons or to improve the clarity and maintainability of your domain).  Functional changes can impact your clients, though that is not necessarily the case.  Optimizations hardly ever have an impact on your clients (other than possibly improved response times from your service calls obviously).  If your service layer accepts and returns domain entities, each possible change is highly likely to have an impact on your clients.  And this impact is not cheap.  In the best case scenario, it means updating your service contracts, regenerating your service proxies and redeploying your clients.  In the worst case scenario, it means making actual changes to the code of your clients.  And for what? Because of changes that shouldn’t have impacted your clients in the first place?

Ideally, your clients are as dumb as they can be.  They should know as little as possible about the actual implementation of the domain because that implementation is simply not relevant to them.  They should present users with data and give them the option to modify that data, to trigger actions and to perform certain tasks.  They should focus squarely on those tasks and pretty much everything else is typically better suited to be done behind your service layer.  If you build your clients with no real knowledge of the actual domain model, but of DTO’s and possible actions to be performed then you can reduce the level of coupling between your clients and your services substantially.

Many of the people who prefer to expose entities often claim that going for the DTO approach introduces too much extra work and too many extra, seemingly unnecessary classes.  For starters, they don’t want to write code that maps entities to DTO’s.  First of all, the amount of code that this requires is in reality very small, not to mention very easy.  Secondly, you can just as well use a library such as AutoMapper to take that pain away from you.  And contrary to what you might think, there is a big performance gain to be had from returning DTO’s over entities, but i’ll get to that in the next section.

Entities are hardly ever the most optimal representation of data

I think we can safely say that most applications need to show data in the following 3 ways:

• In a grid view, either as a total listing of all instances of a certain type of data or the result of a search query or some kind of filtering action
• In dropdown controls or anything else that lets users select pieces of data
• In edit screens where a piece of data needs to be displayed in its entirety, perhaps even to be modified by the user

There are undoubtedly more ways in which data can be presented to the user but i think it’s safe to say that most business applications will certainly rely on the following 3 ways quite heavily.

In the case of a grid view, you’re frequently showing data that is related to more than one entity.  You’ll often need to include the name or the description of some associated entities.  So what exactly is it that you want to do in this situation?  Do you want to return a list of the main entities of the grid view, which all have their required association properties filled in so you can display the columns that you need in the grid view?  Do you actually need all of the properties of these entities (for both the main entities and the associated entities)?  Odds are high that you’re going to be returning a lot more data to the client than you actually need.  And that is what is realistically going to hurt the performance of your system.  Any piece of unnecessary data that you transmit to your clients has a cost associated with it.  The unnecessary data is retrieved from the database.  The entities are then serialized at the service end.  Then they are transmitted to the client.  Then they are deserialized by your client.  All of this is pretty costly, so the more unnecessary data that is included in this operation, the more your performance and the responsiveness of your client (not to mention your database and your server) is impacted negatively.

In the case of dropdown controls or anything else that lets users select pieces of data, you typically only need very few of the properties of that piece of data.  In many cases, the primary key and a name or a description are sufficient.  Do you really need to transmit the entire entity every time for usages like this? Again, keep in mind that all of that extra data that will never be used by your client needs to be retrieved, serialized, transmitted and deserialized again.  Surely, this is an awful waste, no? 

And then there’s the case where a piece of data needs to be displayed in its entirety.  In these cases, you will almost always need all of the properties of the entity that is displayed, but you’ll most often also need to show other data (things that can be selected, or linked to the main entity).  This other data will in most cases fall into the previous category where you’ll only need very little information about the actual entity.  If you’re smart, you’ve chosen the DTO approach to retrieve this data for the data that can be selected, and in that case, you already have all of the infrastructural code in place to project entities or data into DTO’s.  So you might as well reuse it for the main entity as well since you already have the capability to do this.

Always keep in mind that your entities will frequently either contain more data than needed, or less data than needed.  As such, it just doesn’t make much sense to expose entities to your clients since they are hardly ever optimal for client-side usage.  If you really want to think about performance, stop worrying about the supposed cost of mapping to DTO’s (which is truly negligible) and start focusing on what your actually sending to and from your service because this is far more costly than any kind of DTO-mapping really is.

Must your data really come from entities?

If you are displaying data to your user, does that data really need to come from your domain model?  Does it really need to be retrieved by populating a collection of entities to then return them to the client?  Again, keep the form of the data in mind when thinking about this.  In many cases, as i mentioned above, an entity is not the most optimal form of the data that your client needs.  So why even retrieve it through entities? Sure, asking your ORM to retrieve a set of entities based on a set of criteria is often the easiest thing to do, but if the easiest path were the best path, the overall quality of software projects wouldn’t be in the sad state that it’s in today.  If the form of the required data is not identical to the structure of an entity, it’s often far more optimal to simply populate a DTO directly from the data.  With NHibernate, you can easily do this by adding a list of projections to your query and then using a ResultTransformer to populate the DTO’s based on the direct output of the query.  In this case, no entity instance ever needs to be created when you’re just retrieving data, and no extra mapping between the entity and the DTO’s needs to be performed.  Your data access code simply retrieves the resulting data from a query, and puts that data directly in your DTO’s.  There’s no reason why usage of an ORM should prevent you from doing this.   Once again, this approach will offer far more performance benefits than avoiding DTO mapping at all costs ever can.

What about the behavior of your entities?

Do your entities have any behavior in them?  If not, they are already more of a DTO than a true entity.  In fact, if your entities have no behavior at all, you could even wonder why you’re using an ORM in the first place.  Now, behavior can mean many things.  It could mean lazy loading of associations.  It could mean actual business logic.  Obviously, lazy-loading doesn’t (and shouldn’t!) work client-side, but what about your business logic? Do you have business logic that can be executed client-side? Or is it business logic that should only be executed behind the service layer? If so, how do you make the distinction between this to prevent client-side usage from these entities? Whatever you do, you’re pretty much opening up a can of worms that really is better avoided in the first place.

How are you going to deal with technical issues?

Accepting and returning entities from services introduces a host of technical issues that can be quite substantial.  Serialization and deserialization specifically are issues that you need to be worried about.  If you’re using an ORM which does lazy-loading of associations, this will certainly cause serialization issues that you need to work around.  You can either disable lazy loading, or you can make sure that your entities are always fully initialized (as in: always have their associations fully loaded) before they are sent back to the client.  Disabling lazy-loading will cause performance problems in your service layer, either in places where you don’t expect them to be or in places that you haven’t thought of before it’s too late.  Fully loading your entities and their associates before returning them is another performance nightmare waiting to happen so that’s really not an ideal solution either.  You can try to hook into the serialization process or even the lazy-loading features of your ORM but whatever you do in that case will be a hack that will cause issues sooner or later.  And again, all of these problems can very easily be avoided with a solution which, i hope you realize by now, offers plenty more benefits than any solution where you accept/return entities in your service.

Conclusion

Every single downside to exposing entities through services are issues that i have myself encountered in past projects, either ones i’ve worked on myself, or ones that i’ve seen other people work on.  If that’s not enough for you, then maybe you’ll find it interesting to know that some of the brightest and most respected people (like Udi Dahan and Ayende for instance) in the .NET community also actively recommend against exposing entities through services because of the same downsides that i mentioned, though they could probably give you even more downsides that i forgot to cover in this post.  These downsides are not figments of anyone’s imagination.  They are very real, and you really, really ought to think twice before dismissing this advice.