We have this rather large project at work, one that’s been in development for a few years now. This project has had many releases already, and from a business perspective, is quite successful. From a technical point of view, there were definitely some things that could’ve been better. The largest problem is that this project uses a very extensive code-generation process. This code generation process basically retrieves all of the database metadata and generates an entire Data Access Layer (based on basic ADO.NET), a shitload of automated tests to cover that entire DAL, a whole lot of extra classes which form a data-driven business layer (real business logic can still be added though), and again, a shitload of automated tests that cover the data-driven business layer.

Now, the people who originally came up with the code generation obviously had good intentions in mind. They wanted to maximize developer productivity so everyone could implement the required features as fast as possible. And that’s pretty much the reason why people turn to code generation: to increase productivity. Code generation however, is not a good way to do that. In the short term, it definitely increases productivity though. But it comes at a terrible cost: an incredibly large technical debt. The thing about code generation is that it’s basically a shortcut. When it comes to developing software, each shortcut brings some kind of technical debt with it, and sooner or later, you have to pay off that debt, or risk having your knee caps shattered (why yes, i am watching my Soprano DVD’s again!).

Now, the technical debt you incur by implementing a certain feature in the quick-n-dirty way instead of doing it properly from the start is in most cases small and easy to pay back if you don’t put it off too long (obviously, it’s best to simply avoid having to incur any technical debt in the first place). Generating an incredibly large library of code, and then using that code all over the place has an impact that you simply can not recover from when the project has been in development for a long time. If you want to make serious changes in your generated code so you could, for instance, reduce coupling to concrete classes, you may be in for a rough ride.

In our case, most of the things you could do with the generated code could be done in more than one way. Every possible usage scenario was being used at least somewhere in the application’s code base. And not just in a few places, but pretty much all over the code base. This made it pretty much impossible to make changes in the templates that would be used to generate the code, which would introduce compiler errors at least somewhere in the system. And not just a few of them, hundreds of them. Sure, you could start fixing them. And then you could start on your next change in the templates, and it would lead to a few hundred more compiler errors. If you’d simply have to change a bit of syntax here and there, it would be painful, but at least it wouldn’t be something you couldn’t overcome. But if you want to change the behavior of the generated code, you’re in pretty big trouble. All of the code that uses the generated code expects certain behavior to occur. Changing the templates would mean changing a lot of the real code.

When you’re in a situation like this, Michael Feathers (author of Working Effectively With Legacy Code) can’t even help you out. Hell, even Batman would run away like a frightened schoolgirl. Which is basically what we did too. We decided to leave the generated code and the real code as is for now, and we came up with a new architecture (with no code generation) which we’re going to use to develop the new functionality. The old code will slowly be migrated to the new architecture whenever we have room to do it, or whenever we need to make serious changes. It’s not perfect, but starting completely from scratch is an approach that wouldn’t make a lot of business sense anyway, and you have no guarantee whatsoever that the ‘new’ system will actually catch up with the ‘old’ system in a timely manner.

I think this serves as a nice example of how a code generation based approach can really come back to bite you in the ass like Jaws with a vengeance. Everything you gain in initial productivity costs you a lot of flexibility in the long term, which (also in the long term) ends up costing you more productivity than you ever gained in the first place.

Always keep in mind that code generation is usually a solution to a problem that wasn’t solved properly in the first place.

My fellow Dutch ALT.NET member Jan Van Ryswyck wrote a post about how bad commented-out code is. I also truly hate it when i see code that is commented-out. It’s such a waste of time because every time i see it, i immediately start wondering “is this something that we still need to add? Does it simply not work yet? And if so, why not? Who the hell did this? Why is he wasting my time?”. Putting code in comments and leaving it there is generally a sin.

However, there are two situations where i do find it acceptable. Dare i even say helpful. The first situation is when you’ve made a very non-obvious performance improvement to a piece of code that was proven to be slow by a profiler. In those situations, i generally leave a comment saying something like “the following block of code was originally written as follows, but was proven to be a performance bottleneck”. After that i have the original code in comments, followed by a description of what turned out to be the problem. After that, i put the modified code and then i’ll have another comment discussing why the new code avoids the problem. I have to admit that this generally looks ugly. However, it does tend to avoid other people looking at the modified code and thinking “hey, i can make this much simpler” and then proceeding to write a more readable version of that piece of code which might re-introduce the performance problem.

The other situation is in a TestFixture for ‘Learning Tests’ (couldn’t find a good link to describe this concept). Learning Tests are basically tests where you can experiment with a library that you don’t have too much experience with. Learning Tests can be very valuable, but they generally don’t need to be run after you’ve learned whatever it was that you were trying to figure out. But should you throw away the code after that? It might be useful to other team members. But i don’t like to put it on Ignore either because each ignored tests feels like a leftover TODO to me. So in that case, i also put the code in comments.

There’s a debate going on the ALT.NET mailinglist about whether or not teaching developers TDD for a project is worth the cost. I don’t really like the way things are discussed on that list nowadays, so i’m posting my thoughts here.

First of all, it’s definitely true that learning how to do TDD effectively takes some time and the learning curve is rather steep. If you need to bring people who are new to it up to speed, you definitely have to invest some time in that. So, is the time and effort spent on that worth it? I think it most definitely is. If you take the time to teach them how to do it properly, you can enjoy quite a few benefits.

For starters, the developers will improve their skills, not only in writing tests, but also in writing flexible and simple code. This can take some time, but if you’re coaching them properly you should get this situation under control in a couple of weeks (somewhere between 2 and 4 weeks depending on the number of developers that need to be coached, and their general skill level). The quality of the code (and the tests) that they’ll produce after this period can reduce your total future workload by a large quantity. I think most of us know what it’s like to work with code that is anything but flexible, and what a negative impact such a situation can have on your team’s productivity: things that should be easy to change, features that should be easy to add can become mind numbingly complex to implement without introducing bugs to existing code. I’ll gladly coach a few developers for a few weeks if that means avoiding a code base where people are afraid to make changes or where change in general is lengthy and painful.

Also, during this period where you have to do intensive coaching, you can still make progress in the project. It’s not like these developers have to learn these practices and principles on throwaway code. Have them work on real stories for the project. Double the estimates for those stories, and don’t assign any stories to the person(s) doing the coaching. Make sure they have enough room to help the developers that need the help and that the code (and the tests) are up to standards before the stories are considered done. If you’re coaching them right, you might be surprised how fast they’ll pick up the pace. You’ll definitely take a large productivity hit in the first couple of weeks, but after that your team’s productivity will go up rather fast and before you know it, you’ll have made up for all the lost time and with a healthy code base to boot.

Now imagine not investing that time and effort. Without even thinking about who’s going to write the tests, or when they’ll be written, suppose you decide not to spend the time and effort to teach them TDD. You start the project and everyone starts implementing features. Some people will write good, solid, flexible code. And some people won’t. Their code might stay in this unhealthy condition, or it might deteriorate further. But hey, your team’s productivity in the first couple of weeks sure is impressive right? Fast forward a month or two, then tell me how the productivity of your team is. It probably won’t be as high as it was in the first couple of weeks. Oh, and keep an eye on team moral. Chances are that some developers will be terribly frustrated by some of their teammates’ inability to write good code. The rest of the developers might be frustrated because some developers are too demanding and bitch too much about how bad their code is. All in all, chances are high that it’s not going to be a fun team to work on.

Now, as for the teaching and the coaching… this can be a relatively easy job, or it can be a royal pain in the ass. It all starts with the developers’ willingness to learn. If they are not willing, get rid of them. Yes really, get rid of them. You’re only doing them, yourself and the rest of your teammates and your organization a disservice by keeping them on staff. If they are willing to learn, make sure that they realize that you are there to help them. If they are afraid or hesitant to ask questions, then you are definitely doing something wrong. They have to be certain that you will do the best you can to help them, and that they won’t get ridiculed if they ask stupid questions. And let’s face it, we all asked ’stupid’ questions when we were learning these things so you sure as hell shouldn’t make another developer feel bad when he/she is in the same position you were a few years ago. Also, when they do good, make sure you reward them by showing them more trust. You’d be amazed what kind of a positive effect that can have on a developer who’s trying to learn a new approach. And never ever simply tell them what to do and what not to do. Make sure you explain everything! If they see you’re taking the time to make sure they understand why you’ve given the advice that you’ve given, they will usually appreciate it and they will certainly learn a lot more than they would’ve if you had simply told them not to do something.

All in all, deciding against teaching people TDD is one of the worst decisions one could make IMHO. It may not always be fun and it may not always be easy, but in the long term, the quality of your project will benefit greatly from it. If teaching them TDD is painful, then you’re either not doing it right, or you’re working with people you probably shouldn’t be working with anyway. But teaching TDD is generally not hard, as long as one party is willing to really teach, and one party is willing to really learn.

Just finished reading Michael Feathers’ Working Effectively With Legacy Code. The book’s main purpose is to teach you a lot of tricks on how to improve the testability of legacy code. As Feathers himself says, this book will not teach you how to go from bad legacy code to beautiful code in one go. That’s not even possible anyway, unless you start rewriting the legacy code. Instead, this book shows how you can take small incremental steps to get your code into a solid test harness which will then allow you to start refactoring to good code more safely.

The book is divided in 3 parts. The first discusses the mechanics of making changes in code. The second part consists of very conveniently titled chapters that discuss common problems when changing legacy code. Chapters with titles like “I don’t have much time and i have to change it“, “How do I add a feature“, “I need to make a change, what methods should I test?“, “Dependencies on libraries are killing me“, etc… You can quickly look at the table of contents and find the right chapter to help you with the problem you’re facing at that time. The third part of the book contains a series of dependency breaking techniques that are referenced throughout the book.

If you are working on a legacy code base, do yourself a favor and get this book. Put it next to your keyboard, and use it everyday. You won’t regret it.

Ahh, i’m finally able to use NHibernate again at work, so expect more NHibernate related posts in the future

Today i needed a way to add some functionality when NHibernate opens a database connection, and again when NHibernate closes the connection. When the connection is opened, i need to setup some context on the connection for auditing purposes and it needs to be cleared again when the connection is closed. So i started searching on how to plug this into NHibernate. As usual, this was trivially easy to do.

All you need to do is create a type that implements the IConnectionProvider interface. In my case, i only needed to add a bit of behavior so i could just derive from NHibernate’s standard DriverConnectionProvider class and add the stuff i needed at the right time:

Pretty simple huh? All you have to do now is to configure NHibernate to use this new ConnectionProvider in your hibernate.cfg.xml file:

Was that easy or what? If only all frameworks were extensible in such an easy manner

It’s always interesting to look back when you’re finishing up a project. This was my first project since moving from working at clients to working at our own office. We wanted to make some changes in our way of working so we decided to use some of the wacky ideas i’ve posted on this blog for this project. More specifically, we used the Request/Response Service Layer and my MVP approach for writing testable ASP.NET Web Forms.

The first 2 weeks were a bit rocky since the other developers had to learn about Dependency Injection, Inversion Of Control and using a mocking framework for most of our tests. In the first 2 weeks, we didn’t reach any of our time limits, but then again, we expected that because we knew it would take some time to get used to all of these new things. I actually thought it would take 3 weeks before we would start getting up to speed properly. Luckily, the other developers caught on pretty fast and before we knew it, we started beating the time limits.

Obviously, we did our weekly retrospective after each sprint, and i was particularly interested in how they felt about working according to these new concepts. For the server-side implementation of the Request/Response service layer, i didn’t hear a single complaint. Once they got used to it, they seemed very comfortable with the whole Request/Response thing and i think they really liked how the Inversion Of Control container just ‘magically’ wired everything up. For the client side Dispatcher (used to send Requests and fetch Responses) there was one complaint about having to explicitly clear the state if you had to make 2 trips for some reason. Now, the complaint was actually made because i changed the behavior in one specific case after a few weeks, which broke a few screens, so their complaint was definitely justified.

Another complaint that i heard twice, was that the tests for the Controllers of the Web Forms could get pretty complex or even annoying once you had to deal with complex screens. We never really found a way to make that easier, and i don’t really think it can be made easier, to be honest. Complex screens simply require a lot of testing. There’s not a lot you can change about that. I definitely prefer writing a bunch of tests over testing it manually all the time. Having said that, it’s still important to keep in mind that screens still need to be tested (thoroughly) manually from time to time. No matter how many tests you may have, they simply don’t cover everything that happens when a user is actually using it in a browser. By the way, i think i was the biggest offender of this rule.

So how did we do? The project is pretty much finished now… one team member is still working on the project to wrap it all up but he’ll probably be finished in a few days. We stayed under budget, and the issues that were reported by the project leader (who’s also playing the customer proxy role) were all fairly minor, a lot of them related to stupid UI issues. I think most of the issues were related to my total lack of solid ASP.NET knowledge actually

I’m also very proud of the overall quality of the code base. Everything is very readable, simple and flexible while there’s nothing bad hidden in there. It’s actually the first time in my career where i felt this way about a code base where i didn’t write the whole thing myself

Oh and to Joel and Sacha: thanks for doing a great job

When it comes to code, a lot of people are often very opinionated. Supercomputers couldn’t even keep track of the number of times i’ve looked at a piece of code and said “this sucks”. We often qualify code as crappy, bad, decent, clean, good, great, whatever else you can think of. The problem is that this usually comes across as a very subjective opinion. What one person might think of as crappy code can be decent or even good to someone else.

I think it would be better if we started talking about quality of code in terms of how healthy the code is. In a very abstract way, software is basically a life form. After its creation, it starts a life of its own. It tries to fulfill its purpose and to do that, it often has to be changed or improved. Depending on how well it was created, it might be capable of easily adapting to whatever is required of it. Sooner or later, the software will reach a point where it can no longer easily adapt, which usually is the beginning of the end of this life form as new software will be prepared to replace it. Ideally, we’d like to keep working software around as long as possible. Not only because of the financial consequences of not doing so, but because we really shouldn’t be redeveloping the same systems over and over again.

To keep working software around for a long time, we have to take care of its health. It’s important to keep it healthy at all times. As soon as rot is introduced in the internals of a piece of software, it either has to be removed as soon as possible, or it will spread and affect other internals. When too many internals are infected, it becomes very hard or even impossible to help the software. Just like humans need to take care of their bodies, it is important that the code of the software is kept in a healthy state.

So when can we say that code is healthy? When it is easy to modify or improve, without affecting other parts of the system in a negative way. How can we achieve healthy code? By keeping it clean, flexible and readable. To many people, that equals good code. To most managers, the term ‘good code’ doesn’t really mean anything. As long as the code makes money, it’s good code for them. But if the code is not healthy, it will stop making money sooner rather than later. Telling them the code is not healthy as opposed to saying it’s bad or crappy or whatever signals a possible business problem instead of simply hearing a developer complain about code he doesn’t like for whatever reason.

Simply put, communicating about code quality in terms of health allows you to focus on what’s really important (the viability of the software) instead of coming across as someone who simply doesn’t like the code because of personal preferences.

If you’re using my Request/Response service layer, or any other WCF service that might require sending large amounts of data over the wire, you quickly bump into some limits that WCF enforces by default. Among the billions of configuration options for WCF, there are luckily some options that allow you to easily send large amounts of data from a service to a client.

I typically use the following options:

For my binding configuration i usually set the maxReceivedMessageSize, maxStringContentLength and the maxArrayLength properties to their maximum values:

This example shows these settings for the netTcpBinding… i’ve also used them with the wsHttpBinding. Not sure how well it works with other bindings though.

I also set the maxItemsInObjectGraph setting of the DataContractSerializer to make sure i don’t hit the default limit if i have to send a large object graph over the wire:

You have to apply these settings both server and client side to get it working properly and you need to refer to these settings in your service and endpoint settings:

Now, i don’t recommend sending such large amounts of data through WCF services… but in the case of using my Request/Response service layer, the amount of data you’re sending over the wire pretty much depends on which kind of requests (and how many of them) you’re batching so i think it’s better to make sure that at least the configuration allows for it. So obviously, it’s best to keep an eye on the size of your messages to make sure you’re not doing anything crazy. Being able to send your entire database over the wire doesn’t mean it’s a good idea to actually do so

I recently had to do an impromptu technical interview with an applicant at work. I spent the last few years working at clients instead of our home office so it had been a while since i had to conduct a technical interview. So unfortunately, i couldn’t really think of great questions to ask. I did ask a few good questions (at least, i think so), but it got me thinking afterward about what would be a great way to conduct a technical interview. I came up with an approach that i think will be pretty good. I haven’t tried it yet, but next time i’ll definitely do it like this:

1. Ask if they know the Single Responsibility Principle (SRP)

   Some people will know, some don’t. If they don’t, explain it to them. If they do, let them explain it.

2. Ask them to talk about a class in the system they’re currently working on that violates the SRP

   In case they didn’t know about the SRP, this will quickly tell you how easily they can understand new concepts since you’ve just explained it. If they did know about it and were able to explain it correctly, they should be able to give you a good example. If they were able to explain it, but can’t give you a good example, that’s a sign that something isn’t right. It might be an indication that the applicant has trouble mixing theory with practice.

3. Ask them what they would do to fix the problems of that class when given the chance to clean it up

   This should give you some valuable insight as to how the applicant thinks about how code should be structured. I’m not even talking about big design stuff, just simple class design. The answer to this question will most likely enable you to start an interesting discussion about class design and writing code in general, while not distracting the applicant with fictional example classes. After all, we’re talking about code the applicant should know pretty well.

And that’s it. The discussion that the final question should lead into should tell you most of the things you need to know about the applicant. You can talk about TDD, writing code, refactoring, design, pretty much everything… Depending on the position that’s available and the skill-level of the applicant you could also start talking about architecture stuff.

I think this approach allows you to learn a great deal about the applicant’s technical skills in about 20 minutes. I personally don’t care how well an applicant knows a certain API or library, i’m interested in finding out how well the applicant can deal with new stuff, concepts that he/she may not have any experience with. Basically, how well the applicant can adapt to different technical situations.

Oh, and a quick tip to future applicants: mentioning that you read my blog is a tricky approach. While flattering, i may or may not feel the urge to figure out if you’re really reading it or if you’re just giving the posts a quick read

I recently asked you how you would introduce TDD to an audience that has little to no knowledge of, or experience with TDD. Oh, with a time limit of 10 minutes. This post is pretty much the outline of everything i hope to say in those 10 minutes.

What Is TDD

• It is an iterative and incremental way of working, on a coding level.
• It comes down to a series of little steps that we follow step by step, while never trying to do too much at once.
• We try to break down our coding activities in small, easy to manage pieces
• When we need to add/modify behavior in code, we:
  • write a small failing test (sometimes a couple of them, but never too many at once) that indicates the lacking of, or the incorrectness of that behavior
  • write/modify the code to make the test pass, but we do that as simple as possible, perhaps even quick-n-dirty
  • after the test passes, we clean up the code where necessary. While doing so, we focus on keeping the code clean and readable
  • run the tests we just wrote again to make sure everything still works and we didn’t break anything
  • run the entire test suite before committing our code to make sure we didn’t introduce bugs in other places
• We try to follow these rules for all of the code we write (not always possible though)

Why do we prefer this way of working?

• If we stick to the rules, we’ll always have clean code that works
• Our design (in general) is flexible, simple, and never more complex than it has to be
• It allows us to make changes in the code without fear and with confidence, no matter who wrote the code in the first place. After all, we have an extensive test suite which will alert us immediately when we make a mistake.
• We can minimize the time we waste debugging and trying to understand why something doesn’t work anymore.
• We won’t have any code that isn’t being used anywhere.
• We can increase the skill level and the effectiveness of every developer on the team.

How can you get started with TDD

• It takes some time before you can do TDD effectively. Don’t expect all of the benefits to come immediately.
• Read Kent Beck’s Test Driven Development By Example to get into the TDD mindstate
• Try it, and stick with it for a while
• Realize that you will need to learn some new tricks to write code that is easily testable. Read xUnit Test Patterns to improve your skills.
• After a while, you will not want to go back

What do you think? Anything i left out that i really should mention? Are there things i need to stress more or discuss differently? I still have a few days left to make changes, so let me know