Beware The Evils Of Code Generation
Posted by Davy Brion on September 22nd, 2008
We have this rather large project at work, one that’s been in development for a few years now. This project has had many releases already, and from a business perspective, is quite successful. From a technical point of view, there were definitely some things that could’ve been better. The largest problem is that this project uses a very extensive code-generation process. This code generation process basically retrieves all of the database metadata and generates an entire Data Access Layer (based on basic ADO.NET), a shitload of automated tests to cover that entire DAL, a whole lot of extra classes which form a data-driven business layer (real business logic can still be added though), and again, a shitload of automated tests that cover the data-driven business layer.
Now, the people who originally came up with the code generation obviously had good intentions in mind. They wanted to maximize developer productivity so everyone could implement the required features as fast as possible. And that’s pretty much the reason why people turn to code generation: to increase productivity. Code generation however, is not a good way to do that. In the short term, it definitely increases productivity though. But it comes at a terrible cost: an incredibly large technical debt. The thing about code generation is that it’s basically a shortcut. When it comes to developing software, each shortcut brings some kind of technical debt with it, and sooner or later, you have to pay off that debt, or risk having your knee caps shattered (why yes, i am watching my Soprano DVD’s again!).
Now, the technical debt you incur by implementing a certain feature in the quick-n-dirty way instead of doing it properly from the start is in most cases small and easy to pay back if you don’t put it off too long (obviously, it’s best to simply avoid having to incur any technical debt in the first place). Generating an incredibly large library of code, and then using that code all over the place has an impact that you simply can not recover from when the project has been in development for a long time. If you want to make serious changes in your generated code so you could, for instance, reduce coupling to concrete classes, you may be in for a rough ride.
In our case, most of the things you could do with the generated code could be done in more than one way. Every possible usage scenario was being used at least somewhere in the application’s code base. And not just in a few places, but pretty much all over the code base. This made it pretty much impossible to make changes in the templates that would be used to generate the code, which would introduce compiler errors at least somewhere in the system. And not just a few of them, hundreds of them. Sure, you could start fixing them. And then you could start on your next change in the templates, and it would lead to a few hundred more compiler errors. If you’d simply have to change a bit of syntax here and there, it would be painful, but at least it wouldn’t be something you couldn’t overcome. But if you want to change the behavior of the generated code, you’re in pretty big trouble. All of the code that uses the generated code expects certain behavior to occur. Changing the templates would mean changing a lot of the real code.
When you’re in a situation like this, Michael Feathers (author of Working Effectively With Legacy Code) can’t even help you out. Hell, even Batman would run away like a frightened schoolgirl. Which is basically what we did too. We decided to leave the generated code and the real code as is for now, and we came up with a new architecture (with no code generation) which we’re going to use to develop the new functionality. The old code will slowly be migrated to the new architecture whenever we have room to do it, or whenever we need to make serious changes. It’s not perfect, but starting completely from scratch is an approach that wouldn’t make a lot of business sense anyway, and you have no guarantee whatsoever that the ‘new’ system will actually catch up with the ‘old’ system in a timely manner.
I think this serves as a nice example of how a code generation based approach can really come back to bite you in the ass like Jaws with a vengeance. Everything you gain in initial productivity costs you a lot of flexibility in the long term, which (also in the long term) ends up costing you more productivity than you ever gained in the first place.
Always keep in mind that code generation is usually a solution to a problem that wasn’t solved properly in the first place.
September 23rd, 2008 at 5:20 pm
Interesting. I really see nothing in the article that to me indicates the problem was with code generation, it seems you inherited a project with bad code. How is this the fault of code generation? So if the original developers had not used code generation, they would have written good code, but when they went to code generation, the decided to write templates that would generate sh*tty code?
Did you think about this article before you wrote it?
September 23rd, 2008 at 5:24 pm
the problem was that a lot of the bad code was generated, and then used all over the place which makes it pretty much impossible to make changes to the generated code, which limits refactoring possibilities in general
Did you actually read the article before you commented, or did you just skim over it? :p
September 23rd, 2008 at 6:07 pm
I find this a bit vague too. Was the generated code altered? Or did new code rely on implementation details of the generated code? Generally speaking I can’t really see a fundamental difference between generating code from a high level description on the one hand and a compiling a program in written in a high level language to machine code on the other. (I can see practical differences though. For example, a programming language and its compiler are usually a lot more polished than ones own definition schema.) The major benefit of code generation isn’t necessarily the productivity boost you get when writing the code but of the benefit of keeping your definitions in one place when each definition may lead to multiple classes and interactions.
September 23rd, 2008 at 6:36 pm
the generated code was not altered… manually written code relied on implementation details and was very tightly coupled to the generated code. In small projects, that’s easy to rectify. In a large project where the generated code is pretty much used all over the place, you can’t just start cleaning it up one bit at a time. If you need to make a change in the generated code, you need to modify the templates. If you can’t change the templates in a way that it doesn’t break anything in the existing manually written code, then you’re in a pretty bad situation.
The compiler comparison doesn’t really hold up… the generated machine language will (in most cases) not be used by something else that needs to be maintained manually.
September 23rd, 2008 at 6:47 pm
“the problem was that a lot of the bad code was generated, and then used all over the place which makes it pretty much impossible to make changes to the generated code, which limits refactoring possibilities in general”
Again, I just don’t see your point. O/R entities and persistence code (which presumably this was) will be used everywhere, whether it was generated or not. I just don’t see how the fact that the code was generated in your example is relevant. By the sounds of it, you have bad code that is used extensively throughout the system. The code happened to be generated. If the code generated was good code, would it still be a problem?
September 23rd, 2008 at 7:12 pm
Like a couple people have said, it doesn’t really sound like ‘code generation’ is the culprit here, as much as ‘bad design generation’
I’ve seen lots of good code generation that create loosely-coupled systems that preserve the ability to change the template when needed.
Now, in my view, code generation shouldn’t be rushed to, instead relying on it as a method of eliminating a type of duplication.
September 23rd, 2008 at 7:15 pm
if the generated code was good and completely up to our standards, then it would have been less of a problem but it still would’ve had a lot of potential for creating future problems.
In this case, one of the biggest problems was that the manually written code was very tightly coupled to the generated code… trying to reduce coupling by using interfaces proved to be impossible because every possible way of creating one of the generated components turned out to be used at least somewhere in the system. We couldn’t even make that change without introducing hundreds of compiler errors in manually written code. Now, as i mentioned in my post, if it was merely a case of fixing some syntax errors or reference types, it woulnd’t have been that big of an issue.
Now, again as i mentioned in the post, if you need to change the behavior of the generated code, and the manually written code expects certain things to happen and certain things not to happen, then you’re in a pretty bad situation because you simply can’t change it, unless you want a couple of developers to fix hundreds of compiler errors for a few days. Once that’s done, you’ll probably need to modify a lot of manually written code where assumptions were made on the behavior of the generated code.
The point is, because it is generated, you can’t easily change it. The larger the project is, the harder it becomes to make changes to the code generation. And we can’t simply turn off the code generation either, because then we’d have to manually maintain all of that shitty code whenever database changes occur. All in all, it’s probably about a 1000 classes at this point, and a lot of them contain very similar code. Which probably would’ve been written in a more reusable manner had it been written manually from the start. And then we’d at least have more options on making changes when needed or wanted.
September 23rd, 2008 at 7:19 pm
I too am not convinced that Code Generation, per se, is the problem. There is bad code and there is good code – you can create either one by hand or by generation.
I have successfully used the Open Source tool MyGeneration to generate a DAL. My experience is that once you understand and buy into the model of the tool or framework, you don’t have a problem. If you misuse the tool/framework, you can cause problems for yourself.
It sounds more like the generated code was not flexible enough to handle changing requirements. This, to my mind, does not make CG evil. I suspect your team may have the same amount of work to refactor computer-generated code as with hand-generated. Thus, the technical debt is not in the method (CG) but in the design used to build the CG templates.
September 23rd, 2008 at 7:21 pm
@Corey
Yes, the design of the of the generated code is indeed pretty bad. However, due to the fact that a whole lot of that code is generated (and used), it becomes much harder (if not impossible in some cases) to try to improve that design.
i’ve seen good code generation as well, and if the generated code enables loosely-coupled systems, then that at least increases your options. But i do think that as the project increases in size, any kind of code generation eventually becomes more of a problem than the solution it was originally ment to be.
As for the duplication… maybe you’ve seen some really great code generators, but in most cases that i’ve seen, the generated code often has a lot of duplicated code in it. Or perhaps you ment duplicated efforts instead of duplicate code. Either way, writing reusable code could solve both issues and i think it offers you more flexibility in the long run.
September 23rd, 2008 at 9:11 pm
Seems like the problem is caused by tightly coupling the hand written code to the implementation details of the generated code. I don’t see how the problem is related to code generation at all. I.e. how would this issue be any different if the generated code was hand written? If anything, the generator can help you to get out of this mess with a lot less effort than refactoring by hand.
September 23rd, 2008 at 9:39 pm
if the generated code had been hand written, we’d at least be able to make changes in places where it’s actually causing problems. Because it’s generated, we either change the whole thing, or we don’t. If we change it, everything breaks and we need to fix _a lot_ of stuff at once.
For those who think merely improving the generated code through the templates is the solution to the problem, i wonder on what kind of projects you guys use code generation on. For small to medium projects, it’s probably not that big of an issue. For a large project, it’s really not that simple.
September 23rd, 2008 at 9:47 pm
Not to rag on you more, but your project is likely using code generation incorrectly. Code generation is for boilerplate, bog-standard, lowish-level code that never changes. Examples: generating the O/R layer, generating skeleton classes from established interfaces, generating stubs at various levels, generating CRUD sprocs in the db, generating full tests or stubbed tests, etc.
Used well, it does the 80% in no time flat, and the 20% tricky stuff has more time to be developed by hand. Note, code generation will never help you generate the whole nut! It also helps setup a lot of code that adheres to internal standards.
It’s not for:
- generating code in areas where the design is still in flux
- generating incorrect code
- generating incorrect architecture
Replace “code generation” in your piece with “hammer” or “guns”, such as “Beware The Evils Of Hammers”. It’s just a tool. Inherent in generators is the potential, like any other tool, for ignorance and/or malice. Most common issue is no easy way to control the output, and no versioning/backwards compatibility with template changes.
Saying a code generator is evil is like saying compilers are evil, because a code generator is basically a simple “compiler” for a DSL.
September 23rd, 2008 at 9:54 pm
except for generating the O/R layer and tests, i pretty much agree with that
but i didn’t really state that Code Generation _in general_ is evil. But when you’re generating too much code, and depending on it in too many places, i think it definitely is.
and seriously, the O/R layer never changes? Please tell me i misread that…
September 23rd, 2008 at 11:44 pm
I think you still have two options:
- drop the code generation: check in the generated files and remove the generation from your build files;
- for making a tiny change in one particular source file, introduce a conditional in the appropriate template.
The second option is the worst one, of course, but could be justifiable when you’re gradually moving away from your current solution.
A mix of both options would be to have your template skip certain files, which might be least hacky.
September 24th, 2008 at 6:16 am
I have to agree with you. I see code generation as too often used as a productivity boost at the beginning of a project. It works fine at first but then as the project/product grows and evolves the template get more complex, more workarounds (hacks) are added, the original designers move on, etc.
For an example of how code generation can get complex very quickly, see this article from Visual Studio Magazine . The question was “I’m working with Entity Framework (EF), and I’ve run into a problem. All of the generated properties are marked with the DataMember attribute … Can I remove these attributes?”. The answer runs to almost four pages in the printed magazine!
September 24th, 2008 at 3:59 pm
Lots of comments on this article. And fierce debate.
. The second reason I think is that code generators are fun to write and they make the developer feel smart, sort of like a meta-programmer. After all you understand the language so well that you can program the program.
Developers seem to love generating code. I know I do. I think it’s because of emotional reasons rather than rational ones though. Producing lines of code without actually having to work certainly is an appealing idea to any programmer
In almost all cases, and for me this includes generating the O/R layer, generating skeleton classes from established interfaces, generating stubs at various levels, generating CRUD sprocs in the db, generating full tests or stubbed tests, etc… , code generation is at best a hack to get around language limitations (f.i. the webservice proxy ugly thingies), and should be avoided when possible. I think I should be regarded as a CodeSmell.
In above comment :
Replace “code generation” in your piece with “hammer” or “guns”, such as “Beware The Evils Of Hammers”.
Try the other one : “Beware The Evils Of Guns” … Sounds like pretty solid advice to me
.
Ooh, and one thing I truly don’t understand is generating tests. If they can be generated why not define a higher level tests that contains the logic (and/or architecture) you now put in the test generator.
September 24th, 2008 at 4:33 pm
“Ooh, and one thing I truly don’t understand is generating tests. If they can be generated why not define a higher level tests that contains the logic (and/or architecture) you now put in the test generator.”
taking that idea one step further leads me to this:
why not create higher-level reusable classes that contain the code that otherwise would be generated?
which is pretty much what i meant with:
“Always keep in mind that code generation is usually a solution to a problem that wasn’t solved properly in the first place.”
September 24th, 2008 at 4:41 pm
I still don’t see it.
Could you maybe cherry pick a particularly bad code example (some from each tier) to better illustrate your point?
September 24th, 2008 at 5:13 pm
not really, i can’t post code from work here, and it’d be way too much work to come up with a similar (but not real) example
But hey, if code generation works for you, knock yourself out
June 3rd, 2009 at 3:02 pm
Oh yes, code generation is a real flame war
Personally, i am also for not using code-generation if possible. We have a framework which has lots of functionality, and we are keep adding it.
With code generation i would not image how would i be able to change something.
Technically looking at the problem, code generation solves a problem of a loop. That is, based on a standard set, create lots of similar code. So why not just abstract that similar code into some class, and then just make a simple for loop and iterate? That would be the result of code generation.
Of course, one thing i use code generation is to generate an initial draft of my WinForm application window. After i have initial design, i still wire up all code manually, all events, and add custom controls dynamically either at startup time or at run-time.