The Inquisitive Coder - Davy Brion’s Blog

Thinking outside of the typical .NET box

Archive for the 'Software Development' Category


Software Development Books: Investing In Yourself

Posted by Davy Brion on 24th August 2008

I currently have 20 books in my list of recommended books. I’ve read all of those books in the past 2 years, which means i’m averaging about 10 books a year. That’s probably a bit much, and i have to admit i’ve been reading a bit less lately than i used to. But i’ve noticed that a lot of developers that i know hardly ever read books about software development. Which is too bad, because those books really are a cheap way of investing in yourself and your career.

Well, perhaps cheap isn’t the best word to use. After all, some of these books are somewhat expensive, and buying a lot of them certainly adds up. But if you pick the right books, they are usually more than worth the money they cost. Besides, you shouldn’t just look at the price of the book. The most important thing to keep in mind is the knowledge and insight you can get from them. That typically depends on the type of books you buy though. I usually avoid books that are library/framework/language-specific, unless i am very interested in a specific topic.

The books that usually provide the biggest value are the ones that teach you things that you can use in any development environment. It can be about development methodologies, certain concepts or practices, patterns (not just design patterns but architectural patterns, coding patterns, testing patterns, …), as long as the knowledge is reusable. These kinds of books allow you to learn from the experiences of some of the best people in this industry. And the best thing about it is that you can learn a lot from these books in a few days (or weeks), even though it probably took the authors a couple of years to acquire that knowledge and experience. Isn’t that one of the coolest forms of reuse you can think of?

Now obviously, reading a couple of books won’t instantly put you on the same level as the authors, but at the very least it allows you to significantly increase your skills and insight with relatively little effort. And when you start applying that knowledge, you can start reaping the benefits of it pretty soon. It could definitely make your job easier. It most likely will increase the quality of your output at work. It almost certainly increases your value as a software developer.

So do yourself (and everyone else) a favor, and read a good book on software development once in a while. And if you need some help picking out the good books, just look here :)

Posted in Books, Software Development | 10 Comments »

.NET Memory Management

Posted by Davy Brion on 16th August 2008

Introduction

Garbage Collection sure is great, isn’t it? We don’t have to keep track of all the memory we’ve allocated and we don’t need to release that memory when it’s no longer needed. Because that is after all what the Garbage Collector does for us, without us having to worry about it. This is actually a widespread misconception among many .NET developers. It’s true that Garbage Collection makes memory management a lot easier, but we simply can’t rely on it all the time. There are most certainly some things you must always keep in mind when it comes to memory management in .NET.

Different Kinds Of Resources, Different Consequences

In most pieces of code, you use variables. A lot of these variables are references to objects. When your variables go out of scope, they no longer exist. When those variables are references to objects, only your reference is gone and the actual object that was referred to is still in memory somewhere. The Garbage Collector (GC) makes sure that orphaned objects (objects that no longer have a usable reference to them) are removed from memory. This is an automatic process within the .NET runtime. Essentially, the GC periodically performs some checks on objects and if it comes to the conclusion that certain objects are no longer needed, it will remove them from memory. This is a gross oversimplification of how it really works, but that is what it comes down to and that is also how most people think about the GC.

The GC in .NET is very smart and will almost always do a better job of memory management than you will. But there is one huge downside to it: the GC runs periodically and you have no deterministic way of knowing when it will run. Sure, you can instruct the GC to perform a collection but in most cases that will actually have a negative impact on the memory management of your application. I won’t go into the details here, but i will provide a follow-up post on this later on. For now, keep in mind that forcing a garbage collection to occur is really something you should avoid.

So is it really a problem that you don’t know for certain when the GC will run? It depends on which kind of objects that need to be removed from memory. There are basically 2 kinds: managed and unmanaged resources. Managed resources are typical .NET types. Unmanaged resources (also referred to as Native Resources) are things that fall outside of the scope of the .NET managed environment. These are usually things that are available within the Operating System, or that are available through lower-level development API’s (such as the Win32 API for instance). Unmanaged resources can’t be cleaned up automatically by the .NET runtime, so if you use them directly, you are responsible for cleaning up after you’ve used them. Luckily, a lot of managed types are available that take care of the dirty details for you. For instance, if you need to use a file in .NET, you’ll typically use a FileStream instance or something else that easily makes the content of the file available to you, or that easily allows you to write content to a file. The FileStream is a managed type, but it uses unmanaged resources to implement the functionality it offers.

Now think about this: you are using a managed type, so you shouldn’t need to perform any cleanup, right? However, if that managed type uses unmanaged resources then they still need to be cleaned up. And those unmanaged resources should be cleaned up as soon as possible because they can be quite expensive. These types usually offer a way to clean up the unmanaged resources they use in a deterministic manner. They usually implement the IDisposable interface, which exposes a Dispose method. When you call the Dispose method, the unmanaged resources are cleaned up right then and there and then there’s no need to wait for the garbage collector, which again, could be quite expensive when you (either directly or indirectly) have a bunch of unmanaged resources in memory that are waiting to be cleaned up.

Disposable Managed Resources

I call types that implement the IDisposable interface Disposable Managed Resources. They are indeed managed types and thus they are guaranteed to be cleaned up when the GC comes around and notices that instances of these types are no longer needed. However, if you merely trust on the GC to clean up instances of these types you are taking quite a risk. Any expensive resource it may hold might be cleaned up a lot later than it could have been. Which can be very inefficient, and thus, quite costly.

If a type implements the IDisposable interface, it usually has a good reason for doing so (there are of course exceptions to the rule). It is essentially a way of telling consumers of the type that it offers a deterministic way of cleaning up the resources it consumes and i believe you should take advantage of that. If you don’t, you may end up with inefficient memory management, and that’s when people start complaining that Garbage Collection is ‘evil’ or that it ’sucks’. In most cases, people who feel that way simply don’t know how to use it properly. The IDisposable interface and the Dispose Pattern (which we’ll get to in a minute) allow you to avoid most of the issues that are generally attributed to the Garbage Collection in general.

So how do we deal with this? First of all, if a type uses unmanaged resources, it should implement the IDisposable interface using the Dispose Pattern. Secondly, if any of your types use other types that implement IDisposable, it is your responsibility to make sure these types are properly disposed of. You either dispose them when you no longer need them, or you must implement IDisposable and the Dispose Pattern yourself to make sure the Disposable Managed Resources you depend on are indeed properly disposed of. If you need to implement IDisposable, your type effectively becomes a Disposable Managed Resource itself. A lot of people think that this isn’t necessary, but by not doing so they are breaking the contract that the IDisposable interface implies. If a type implements IDisposable then either that type or any other type it may use probably uses unmanaged resources somewhere and you want to see these get cleaned up as soon as possible. Because of that, i believe it’s best to implement IDisposable yourself if you’re holding any reference to a type that also implements it.

The Dispose Pattern

Implementing the IDisposable interface can be as easy as merely providing a public Dispose method where you perform your cleanup. That’s really not a good way of doing it though, as it could lead to a bunch of other problems which might actually make the situation worse. The best way to implement the IDisposable interface is to implement the Dispose Pattern, which looks like this:

    public class MyDisposableClass : IDisposable

    {

        private bool disposed = false;

 

        public void Dispose()

        {

            Dispose(true);

            // prevent this object from being placed in the finalization queue if some

            // derived class provides a finalizer

            GC.SuppressFinalize(this);

        }

 

        protected virtual void Dispose(bool disposing)

        {

            if (!disposed)

            {

                if (disposing)

                {

                    // dispose all Disposable Managed Resources here

                }

 

                // dispose all Unmanaged Resources here

 

                // set disposed to true to prevent the code above from being

                // executed a second time

                disposed = true;

            }

        }

    }

If you don’t need a finalizer it’s best not to provide one (we’ll discuss finalizers later on). If you do need one, you would implement it like this:

        public ~MyDisposableClass()

        {

            Dispose(false);

        }

So what’s so good about the Dispose pattern? Well, we make a clear distinction between what should happen when the object is disposed through the Dispose method, or when it’s being cleaned up by the finalizer. When the clean up occurs through a call to Dispose, it calls the protected virtual Dispose method with the disposing parameter set to true. When this parameter is true, you should call the Dispose method of each Disposable Managed Resource you’re holding a reference to. Outside of the if-statement, you should clean up each unmanaged resource you may be holding.

If your class (or a derived class) implements a finalizer method, it should call the Dispose method with the disposing parameter set to false. The reason for this is that the order in which objects are finalized is not specified. If you’re not careful, you could accidentally call the Dispose method of a Disposable Managed Resource which may have already been finalized as well. This would cause an exception and a finalizer method should never ever throw an exception because that could keep other objects from being finalized.

If you can, try to put this pattern into a reusable base class and make your Disposable Managed Resource holding types inherit from this. An example of this approach can be found here.

Using Disposable Managed Resources

In a lot of cases, you will use a Disposable Managed Resource within the scope of one method. In that case, you certainly don’t need to implement IDisposable. There are two things to keep in mind though. If you receive the Disposable Managed Resource as a method parameter, you are not responsible for disposing it! Somebody else created it, let them take care of the disposal. Things will most likely go wrong if you dispose objects that have been giving to you by somebody else. So in this case, you would just use the object and not worry about it any further:

        public void DoSomethingInteresting(MyDisposableClass disposableResource)

        {

            // … some code could go here

            disposableResource.DoWhateverItIsThatMakesYouSoSpecial();

            // … some code could go here

 

            // we reach the end of the method and we haven’t Disposed the

            // Disposable Managed Resource because it wasn’t our responsibility

        }

However, if you create the object yourself, then you are responsible for getting rid of it properly. In that case, use a using block to make sure the object is disposed of even in case of unhandled exceptions:

        public void DoSomethingInteresting()

        {

            // … some code could go here

            using (var myDisposableResource = new MyDisposableClass())

            {

                myDisposableResource.DoWhateverItIsThatMakesYouSoSpecial();

            }

            // … some code could go here

 

            // we reach the end of the method and we properly Disposed the

            // Disposable Managed Resource because it was our responsibility

        }

In case you don’t know, the using-block guarantees that the Dispose method will be called when we leave the scope of the block. It can get pretty tricky sometimes if you’re passing the Disposable Managed Resources to other objects that will hold a reference to them which will remain in use after you’ve left the using block. In this scenario, it’s possible that the object you’ve passed the Disposable Managed Resource to will try to invoke methods on the disposed instance. Also a situation you most certainly want to avoid because it will either cause unexpected behavior or even unhandled exceptions. The unhandled exceptions are easy to figure out, but the unexpected behavior might be difficult to debug. The key in this scenario, is to figure out a way to not call Dispose on the resource until the other object is done with it. Sometimes it might be enough to put your using block on a slightly higher level, other times you’ll have to keep a reference to the resource yourself, which effectively means you’ve now become an owner of the Disposable Managed Resource.

Owning Disposable Managed Resources

If you need to hold a reference to a Disposable Managed Resource, you are either the owner of the object or at least someone who owns a stake in the lifetime of the object. If you are the owner, and you do not pass the Disposable Managed Resource to any other object then the solution is easy: implement IDisposable. If you do pass the Disposable Managed Resource to other objects, then it can get tricky. Will they only use it to perform some action or will they hold a reference to it? If they hold a reference to it, it means that they too have a stake in the lifetime of the resource. Depending on the implementation of the other types, they may or may not call the Dispose method of the instance you created. Which could cause unexpected behavior or exceptions in your code. Make sure you are prepared for that if you’re passing these objects around. But you should definitely dispose of them in your own Dispose method.

If you didn’t create the Disposable Managed Resource, but you do need to hold a reference to it, then you’ve got some questions to answer. Does the type which provided you with the instance expect you to be solely responsible for the lifetime of the object? In case of factory classes or factory methods, the answer is usually ‘yes’. If you’ve been given the instance by something that will probably make use of that same instance, i think it’s better not to dispose it. Some people will probably disagree, but i don’t think it’s my responsibility to dispose objects that i didn’t create, unless the instance was given to me by a factory class or a factory method. I would expect that the real owner of the resource would dispose of it. Still, it’s a difficult call to make.

Finalizers

Finally, i’d like to add a few words about Finalizers in .NET. A finalizer is similar to a destructor in C++, in that it is the code that is run when your object is being removed from memory. The big difference is that you never know for sure when it will be run, which is why the IDisposable interface and the Dispose pattern was created. There are a lot of misconceptions going around about finalizers, so keep the following comments in mind.

First of all, finalizers only make sense when you have direct references to unmanaged resources. If you have unmanaged resources, implement IDisposable and provide a finalizer that calls the Dispose overload with the disposing parameter set to false. This should really be the only code inside your finalizer method.

Finalizers also come with a performance penalty. When finalizable objects are created, pointers to these objects are added to the finalization queue of the GC. These finalizable objects are collected less frequently because the GC performs finalizations through its finalization queue. This finalization process is not executed as frequently as regular garbage collections because it’s quite costly. So not only are finalizable objects cleaned up less frequently, it takes more processing power to do so. So if you have a lot of finalizable instances, your code’s performance can easily be impacted because of this.

Conclusion

I hope i made it clear that there is a lot more to automatic memory management in .NET than simply relying on the GC to take care of all of the details for you. There is a lot more stuff that i didn’t cover in this post (it’s long enough already) but most .NET developers will probably get by pretty well with the advice given in this post. It’s not an easy subject to explain so i hope everything was clear :)

Posted in Memory Management, Patterns, Performance, Software Development | 3 Comments »

The Wannabe Developer’s Manifesto

Posted by Davy Brion on 10th August 2008

UPDATE: don’t take this post too seriously… i am not targeting any specific individuals or groups of people or anything like that, it’s just a commentary on a ‘career approach’ that i’ve unfortunately seen more than a few people get away with in past experiences.

I’m sure most of us have been confronted with a ‘Wannabe Developer’ at least once, probably a lot more even. The Wannabe Developers are such a large group that i think they should have their own manifesto.

I dislike Wannabe Developers as much as anyone, but i’m in a generous mood today so i’ve taken the liberty to write that manifesto. I can only hope they find it useful.


The Wannabe Developer Manifesto

We are continuously searching for new and improved ways to increase our visibility within our environment, for that is our ultimate goal. We will strive to trick as many people as possible into considering us as experts in our field. In order to achieve that, everything (and we do mean everything) goes. We will lie, cheat and deceive if that is what it takes.

We will never be around once the code goes into maintenance mode, so we do not waste time learning how to properly develop software. If anything, any maintenance problems that occur once we’ve left will actually improve our standing with the client because everything went great while we were there. Surely, whatever problems that may occur must be related to the inferior capabilities of the people who are currently working on the code.

We will actively engage in Buzzword Driven Development. We have discovered that using many popular buzzwords to convince clients of our capabilities is an extremely effective way into duping managers to think the world of us. This allows us to charge extremely high rates, which leads to bigger cars which in turn increases our standing and visibility among our peers.

When confronted with Real Developers, we will try to fool them by using as many software development pattern names that we can think of. If necessary, we will invent pattern names and then look at the other participants in the discussion in disbelief, as if they should be ashamed because they don’t know what we’re talking about.

We will take advantage of every chance we get to discredit the Real Developers, in order to convince management to reduce their responsibilities, and assign them to us.

And last but not least, we will accumulate as many software development certifications as possible. Real Developers know that certifications are meaningless, but managers and clients still believe in them and are willing to pay extra for people who have them.

There’s a lot of money to be made in this field, and it is our responsibility to get as much of it as possible.

Cheers,
The Wannabe Developers

Posted in Software Development | 10 Comments »

ALT.NET: what does it mean?

Posted by Davy Brion on 8th August 2008

A lot of people think ALT.NET is about using NHibernate, Castle Windsor, StructureMap, Resharper, Rhino Mocks, MonoRail, or whatever tool or library you can think of. It’s really not about that.

From altdotnet.org:

We are a self-organizing, ad-hoc community of developers bound by a desire to improve ourselves, challenge assumptions, and help each other pursue excellence in the practice of software development.

And that is really the essence of what ALT.NET is about. It is about continuous improvement, and sharing knowledge. Nothing more, nothing less. Whatever concept or tool that allows us to improve either ourselves or the way we work, should be used. No matter where the concept came from, or who created the tool.

We are often thought of as being anti-Microsoft. And although some of us probably are, that’s not what it’s about. We want quality. We want to work with as little friction as possible. I am pretty sceptical (to say the least) whenever Microsoft promises us the latest and greatest in whatever new product they’re working on, because i’ve felt the pain of their latest and greatest far too often to blindly believe whatever claims they make. But if they release something that allows me to do what i want to do without restricting my development habits (and i think ASP.NET MVC is looking pretty good on that front) then i will use it. Gladly even. Because i really don’t care where a tool or a library comes from, as long as it’s good. I definitely have my preferred set of libraries/tools but i’ll drop them in a heartbeat when something better comes along.

So when is a developer ‘ALT.NET’? According to altnetpedia.com, you are ALT.NET if you are:

  • The type of developer who uses what works while keeping an eye out for a better way.
  • You reach outside the mainstream to adopt the best of any community: Open Source, Agile, Java, Ruby, etc.
  • You’re not content with the status quo. Things can always be better expressed, more elegant and simple, more mutable, higher quality, etc.
  • You know tools are great, but they only take you so far. It’s the principles and knowledge that really matter. The best tools are those that embed the knowledge and encourage the principles

Now tell me: How on earth did it ever get this far in our industry for this mindset to be considered alternative?

Posted in ALT.NET, Software Development | 4 Comments »

Can’t We Just Stop Living In The Past?

Posted by Davy Brion on 5th August 2008

So we were about to start working on a new project at work, and everything was looking good. We were planning on using NHibernate for our data access, but today the client asked us to implement a “standard” data access layer with stored procedures.

Much like a Vietnam veteran who’s suddenly reminded of his traumatic experiences in the war, i too was speechless. Not only that, i was shocked. Chills going down my spine. Didn’t all those anti-war protests teach us anything? Why must we revert to all this useless violence? How many innocent developers must suffer needlessly due to these delusions? Why are the wrong people once again making the decisions?

I seriously don’t get it… i’ve been trying to comprehend this for a few hours now, and i just can’t. Now sure, i could just “let it go”, do my job and not worry about it. Then again, if i were someone like that, i probably wouldn’t have a technical blog and you probably wouldn’t be reading my posts anyway. See, i’m pretty passionate about software development, more specifically, about trying to do it in a good way. And when i’m subjected to ignorant constraints like these, i shiver. I tremble. I get pissed. I vent.

Still, in this case, there’s really nothing else we can do for various reasons i won’t get into here. A “standard” data access layer with stored procedures is what they want, so that is what they will get. It’s their money, after all. But still, what a stunning display of ignorance and shortsightedness. It’s 2008. This myth that stored procedures are the most secure and best performing way of designing your data access layer really needs to be put to rest. I sincerely doubt that any of the much-touted benefits of going with a stored-procedure data layer is actually still relevant in this day and age. Not to mention the fact that it is simply a waste of money. There are much more cost-effective ways of developing software these days and they really don’t necessarily mean you’ll end up with inefficient data access. Perhaps some day, we will no longer be confronted with delusional constraints that are only based on sheer ignorance. I for one, can’t wait. This might actually be one of the few good things about Microsoft’s upcoming Entity Framework. Perhaps even the IT dinosaurs will start to realize that maybe ORM isn’t all that evil or difficult.

Posted in Rants, Software Development | 19 Comments »

What do you like about coding?

Posted by Davy Brion on 2nd August 2008

I not only like to code, i simply love it. Here’s my list of reasons why:

  • It’s challenging.

    It can be challenging in many ways, both positive and negative. My favorite kinds of challenges are trying to understand a complex problem and then trying to find an elegant solution to it, and trying to improve performance bottlenecks when it’s necessary to do so. These are definitely my favorite ways of spending the workday :)

  • It enables me to be creative.

    Creativity is cool and i’ve always wanted to be creative. Unfortunately i don’t have too many creative outlets. I always wanted to be able to draw or paint, or write, or create music, but all of my attempts proved that i sucked. Badly, i might add. When i started coding i noticed i could get creative with code. That was definitely one of the biggest reasons why i started spending more and more time on writing code.

  • It stimulates you to keep learning and improving.

    I’m always looking for ways to improve and interesting new things to learn. No matter how good you think a piece of code is, or a certain approach, or whatever, there’s always something that can still be improved. Every time i look at code that i wrote a few months before that, i often think “ugh, i should’ve done that differently”. Which is a good thing because it means you’ve improved in those months. If i don’t see any room for improvement, i’m disappointed because it means i probably haven’t improved that much since writing that code.

  • It can give you a view into another world.

    If you’re working on applications with an interesting problem domain, you can get a very interesting view into a world that you don’t really know anything about. It allows you to pick up some interesting knowledge about a domain of which you otherwise wouldn’t have known a lot (or anything) about.

  • It can make your life (or others) easier.

    Who doesn’t want to see repetitive tasks automated? Everything you can automate should (in theory) make your life a bit easier and everything that makes your life easier is good :)

  • Creating value can be very fulfilling.

    When you’re coding, you’re actually creating something that has some kind of value. If you’re writing application code, you’re creating value for your users. Each feature you implement will mean something to someone. And if you do a good job, those users might like or even love that feature. I don’t know about you, but it always makes me feel good when people like to use something i wrote. Same thing goes for writing code for other developers. If you do a good job, they will enjoy using the code you wrote because it should make their jobs easier. Again, that is something that makes me feel good about the code i wrote :)

  • It’s an easy way to make a living.

    Coding is not always easy, but let’s be honest here… it’s a pretty easy way to make a comfortable living compared to what a lot of other people have to do for money. For me, it means making a good living by doing something i actually enjoy most of the time. It also means i can completely minimize the amount of physical labor that i have to do, which is definitely a good thing. For everyone involved actually :)

Are there any reasons why you like coding that i haven’t listed? Go ahead and share your reasons :)

Posted in Software Development | 18 Comments »

Batching++

Posted by Davy Brion on 30th July 2008

The usual example:

                dispatcher.Add(new GetProductCategoriesRequest(), new GetSuppliersRequest());

                View.ProductCategories = dispatcher.Get<GetProductCategoriesResponse>().ProductCategories;

                View.Suppliers = dispatcher.Get<GetSuppliersResponse>().Suppliers;

                View.DataBind();

Thanks to my request/response service layer, those are two service requests that will be performed in one service call. On the server side, these two requests are dealt with separately. But they merely retrieve data from the database. Wouldn’t it be cool if those 2 queries were performed in one database roundtrip instead of two? Wouldn’t it be cool if we could batch the queries that will be performed by read-only service requests into a single database roundtrip? It sure as hell would be. Implementing this was one of those irresistible challenges you just can’t say no to, so from now on we can actually do this. I’m not gonna get into the actual implementation of how I got it working (check the code in the library for that if you’re interested), but i will show you how you can do this in your application.

First of all, if you have requests that merely return data, inherit from ReadOnlyRequest instead of the usual Request type:

    [Serializable]

    public class GetProductCategoriesRequest : ReadOnlyRequest {}

 

    [Serializable]

    public class GetSuppliersRequest : ReadOnlyRequest {}

The handlers for these requests should inherit from the ReadOnlyRequestHandler base class instead of the usual RequestHandler class. Btw, my ReadOnlyRequestHandler class inherits from my UoWRequestHandler class, which requires a IUnitOfWork instance to be passed to the constructor. So we need to put an IUnitOfWork parameter into the constructor along with our other dependencies if we want the IOC container (or the request processor) to pass us a valid IUnitOfWork instance.

    public class GetProductCategoriesHandler : ReadOnlyRequestHandler<GetProductCategoriesRequest>

    {

        private readonly IRepository<ProductCategory> repository;

 

        public GetProductCategoriesHandler(IUnitOfWork unitOfWork, IRepository<ProductCategory> repository)

            : base(unitOfWork)

        {

            this.repository = repository;

        }

 

        public override void AddQueries(IQueryBatcher queryBatcher, GetProductCategoriesRequest request)

        {

            queryBatcher.AddCriteria(“categories”, repository.CriteriaForAll());

        }

 

        public override Response GetResults(IQueryBatcher queryBatcher, GetProductCategoriesRequest request)

        {

            var categories = queryBatcher.GetEnumerableResult<ProductCategory>(“categories”);

 

            return new GetProductCategoriesResponse { ProductCategories = categories.ToDTOs() };

        }

    }

The ReadOnlyRequestHandler class defines two abstract methods: the AddQueries and GetResults method. Both are passed an instance of IQueryBatcher and the actual request object so you can use the request parameters if there are any.

In the AddQueries method, you simply add the queries this request needs to perform to the batcher (with a string key value of course for later retrieval), and in the GetResults method you can get the results (with the key values used in AddQueries) back from the batcher and you can construct your Response object. The GetResults method will be called after each ReadOnlyRequestHandler for the current request batch has added the queries to be performed to the batcher. The queries are then all executed, and each ReadOnlyRequestHandler will have its GetResults method called so each handler can construct the proper response.

And that’s it basically… The calling code doesn’t even change.

Now there are obviously some limitations. If you mix ReadOnlyRequests with other Requests, then each ReadOnlyRequest will be handled as if it were a regular Request and they will each get their very own IQueryBatcher instance. I don’t think there’s any strategy to automagically determine which ReadOnlyRequests can be batched together while still correctly executing the other Requests since those requests might influence the results of ReadOnlyRequests that are present further in the batch.

Anyways, you can find an updated version of the library here. Check it out, it’s some pretty slick shit, if i do say so myself ;)

Note: i only added this stuff today, so the previous build (080729) doesn’t have this yet, in case you only downloaded that today, and the stats show that some of you actually did download it ;)

Posted in NHibernate, Performance, Software Development, WCF | 1 Comment »

Small update to the library

Posted by Davy Brion on 29th July 2008

Added a couple more things to my library… the first being my base repository implementation for nhibernate, and also everything you need for my NHibernate session management tricks.

So yea, NHibernate is now a dependency, allthough you obviously only need to include it if you’re using anything specific to NHibernate.

Posted in Software Development | No Comments »

Onion Architecture?

Posted by Davy Brion on 29th July 2008

Jeffrey Palermo describes the Onion Architecture in this great post. If you haven’t read it yet, read it now. If you don’t like it, you just might be in the wrong business :)

The only thing i don’t like about it is the name… honestly, onion architecture? I once referred to an architecture as being comparable to an onion, in the sense that as i went through the layers and got exposed to the internals, i got more and more tears in my eyes. You don’t want people getting thoughts like that when you’re describing your preferred architecture. We really do need a better name for this :)

Don’t look at me though… i suck at coming up with good names :)

Posted in Software Development | 9 Comments »

The Request/Response Service Layer

Posted by Davy Brion on 27th July 2008

Introduction

When you’re trying to design a service layer API, there are two things you always need to be very careful of. The first is making sure the API isn’t chatty. A service layer is usually deployed on another physical machine than the client layer, so you want clients to be able to do everything they need to do with a minimum of roundtrips, because those are after all rather expensive. The second thing you want to avoid is the implicit coupling between the client of the service and the service itself. In your quest to avoid the chatty interface, you might start grouping several service operations together in more coarse-grained operations to avoid the chatty communication that might otherwise occur. This can be good from a performance point of view, but it usually introduces a certain implicit coupling between the service and the client, because a specific grouping of operations might be beneficial to one client, but might be completely inappropriate to another type of client of the service.

I really had difficulties coming up with an approach that offered nice coarse-grained service interfaces while at the same time making sure the interfaces weren’t too ‘driven’ by a client’s specific requirements. So i basically wanted a way to keep my service operations as specific as they could be (very fine-grained), while avoiding the pitfall of chatty communication by batching calls to the service layer together whenever it makes sense from the client’s point of view. I eventually ended up with an approach that i’ve already documented here and here. Read those posts first before continuing with this post because the rest of this post builds upon the content of the previous posts.

The idea is to basically consider each service operation as a request which can have a response. For each request that you define, you need to provide a handler which does whatever it needs to do to handle the request and returns a response, or an empty response if no response is needed for a specific request. You then only need one service method which receives incoming requests, delegates them to the proper handlers, and returns each response in the order of the incoming requests. On a side note: this approach probably doesn’t sit well with many SOA people, but then again, i couldn’t care less about SOA. I just want good software no matter what acronym i can use to describe it.

Anyway, i like this approach so much that i wanted to make it available to whoever is interested in it. You can find it in my open source library which you can find here. I won’t go into the details of the implementation, because those have mostly been covered already in my previous posts on this subject. The rest of this post focuses solely on how you can use this approach in your projects.

Defining Requests and Responses

The following two base classes need to be used to define request and response types:

namespace Brion.Library.Common.Messaging

{

    [Serializable]

    public abstract class Request {}

 

    [Serializable]

    public abstract class Response { }

 

}

Suppose you have a service operation that retrieves a list of products based on some parameters the user could provide in the UI. You would define the request like this:

    [Serializable]

    public class GetProductOverviewsRequest : Request

    {

        public string NamePattern { get; set; }

        public int? CategoryId { get; set; }

        public int? SupplierId { get; set; }

    }

And the response would look like this:

    [Serializable]

    public class GetProductOverviewsResponse : Response

    {

        public ProductOverviewDTO[] Products { get; set; }

    }

Keep in mind that these types always need to be marked with the Serializable attribute and that these types need to be in an assembly that you can share between both the client and the service.

Handling Requests

Incoming requests are handled by a class which implements the IRequestProcessor interface:

namespace Brion.Library.Common.Messaging

{

    public interface IRequestProcessor : IDisposable

    {

        Response[] Process(params Request[] requests);

    }

}

For each request that is received, the request processor will create a request handler which must be of the type IRequestHandler<TRequest> where TRequest is the type of the request instance. The request object is then passed along to the handler’s Handle method, which will return a proper Response object.

The simplest way to create a request handler is to inherit from the RequestHandler<TRequest> class, like this:

    public class GetProductOverviewsHandler :

        Brion.Library.ServerSide.Messaging.RequestHandler<GetProductOverviewsRequest>

    {

        public IProductRepository ProductRepository { get; private set; }

 

        public GetProductOverviewsHandler(IProductRepository productRepository)

        {

            ProductRepository = productRepository;

        }

 

        public override Response Handle(GetProductOverviewsRequest request)

        {

            var products = ProductRepository.FindAll(request.NamePattern, request.SupplierId,

                request.CategoryId);

 

            return new GetProductOverviewsResponse { Products = products.ToOverviewDTOs() };

        }

    }

The request processor uses the Castle Windsor Inversion Of Control container to resolve and create the instances of the request handler types. This allows you to use dependency injection for your request handlers. In the example posted above, you can see that the constructor of the handler takes an IProductRepository instance. Because the Windsor container also knows about the IProductRepository component in my application, it can correctly instantiate the GetProductOverviewsHandler instance with a valid IProductRepository instance. Keep in mind that this is optional though. You don’t have to use dependency injection for your request handlers if you don’t want to.

If you do want to use it, your application will have to use the same Windsor container instance as this library does. To make this possible, the library contains the following class:

namespace Brion.Library.Common

{

    public static class IoC

    {

        private static IWindsorContainer container = new WindsorContainer();

 

        /// <summary>

        /// Gets/Sets the current Windsor container. Applications can either use

        /// the reference to the container that this property provides, or they

        /// can set their own reference through the setter.

        /// </summary>

        public static IWindsorContainer Container { get; set; }

    }

}

By default, an empty container is created which you can access (and thus, configure) through the IoC.Container property. Or if you want the library to use your own Windsor container instance you can simply set the instance and then the library will use your container. For the moment, it is not possible to use another container (like StructureMap or Unity) yet, but that possibility might be added if people want it, or if someone submits a patch :)

Obviously, you need to register your request handlers somewhere in your application code, preferably before you start hosting the service layer. You can do that like this:

            Brion.Library.ServerSide.ComponentRegistration

                .RegisterRequestHandlersFrom(Assembly.GetExecutingAssembly());

That basically registers each valid request handler (each type that inherits from the RequestHandler class provided by the library) that is present in the given assembly. You can also very easily provide your own custom base request handler, in case you want to provide some extra stuff. You’d simply need to inherit from the library’s RequestHandler class, add whatever it is you need, and let your request handlers inherit from your new class instead of directly inheriting from the library’s RequestHandler.

Hosting The Service Layer

There are a lot of options for hosting the service layer. Most people will probably host it as a WCF service so we’ll cover that scenario here. The service contract looks like this:

namespace Brion.Library.Common.WCF

{

    [ServiceContract]

    public interface IWcfRequestProcessor

    {

        [OperationContract]

        [ServiceKnownType("GetKnownTypes", typeof(KnownTypeProvider))]

        Response[] Process(params Request[] requests);

    }

}

This is practically identical to the IRequestProcessor interface shown earlier in the article. The difference is that IRequestProcessor uses no WCF attributes. The actual implementation of IRequestProcessor is completely decoupled from WCF as well so you can use this approach without having to use WCF if you want to. The implementation of the IWcfRequestProcessor simply forwards each call to the real request processor.

Notice the usage of the KnownTypeProvider class. This makes sure that each of your Request or Response derived types will be properly recognized as a KnownType. All you have to do is register them, like this:

            KnownTypeProvider.RegisterDerivedTypesOf<Request>(sharedAssembly);

            KnownTypeProvider.RegisterDerivedTypesOf<Response>(sharedAssembly);

This registers each Request or Response derived type from the sharedAssembly reference (which is a reference to the shared assembly containing the request and response types) with the KnownTypeProvider. You need to do this before you start hosting the service. It’s best to combine this task with the registration of your request handlers.

The actual hosting of the service is typical WCF and is entirely up to you. Here’s a simple example of self-hosting the service in a console app:

    <services>

      <service name=Brion.Library.ServerSide.WCF.WcfRequestProcessor behaviorConfiguration=MyBehavior>

        <endpoint contract=Brion.Library.Common.WCF.IWcfRequestProcessor binding=netNamedPipeBinding

              bindingConfiguration=MyNamedPipeBinding address=net.pipe://localhost/RequestProcessor/>

      </service>

    </services>

            using (var host = new ServiceHost(typeof(WcfRequestProcessor)))

            {

                host.Open();

 

                Console.WriteLine(“press ENTER to quit”);

                Console.ReadLine();

            }

You’ll most likely prefer an alternative way of hosting the service, but again, that is entirely up to you and is typical WCF stuff. All you need to know is that the service contract is the Brion.Library.Common.WCF.IWcfRequestProcessor interface, and the actual implementation of the service is the Brion.Library.ServerSide.WCF.WcfRequestProcessor class.

Communicating With The Service Layer

In your client layer, you can choose between using a direct proxy type for the IWcfRequestProcessor service, or you can use the Dispatcher class. The Dispatcher class is somewhat of a wrapper around the direct proxy to make it easier to use and to get a nicer syntax. The Dispatcher class implements the IDispatcher interface:

namespace Brion.Library.ClientSide.Messaging

{

    public interface IDispatcher : IDisposable

    {

        IEnumerable<Request> Requests { get; }

        IEnumerable<Response> Responses { get; }

 

        void Add(Request request);

        void Add(params Request[] requestsToAdd);

        void Add(string key, Request request);

        TResponse Get<TResponse>() where TResponse : Response;

        TResponse Get<TResponse>(string key) where TResponse : Response;

        TResponse Get<TResponse>(Request request) where TResponse : Response;

        void Clear();

    }

}

There are 3 implementations of this interface. The first is the Dispatcher class itself, which has a dependency on an IRequestProcessor instance. The second is the WcfDispatcher class which inherits from the Dispatcher class and will supply its base class with a RequestProcessorProxy instance to satisfy the IRequestProcessor dependency. The third implementation is the DispatcherStub class, which is only useful for your tests. If you want to use dependency injection through the container, you can register the correct components at client start-up time like this:

            Brion.Library.ClientSide.ComponentRegistration.RegisterDispatcherForWcfUsage();

The container will then have the correct configuration to provide you with IDispatcher instances that will correctly use the RequestProcessorProxy underneath. If you don’t want to use the container client-side, you can simply instantiate instances of the WcfDispatcher class.

So, how do you use the dispatcher? It’s pretty easy really. Suppose you want to dispatch two requests to the service, you could do so like this:

                dispatcher.Add(new GetProductCategoriesRequest(), new GetSuppliersRequest());

                View.ProductCategories = dispatcher.Get<GetProductCategoriesResponse>().ProductCategories;

                View.Suppliers = dispatcher.Get<GetSuppliersResponse>().Suppliers;

The requests won’t be dispatched to the service until you actually call one of the Get method overloads for the first time. So you can add as much requests as you like, they won’t be dispatched until you try to retrieve a response. And when those requests are dispatched, it is done in only one roundtrip.

If you just want to dispatch a single request, you could do so like this:

            var request = new GetProductOverviewsRequest

                { NamePattern = name, CategoryId = productCategoryId, SupplierId = supplierId };

            var response = dispatcher.Get<GetProductOverviewsResponse>(request);

            View.Products = response.Products;

Keep in mind that you do need to define the service endpoint in your client-side application’s configuration file. You also need to register the KnownTypes client-side before you start using the Dispatcher or the service proxy in the manner discussed earlier.

Stubbing The Service Layer During Testing

If you write tests for your client-side code, you’ll be glad to hear that i’ve included a DispatcherStub class which makes it easy to prepare responses to return and to inspect requests that were sent from the code you’re testing. The approach i outlined in this post doesn’t really lend itself to easy mocking, so the DispatcherStub class makes it all much easier.

First of all, make sure your client code always uses references of the IDispatcher type instead of directly using the Dispatcher or WcfDispatcher types. Then in your tests, inject DispatcherStub instances instead of real Dispatchers. The DispatcherStub class provides a few extra methods which you can use in your tests:

        [Test]

        public void RetrievesCategoriesAndSuppliersOnLoad()

        {

            var categoriesToReturn = new ProductCategoryDTO[0];

            var suppliersToReturn = new SupplierDTO[0];

 

            dispatcher.SetResponsesToReturn(

                new GetProductCategoriesResponse { ProductCategories = categoriesToReturn },

                new GetSuppliersResponse { Suppliers = suppliersToReturn });

 

            view.Expect(v => v.ProductCategories = categoriesToReturn);

            view.Expect(v => v.Suppliers = suppliersToReturn);

            view.Expect(v => v.DataBind());

 

            mocks.ReplayAll();

            CreateController().Load();

 

            view.VerifyAllExpectations();

            Assert.IsNotNull(dispatcher.GetRequest<GetProductCategoriesRequest>());

            Assert.IsNotNull(dispatcher.GetRequest<GetSuppliersRequest>());

        }

The SetResponsesToReturn and GetRequest methods make it much easier to stub the service layer than traditional mocking would.

Something you’ll often want to test is that requests have been sent with the correct parameters:

            var request = dispatcher.GetRequest<GetProductOverviewsRequest>();

            Assert.AreEqual(productPattern, request.NamePattern);

            Assert.AreEqual(categoryId, request.CategoryId);

            Assert.AreEqual(supplierId, request.SupplierId);

Time to wrap it up

Again, i really like this approach. Unfortunately i haven’t had the chance to use this in a real project yet, but based on my experiments i’m already very happy with it and am looking forward to use this in a real project. If you’d like to use this approach, download the library here, play around with it, try it out, try to break it, and let me know what needs to be fixed/changed/added. Or send patches ;)

Posted in Patterns, Software Development, WCF | 3 Comments »