Archive for July, 2008

Some stuff you may have missed

5 commentsWritten on July 31st, 2008 by
Categories: About The Blog

Over the past month, the stats of my blog have basically quadrupled and the (estimated) subscriber count went from 40 to 200. I'm obviously very happy about that. Now, 200 subscribers is peanuts compared to what many other bloggers have, but considering the fact that i have no 'claim to fame' whatsoever outside of my posts, and that i don't work for a large well known company, i'd say it's pretty good so far.

But anyways, for those of you who've only joined us recently, i've listed a couple of my favorite older posts on the right sidebar of the blog. For those of you who're reading this through an RSS reader, here's the list:

I think those posts are pretty good, so maybe some of you will like them as well :)

Anyways, i'm sure i'll have a ton of interesting stuff to post in the coming months as well. For the first time in my career, i'm in a position where i'm actually doing interesting stuff all day long, whether it's interesting coding, thinking about interesting problems with some smart people or trying to provide some technical guidance/coaching to younger developers (don't think i'm an old fart though, i'm only 27). The only difficulty i'm having so far is going home on time :) . I expect this situation to lead to a lot of interesting posts in the future so stay tuned...

Batching++

2 commentsWritten on July 30th, 2008 by
Categories: NHibernate, Performance, Software Development, WCF

The usual example:

                dispatcher.Add(new GetProductCategoriesRequest(), new GetSuppliersRequest());

                View.ProductCategories = dispatcher.Get<GetProductCategoriesResponse>().ProductCategories;

                View.Suppliers = dispatcher.Get<GetSuppliersResponse>().Suppliers;

                View.DataBind();

Thanks to my request/response service layer, those are two service requests that will be performed in one service call. On the server side, these two requests are dealt with separately. But they merely retrieve data from the database. Wouldn't it be cool if those 2 queries were performed in one database roundtrip instead of two? Wouldn't it be cool if we could batch the queries that will be performed by read-only service requests into a single database roundtrip? It sure as hell would be. Implementing this was one of those irresistible challenges you just can't say no to, so from now on we can actually do this. I'm not gonna get into the actual implementation of how I got it working (check the code in the library for that if you're interested), but i will show you how you can do this in your application.

First of all, if you have requests that merely return data, inherit from ReadOnlyRequest instead of the usual Request type:

    [Serializable]

    public class GetProductCategoriesRequest : ReadOnlyRequest {}

 

    [Serializable]

    public class GetSuppliersRequest : ReadOnlyRequest {}

The handlers for these requests should inherit from the ReadOnlyRequestHandler base class instead of the usual RequestHandler class. Btw, my ReadOnlyRequestHandler class inherits from my UoWRequestHandler class, which requires a IUnitOfWork instance to be passed to the constructor. So we need to put an IUnitOfWork parameter into the constructor along with our other dependencies if we want the IOC container (or the request processor) to pass us a valid IUnitOfWork instance.

    public class GetProductCategoriesHandler : ReadOnlyRequestHandler<GetProductCategoriesRequest>

    {

        private readonly IRepository<ProductCategory> repository;

 

        public GetProductCategoriesHandler(IUnitOfWork unitOfWork, IRepository<ProductCategory> repository)

            : base(unitOfWork)

        {

            this.repository = repository;

        }

 

        public override void AddQueries(IQueryBatcher queryBatcher, GetProductCategoriesRequest request)

        {

            queryBatcher.AddCriteria("categories", repository.CriteriaForAll());

        }

 

        public override Response GetResults(IQueryBatcher queryBatcher, GetProductCategoriesRequest request)

        {

            var categories = queryBatcher.GetEnumerableResult<ProductCategory>("categories");

 

            return new GetProductCategoriesResponse { ProductCategories = categories.ToDTOs() };

        }

    }

The ReadOnlyRequestHandler class defines two abstract methods: the AddQueries and GetResults method. Both are passed an instance of IQueryBatcher and the actual request object so you can use the request parameters if there are any.

In the AddQueries method, you simply add the queries this request needs to perform to the batcher (with a string key value of course for later retrieval), and in the GetResults method you can get the results (with the key values used in AddQueries) back from the batcher and you can construct your Response object. The GetResults method will be called after each ReadOnlyRequestHandler for the current request batch has added the queries to be performed to the batcher. The queries are then all executed, and each ReadOnlyRequestHandler will have its GetResults method called so each handler can construct the proper response.

And that's it basically... The calling code doesn't even change.

Now there are obviously some limitations. If you mix ReadOnlyRequests with other Requests, then each ReadOnlyRequest will be handled as if it were a regular Request and they will each get their very own IQueryBatcher instance. I don't think there's any strategy to automagically determine which ReadOnlyRequests can be batched together while still correctly executing the other Requests since those requests might influence the results of ReadOnlyRequests that are present further in the batch.

Anyways, you can find an updated version of the library here. Check it out, it's some pretty slick shit, if i do say so myself ;)

Note: i only added this stuff today, so the previous build (080729) doesn't have this yet, in case you only downloaded that today, and the stats show that some of you actually did download it ;)

Small update to the library

No Comments »Written on July 29th, 2008 by
Categories: Software Development

Added a couple more things to my library... the first being my base repository implementation for nhibernate, and also everything you need for my NHibernate session management tricks.

So yea, NHibernate is now a dependency, allthough you obviously only need to include it if you're using anything specific to NHibernate.

Onion Architecture?

13 commentsWritten on July 29th, 2008 by
Categories: Software Development

Jeffrey Palermo describes the Onion Architecture in this great post. If you haven't read it yet, read it now. If you don't like it, you just might be in the wrong business :)

The only thing i don't like about it is the name... honestly, onion architecture? I once referred to an architecture as being comparable to an onion, in the sense that as i went through the layers and got exposed to the internals, i got more and more tears in my eyes. You don't want people getting thoughts like that when you're describing your preferred architecture. We really do need a better name for this :)

Don't look at me though... i suck at coming up with good names :)

The Request/Response Service Layer

9 commentsWritten on July 27th, 2008 by
Categories: Patterns, Software Development, WCF

Update: i also have a complete series of posts on this which you may find interesting.

Introduction

When you're trying to design a service layer API, there are two things you always need to be very careful of. The first is making sure the API isn't chatty. A service layer is usually deployed on another physical machine than the client layer, so you want clients to be able to do everything they need to do with a minimum of roundtrips, because those are after all rather expensive. The second thing you want to avoid is the implicit coupling between the client of the service and the service itself. In your quest to avoid the chatty interface, you might start grouping several service operations together in more coarse-grained operations to avoid the chatty communication that might otherwise occur. This can be good from a performance point of view, but it usually introduces a certain implicit coupling between the service and the client, because a specific grouping of operations might be beneficial to one client, but might be completely inappropriate to another type of client of the service.

I really had difficulties coming up with an approach that offered nice coarse-grained service interfaces while at the same time making sure the interfaces weren't too 'driven' by a client's specific requirements. So i basically wanted a way to keep my service operations as specific as they could be (very fine-grained), while avoiding the pitfall of chatty communication by batching calls to the service layer together whenever it makes sense from the client's point of view. I eventually ended up with an approach that i've already documented here and here. Read those posts first before continuing with this post because the rest of this post builds upon the content of the previous posts.

The idea is to basically consider each service operation as a request which can have a response. For each request that you define, you need to provide a handler which does whatever it needs to do to handle the request and returns a response, or an empty response if no response is needed for a specific request. You then only need one service method which receives incoming requests, delegates them to the proper handlers, and returns each response in the order of the incoming requests. On a side note: this approach probably doesn't sit well with many SOA people, but then again, i couldn't care less about SOA. I just want good software no matter what acronym i can use to describe it.

Anyway, i like this approach so much that i wanted to make it available to whoever is interested in it. You can find it in my open source library which you can find here. I won't go into the details of the implementation, because those have mostly been covered already in my previous posts on this subject. The rest of this post focuses solely on how you can use this approach in your projects.

Defining Requests and Responses

The following two base classes need to be used to define request and response types:

namespace Brion.Library.Common.Messaging

{

    [Serializable]

    public abstract class Request {}

 

    [Serializable]

    public abstract class Response { }

 

}

Suppose you have a service operation that retrieves a list of products based on some parameters the user could provide in the UI. You would define the request like this:

    [Serializable]

    public class GetProductOverviewsRequest : Request

    {

        public string NamePattern { get; set; }

        public int? CategoryId { get; set; }

        public int? SupplierId { get; set; }

    }

And the response would look like this:

    [Serializable]

    public class GetProductOverviewsResponse : Response

    {

        public ProductOverviewDTO[] Products { get; set; }

    }

Keep in mind that these types always need to be marked with the Serializable attribute and that these types need to be in an assembly that you can share between both the client and the service.

Handling Requests

Incoming requests are handled by a class which implements the IRequestProcessor interface:

namespace Brion.Library.Common.Messaging

{

    public interface IRequestProcessor : IDisposable

    {

        Response[] Process(params Request[] requests);

    }

}

For each request that is received, the request processor will create a request handler which must be of the type IRequestHandler<TRequest> where TRequest is the type of the request instance. The request object is then passed along to the handler's Handle method, which will return a proper Response object.

The simplest way to create a request handler is to inherit from the RequestHandler<TRequest> class, like this:

    public class GetProductOverviewsHandler :

        Brion.Library.ServerSide.Messaging.RequestHandler<GetProductOverviewsRequest>

    {

        public IProductRepository ProductRepository { get; private set; }

 

        public GetProductOverviewsHandler(IProductRepository productRepository)

        {

            ProductRepository = productRepository;

        }

 

        public override Response Handle(GetProductOverviewsRequest request)

        {

            var products = ProductRepository.FindAll(request.NamePattern, request.SupplierId,

                request.CategoryId);

 

            return new GetProductOverviewsResponse { Products = products.ToOverviewDTOs() };

        }

    }

The request processor uses the Castle Windsor Inversion Of Control container to resolve and create the instances of the request handler types. This allows you to use dependency injection for your request handlers. In the example posted above, you can see that the constructor of the handler takes an IProductRepository instance. Because the Windsor container also knows about the IProductRepository component in my application, it can correctly instantiate the GetProductOverviewsHandler instance with a valid IProductRepository instance. Keep in mind that this is optional though. You don't have to use dependency injection for your request handlers if you don't want to.

If you do want to use it, your application will have to use the same Windsor container instance as this library does. To make this possible, the library contains the following class:

namespace Brion.Library.Common

{

    public static class IoC

    {

        private static IWindsorContainer container = new WindsorContainer();

 

        /// <summary>

        /// Gets/Sets the current Windsor container. Applications can either use

        /// the reference to the container that this property provides, or they

        /// can set their own reference through the setter.

        /// </summary>

        public static IWindsorContainer Container { get; set; }

    }

}

By default, an empty container is created which you can access (and thus, configure) through the IoC.Container property. Or if you want the library to use your own Windsor container instance you can simply set the instance and then the library will use your container. For the moment, it is not possible to use another container (like StructureMap or Unity) yet, but that possibility might be added if people want it, or if someone submits a patch :)

Obviously, you need to register your request handlers somewhere in your application code, preferably before you start hosting the service layer. You can do that like this:

            Brion.Library.ServerSide.ComponentRegistration

                .RegisterRequestHandlersFrom(Assembly.GetExecutingAssembly());

That basically registers each valid request handler (each type that inherits from the RequestHandler class provided by the library) that is present in the given assembly. You can also very easily provide your own custom base request handler, in case you want to provide some extra stuff. You'd simply need to inherit from the library's RequestHandler class, add whatever it is you need, and let your request handlers inherit from your new class instead of directly inheriting from the library's RequestHandler.

Hosting The Service Layer

There are a lot of options for hosting the service layer. Most people will probably host it as a WCF service so we'll cover that scenario here. The service contract looks like this:

namespace Brion.Library.Common.WCF

{

    [ServiceContract]

    public interface IWcfRequestProcessor

    {

        [OperationContract]

        [ServiceKnownType("GetKnownTypes", typeof(KnownTypeProvider))]

        Response[] Process(params Request[] requests);

    }

}

This is practically identical to the IRequestProcessor interface shown earlier in the article. The difference is that IRequestProcessor uses no WCF attributes. The actual implementation of IRequestProcessor is completely decoupled from WCF as well so you can use this approach without having to use WCF if you want to. The implementation of the IWcfRequestProcessor simply forwards each call to the real request processor.

Notice the usage of the KnownTypeProvider class. This makes sure that each of your Request or Response derived types will be properly recognized as a KnownType. All you have to do is register them, like this:

            KnownTypeProvider.RegisterDerivedTypesOf<Request>(sharedAssembly);

            KnownTypeProvider.RegisterDerivedTypesOf<Response>(sharedAssembly);

This registers each Request or Response derived type from the sharedAssembly reference (which is a reference to the shared assembly containing the request and response types) with the KnownTypeProvider. You need to do this before you start hosting the service. It's best to combine this task with the registration of your request handlers.

The actual hosting of the service is typical WCF and is entirely up to you. Here's a simple example of self-hosting the service in a console app:

    <services>

      <service name="Brion.Library.ServerSide.WCF.WcfRequestProcessor" behaviorConfiguration="MyBehavior">

        <endpoint contract="Brion.Library.Common.WCF.IWcfRequestProcessor" binding="netNamedPipeBinding"

              bindingConfiguration="MyNamedPipeBinding" address="net.pipe://localhost/RequestProcessor"/>

      </service>

    </services>

            using (var host = new ServiceHost(typeof(WcfRequestProcessor)))

            {

                host.Open();

 

                Console.WriteLine("press ENTER to quit");

                Console.ReadLine();

            }

You'll most likely prefer an alternative way of hosting the service, but again, that is entirely up to you and is typical WCF stuff. All you need to know is that the service contract is the Brion.Library.Common.WCF.IWcfRequestProcessor interface, and the actual implementation of the service is the Brion.Library.ServerSide.WCF.WcfRequestProcessor class.

Communicating With The Service Layer

In your client layer, you can choose between using a direct proxy type for the IWcfRequestProcessor service, or you can use the Dispatcher class. The Dispatcher class is somewhat of a wrapper around the direct proxy to make it easier to use and to get a nicer syntax. The Dispatcher class implements the IDispatcher interface:

namespace Brion.Library.ClientSide.Messaging

{

    public interface IDispatcher : IDisposable

    {

        IEnumerable<Request> Requests { get; }

        IEnumerable<Response> Responses { get; }

 

        void Add(Request request);

        void Add(params Request[] requestsToAdd);

        void Add(string key, Request request);

        TResponse Get<TResponse>() where TResponse : Response;

        TResponse Get<TResponse>(string key) where TResponse : Response;

        TResponse Get<TResponse>(Request request) where TResponse : Response;

        void Clear();

    }

}

There are 3 implementations of this interface. The first is the Dispatcher class itself, which has a dependency on an IRequestProcessor instance. The second is the WcfDispatcher class which inherits from the Dispatcher class and will supply its base class with a RequestProcessorProxy instance to satisfy the IRequestProcessor dependency. The third implementation is the DispatcherStub class, which is only useful for your tests. If you want to use dependency injection through the container, you can register the correct components at client start-up time like this:

            Brion.Library.ClientSide.ComponentRegistration.RegisterDispatcherForWcfUsage();

The container will then have the correct configuration to provide you with IDispatcher instances that will correctly use the RequestProcessorProxy underneath. If you don't want to use the container client-side, you can simply instantiate instances of the WcfDispatcher class.

So, how do you use the dispatcher? It's pretty easy really. Suppose you want to dispatch two requests to the service, you could do so like this:

                dispatcher.Add(new GetProductCategoriesRequest(), new GetSuppliersRequest());

                View.ProductCategories = dispatcher.Get<GetProductCategoriesResponse>().ProductCategories;

                View.Suppliers = dispatcher.Get<GetSuppliersResponse>().Suppliers;

The requests won't be dispatched to the service until you actually call one of the Get method overloads for the first time. So you can add as much requests as you like, they won't be dispatched until you try to retrieve a response. And when those requests are dispatched, it is done in only one roundtrip.

If you just want to dispatch a single request, you could do so like this:

            var request = new GetProductOverviewsRequest

                { NamePattern = name, CategoryId = productCategoryId, SupplierId = supplierId };

            var response = dispatcher.Get<GetProductOverviewsResponse>(request);

            View.Products = response.Products;

Keep in mind that you do need to define the service endpoint in your client-side application's configuration file. You also need to register the KnownTypes client-side before you start using the Dispatcher or the service proxy in the manner discussed earlier.

Stubbing The Service Layer During Testing

If you write tests for your client-side code, you'll be glad to hear that i've included a DispatcherStub class which makes it easy to prepare responses to return and to inspect requests that were sent from the code you're testing. The approach i outlined in this post doesn't really lend itself to easy mocking, so the DispatcherStub class makes it all much easier.

First of all, make sure your client code always uses references of the IDispatcher type instead of directly using the Dispatcher or WcfDispatcher types. Then in your tests, inject DispatcherStub instances instead of real Dispatchers. The DispatcherStub class provides a few extra methods which you can use in your tests:

        [Test]

        public void RetrievesCategoriesAndSuppliersOnLoad()

        {

            var categoriesToReturn = new ProductCategoryDTO[0];

            var suppliersToReturn = new SupplierDTO[0];

 

            dispatcher.SetResponsesToReturn(

                new GetProductCategoriesResponse { ProductCategories = categoriesToReturn },

                new GetSuppliersResponse { Suppliers = suppliersToReturn });

 

            view.Expect(v => v.ProductCategories = categoriesToReturn);

            view.Expect(v => v.Suppliers = suppliersToReturn);

            view.Expect(v => v.DataBind());

 

            mocks.ReplayAll();

            CreateController().Load();

 

            view.VerifyAllExpectations();

            Assert.IsNotNull(dispatcher.GetRequest<GetProductCategoriesRequest>());

            Assert.IsNotNull(dispatcher.GetRequest<GetSuppliersRequest>());

        }

The SetResponsesToReturn and GetRequest methods make it much easier to stub the service layer than traditional mocking would.

Something you'll often want to test is that requests have been sent with the correct parameters:

            var request = dispatcher.GetRequest<GetProductOverviewsRequest>();

            Assert.AreEqual(productPattern, request.NamePattern);

            Assert.AreEqual(categoryId, request.CategoryId);

            Assert.AreEqual(supplierId, request.SupplierId);

Time to wrap it up

Again, i really like this approach. Unfortunately i haven't had the chance to use this in a real project yet, but based on my experiments i'm already very happy with it and am looking forward to use this in a real project. If you'd like to use this approach, download the library here, play around with it, try it out, try to break it, and let me know what needs to be fixed/changed/added. Or send patches ;)