The Inquisitive Coder - Davy Brion’s Blog

Trying to walk that thin line between intelligence and ignorance

Archive for May, 2008

How a simple foreach statement can waste an afternoon

Posted by Davy Brion on 27th May 2008

We all like the foreach statement, right? It’s easy to use. It looks good. It does a good job of what it’s supposed to do. What’s not to like? Well… today i learned it can actually be a great hiding place for performance issues.

I wrote the following code a while ago:

        public void ProcessGroupsAndTheirMembers(ActiveDirectoryConfiguration adConfig)

        {

            List<GroupPrincipal> groupPrincipals = GetABunchOfGroupsFromActiveDirectory(adConfig);

 

            foreach (var groupPrincipal in groupPrincipals)

            {

                HandleGroup(groupPrincipal);

                DealWithMembers(groupPrincipal.Members);

            }

        }

it’s actually a simplified version of the code i wrote, but you get the idea. It doesn’t look so bad, right? It fetches a bunch of groups from an Active Directory store, then it processes the groups and the members of those groups. It turns out there are actually a few problems with this code. First of all, when you retrieve a GroupPrincipal, there’s no way to make it fetch its Members collection in the same roundtrip (if i’m mistaken, please do correct me). So the Members property of the GroupPrincipal is a lazy-loaded collection. When you access it, it goes back to the Active Directory to fetch all the member Principals. There’s not really anything i can do about that, due to the limitations in how you can retrieve GroupPrincipals (again, unless i’m mistaken).

So basically, we fetch a bunch of data (the groups) and then when we loop through the retrieved data we fetch more data (the members) for each item in the loop. So we are making a hell of a lot of roundtrips if we have a lot of groups. I despise situations like that. And i never do this unless i can’t avoid it. As unfortunate as that is, it’s not the real problem that lurks in this code.

If you don’t have a lot of groups, then this code works perfectly and the data is processed quickly and the memory is cleaned up pretty soon after we leave ProcessGroupsAndTheirMembers method. Unless you suddenly have to loop through 6000 groups. And almost all of them have at least a few Members, some even have a lot of them. Keep in mind that for each group, we go back to the Active Directory store to retrieve the members. So that is at least one big query (to retrieve all of the groups) and another 6000 to retrieve all the members. As if that’s not bad enough, the Active Directory store turns out to be pretty slow. All of a sudden, the code that used to run in a matter of seconds takes 9 minutes.

So you fire up your tools to help you diagnose the problem… the profiler quickly shows that the code spends most of its time in the ProcessGroupsAndTheirMembers method. Process Explorer shows stable cpu usage (low at 25%, but stable… no peaks) and ever-increasing memory usage (all the way up to 400mb). This is the time where you get that warm and fuzzy “oh fuck…”-feeling. So you start experimenting with changes, and you test it… each time you test it you basically have to wait 9 minutes if the change didn’t have any effect. Joy…

It’s actually really simple once you figure it out… each GroupPrincipal object takes up some memory space. If its Members collection is filled up, the GroupPrincipal will hold references to each member Principal in the collection. The object graph that you are holding in memory basically increases each time you pass through the loop because each GroupPrincipal will hold all of its Members after we’ve processed it.

But hey, we have garbage collection! It’ll clean up the used memory! Yea it does… eventually. Do you know how many garbage collections could occur in a period of 9 minutes? A lot of them actually. Especially if your code is aggressively requesting more and more memory space.

The problem, of course, is with the foreach statement (duh, i already gave it away in the title). As you can see, we don’t really do anything with the GroupPrincipal once we’ve processed it. Yet it’s still kept in the groupPrincipals list, for the duration of the entire loop. And we can’t remove it from the list while we’re in the foreach because then the underlying iterator will throw exceptions once we move to the next item. The trick was simply to replace the foreach with a do-while-loop (how old-school!) and to get rid of the GroupPrincipal once it was processed:

        public void ProcessGroupsAndTheirMembers(ActiveDirectoryConfiguration adConfig)

        {

            List<GroupPrincipal> groupPrincipals = GetABunchOfGroupsFromActiveDirectory(adConfig);

 

            do

            {

                GroupPrincipal groupPrincipal = groupPrincipals[0];

                HandleGroup(groupPrincipal);

                DealWithMembers(groupPrincipal.Members);

                groupPrincipals.RemoveAt(0);

                groupPrincipal.Dispose();

            } while (groupPrincipals.Count > 0);

        }

When i ran this code, memory usage remained stable and cpu usage actually went down to 5%. The time needed to process the groups went from 9 minutes to 5. Still a lot, but as evidenced by the very low cpu usage, the code is constantly waiting for the data from Active Directory to cross the wire and then it quickly processes it, and then it waits for the next bunch of data.

So, as this story clearly demonstrates, the foreach statement can be quite the evil bitch even though it’s usually the nice girl-next-door kinda statement. It’s too bad i wasted a few hours on this… well… honestly, a part of me loves situations like these in a weird, sick and twisted kinda way. You always learn something very interesting from it :)

Hope you enjoyed this episode of How The Code Turns.

Share/Save/Bookmark

Posted in Performance | 2 Comments »

The Multithreaded Task Executor

Posted by Davy Brion on 26th May 2008

Sometimes, you’ve got a bunch of actions that you need to execute in a loop. The problem is that those actions are all performed synchronously so this could take some time depending on the action. But, if the action itself is thread-safe, and the actions are not dependent on the results of previous actions, you might get much better performance if you spread that workload over a few different threads. Especially if you have multiple CPU cores.

Wouldn’t it be cool if you could do something like this:

            MultiThreadedTaskExecutor taskExecutor = new MultiThreadedTaskExecutor(numberOfThreadsToUse);

 

            foreach (Input input in inputs)

            {

                Input newVariable = input;

                taskExecutor.QueueTask(() => ProcessInput(newVariable));

            }

 

            taskExecutor.RunTasksAndWait();

Note: the reason why you need to use a new variable inside the loop is to avoid that the ‘input’ variable (not the reference to the object, but the actual variable) is captured by the anonymous method. If you don’t use a new variable, each created anonymous method would refer to the ‘input’ loop variable, which by the time all the tasks are executed points to the last Input instance in the inputs collection. The result would be that each task is executed on the same input instance. This is a known issue with variable capturing and anonymous methods.

Anyways… the code above basically spreads the workload over the given number of threads.

The rough, not-quite-production-ready code of the MultiThreadedTaskExecutor class looks like this:

    public class MultiThreadedTaskExecutor

    {

        private readonly List<Thread> threads = new List<Thread>();

        private readonly List<EventWaitHandle> eventWaitHandles = new List<EventWaitHandle>();

        private readonly List<Type> swallowedExceptionTypes;

 

        private readonly Queue<Action> taskQueue;

        private readonly object queueMonitor = new object();

 

        public MultiThreadedTaskExecutor(int numberOfThreads)

        {

            swallowedExceptionTypes = new List<Type>();

            taskQueue = new Queue<Action>();

            CreateThreads(numberOfThreads);

        }

 

        public void QueueTask(Action task)

        {

            taskQueue.Enqueue(task);

        }

 

        public void AddExceptionTypeToSwallow(Type type)

        {

            swallowedExceptionTypes.Add(type);

        }

 

        public void RunTasksAndWait()

        {

            foreach (Thread thread in threads)

            {

                thread.Start();

            }

 

            WaitHandle.WaitAll(eventWaitHandles.ToArray());

        }

 

        private void CreateThreads(int number)

        {

            for (int i = 0; i < number; i++)

            {

                var eventWaitHandle = new EventWaitHandle(false, EventResetMode.ManualReset);

                eventWaitHandles.Add(eventWaitHandle);

                threads.Add(new Thread(() => ProcessTasks(eventWaitHandle)));

            }

        }

 

        private void ProcessTasks(EventWaitHandle eventWaitHandle)

        {

            try

            {

                Action action;

 

                while ((action = GetTask()) != null)

                {

                    try

                    {

                        action();

                    }

                    catch (Exception e)

                    {

                        if (!swallowedExceptionTypes.Contains(e.GetType())) throw;

                    }

                }

            }

            finally

            {

                eventWaitHandle.Set();

            }

        }

 

        private Action GetTask()

        {

            lock (queueMonitor)

            {

                if (taskQueue.Count == 0) return null;

                return taskQueue.Dequeue();

            }

        }

    }

So how does it work? It uses a queue to hold each task that was added by the consumer of the class. It creates the given amount of threads and also creates an EventWaitHandle for each thread. Then when the user starts the execution with a call to RunTasksAndWait, each thread is started and then the call to RunTasksAndWait will wait until each thread is finished. In the meantime, each thread will get the next task off the queue and executes it. If an exception is thrown within the task, it is caught and is either swallowed or rethrown (the consumer can add exception types that can be swallowed). Each thread keeps doing this until it can’t get a new task off the queue. When that happens, the thread signals the EventWaitHandler and then it dies. When all threads are dead, RunTasksAndWait will stop blocking and control is returned to the caller. All of the tasks have been executed and the workload has been spread over the given amount of threads.

Note: due to the call to WaitHandle.WaitAll, this won’t work on STA threads because the implementation of WaitHandle.WaitAll simply throws a NotSupportedException on STA threads.

Keep in mind that this is a rough version of this code… it definitely needs a bit more polish (better exception handling for when the threads are interrupted and stuff like that mostly) but you get the idea :)

But if you know a better way to do this, or if you spot flaws in this implemenation, i’d love to hear about it :)

Share/Save/Bookmark

Posted in Multithreading, Patterns | 4 Comments »

The First 100 Posts

Posted by Davy Brion on 25th May 2008

This is my 100th post. I started this blog on June 17th in 2007 so it took me little less than a year to reach this first milestone. To be honest, when i started i never thought i’d stay interested in maintaining a blog for so long. And i sure didn’t expect to reach 100 posts. Granted, about 20 of those posts are posts that merely link to other content, but a good 80 of them are indeed ‘original content’… not a bad ratio if i may say so myself ;)

Anyways, it’s always good to look back when you reach a milestone (even if it is one you made up yourself), so lets have a look shall we?

These are the posts that have received the most views (ordered descending):

  1. NHibernate Mapping Examples

    I remember spending a full week of vacation time working on this… i guess the work payed off because this post really brought a lot of people to this blog. I can only hope that it actually helped a few people as well :)

  2. Visual Studio 2008 Release Date

    Wow, what a fluke. I merely thought it was an interesting link for the 3 people who were following my blog at that time. A couple weeks later it was the 3rd result in Google when you searched for ‘visual studio 2008 release date’. That was actually a bit frustrating because posts that i was actually proud of didn’t get any attention at all, while this one was getting a lot of views, but you knew the visitors would’nt stick around.

  3. Multilingual Data And NHibernate

    I kinda liked this one… I think it’s a pretty nice approach but due to some unfortunate circumstances i was never able to use this in a real project. Too bad cause i really wanted to see how it would turn out.

  4. Read Only Generic Dictionary

    Not much to say about this one… apparantly i’m not the only one who’d like something like this to be included in the .NET Framework :)

  5. Integration Tests With NHibernate

    This one wasn’t bad either… I wouldn’t go with this kind of integration testing anymore, but the research and experiments that led to this post resulted in my first (and only, so far) patch to the nhibernate source code which was accepted and has been included since the 1.2.1 version. Yay :)

  6. Creating A Sortable Bindinglist

    Meh… there’s not really a lot i can say about this one since it’s a pretty simple and short post. Apparantly a lot of people where looking for something like this though

  7. Native ID Generation With NHibernate

    Apparantly, NHibernate’s native ID generator is something that leaves a few people confused.

  8. Easy And Fast Sorting Of Objects

    Again, something a lot of people are looking for. I could’ve done a much better job on that post, but it is what it is. It did introduce me to Marc Brooks’ DynamicComparer which i really like a lot. It’s too bad that a lot of people will probably use LINQ’s ordering capabilities instead of this since this approach will definitely have much better performance than LINQ’s ordering capabilities.

  9. ObjectDataSource And Sorting Collections

    I liked this one… it’s short yet not too short. I love using this approach :)

  10. Sending NHibernate Entities Over The WCF Wire

    Mixed feelings about this one… The post itself wasn’t bad, but i’d never use this approach on a real project. (in case you’re wondering: nhibernate entities at the business side, flat-and-screen-based DTO’s client side)

So those are the posts that have gotten the most views so far… and none of them is among my list of personal favorites (simply ordered by date):

Simply browsing the entire list of posts made me realize just how much i’ve learned in the past year. A lot has changed in that year as well… due to an unfortunate situation i quit working at a client i’d been working for for over 5 years. It was an awkward situation for everyone involved, but i did learn a lot from it. It’s too bad some friendships were hurt or even destroyed in the process, but then again, the mere fact that those friendships were impacted probably says a lot about how real or valuable those friendships were. Oh well, fake friends were lost, real friends were recognized and valuable lessons were learned. So for the last couple of months i’ve been working at another client, and it’s definitely nice to see what kind of things are different, and also what kind of things are the same in these large companies. And i’ve come to the conclusion that it’s just not right for me. I don’t wanna work at large companies anymore, at least not for the next few years. I’ve got about one month to go, and then i’ll get some experience in another scenario: working at a relatively small software development company. I don’t know yet what to expect, but i’m looking forward to find out. Oh, and i’m also looking forward to writing the next 100 posts obviously :)

For those of you who’ve been following this blog: thanks, i appreciate it :)

Share/Save/Bookmark

Posted in About The Blog | 4 Comments »

Storing data in the HttpSession

Posted by Davy Brion on 25th May 2008

As we all know (i’d hope!), storing data in the HttpSession is pretty bad. HttpSessions remain in memory long after (depending on the configured timeout) the user has performed his/her last action on your site. Think about that for a second… everything you store in the HttpSession will remain in memory for a while even though it has no chance of being used again, since the user is already gone. If you don’t have a large amount of concurrent users, this might not cause any noticeable problems. But you have to think about the multiplier effect. As your user base increases, so do your memory requirements. You’ll need more memory to keep serving those users. But if you store a lot of data in the HttpSession, your wasted memory will increase along with your increased user base. While you’re serving your active users, you’re still holding a lot of data in memory from users who’ve left your site 5 or 10 minutes ago. Because that unneeded data is still referenced from HttpSessions that will never be used again, it prevents the garbage collector from releasing that memory. When your memory usage gets to the point where the operating system needs to start paging, you’ll notice dramatic slowdowns in your site’s performance.

So you really want to limit the information you store in the HttpSession to that which pertains to the actual ’session’ of the user. Identification of the user, chosen language, stuff like that. But don’t use the HttpSession to store data that is related to the current screen the user is in. This is unfortunately commonly done, but this can easily lead to the problems mentioned above. It’s better to try to work as stateless as possible as this allows your application to scale to a large user base more easily. Don’t store retrieved data in the HttpSession, just retrieve it again when it’s needed. However, some kinds of data are very expensive to retrieve. In these cases, it might actually be better to store it in the HttpSession to avoid having to retrieve it excessively. So how do you avoid that these stored pieces of data have a negative impact on the memory usage of your application?

The Release It book mentions a great trick for this. I haven’t used it yet in an application, but it definitely makes a lot of sense. Instead of storing a reference to data in your HttpSession (and thus, preventing that data from being garbage collected until the HttpSession is garbage collected), you should use a ’soft reference’. A soft reference is kinda like a simple pointer to the data, but it does not prevent the data from being garbage collected. The soft reference doesn’t count as a real reference to the garbage collector, so when the collector needs to clear memory, the data you reference with a soft reference will be garbage collected (if it’s not referenced anywhere else that is…). If that happens your application code has to deal with it. In that case, you should retrieve the data again. Will this cause you to retrieve the expensive piece of data more times than you would have to if you had stored it with a normal reference in the HttpSession? Possibly. You can’t give a definitive answer to that question because it depends on how frequently the garbage collector needs to clear memory. But you will most likely benefit from possibly still having that data in memory. If it’s still there, great! Use it. If it’s no longer there, fetch it again. You’ll probably reduce the amount of times you need to retrieve it, but you’ll also avoid keeping the data from being garbage collected when the server is low on memory.

In .NET, you can use the WeakReference class to obtain a ’soft reference’. Lets demonstrate this approach with a quick-n-dirty example:

When you want to store data in the HttpSession, instead of doing this:

            IEnumerable<OrderView> outstandingOrders

                = orderManagementServiceProxy.GetOverviewOfAllOutstandingOrders();

 

            Session["outstandingOrders"] = outstandingOrders;

You could do this:

            IEnumerable<OrderView> outstandingOrders

                = orderManagementServiceProxy.GetOverviewOfAllOutstandingOrders();

 

            Session["outstandingOrders"] = new WeakReference(outstandingOrders);

And when you’d need to retrieve that data in a later request, you could do this:

            WeakReference reference = Session["outstandingOrders"] as WeakReference;

 

            if (reference != null && reference.IsAlive)

            {

                outstandingOrders = reference.Target as IEnumerable<OrderView>;

            }

            else

            {

                outstandingOrders = orderManagementServiceProxy.GetOverviewOfAllOutstandingOrders();

            }

Like i said, this is a quick-n-dirty example… normally you’d want to prevent direct access to the HttpSession and you’d probably write some kind of class that takes care of wrapping the reference in a WeakReference and unwrapping it from a WeakReference but this code is just to illustrate the approach.

Share/Save/Bookmark

Posted in ASP.NET | No Comments »

NHibernate and virtual methods/properties

Posted by Davy Brion on 24th May 2008

I love NHibernate but one of the things that bothers the hell out of me is that i keep forgetting to add the virtual keyword to each method or property in my entities. And since NHibernate needs your classes’ properties and methods to be virtual, this causes run-time errors when i run my tests. Since i’m already using custom compile time checks, i figured i might as well add another one… from now on, i want my compilation to fail if any of my NHibernate entities have public methods/properties that aren’t marked virtual.

Once again, it’s PostSharp to the rescue:

using System;

using System.Reflection;

 

using PostSharp.Extensibility;

using PostSharp.Laos;

 

namespace Northwind.Aspects

{

    [Serializable]

    [AttributeUsage(AttributeTargets.Assembly | AttributeTargets.Method | AttributeTargets.Property)]

    [MulticastAttributeUsage(MulticastTargets.Method, TargetMemberAttributes = MulticastAttributes.Managed |

        MulticastAttributes.NonAbstract | MulticastAttributes.Instance |

        MulticastAttributes.Protected | MulticastAttributes.Public)]

    public class RequireVirtualMethodsAndProperties : OnMethodBoundaryAspect

    {

        public override bool CompileTimeValidate(MethodBase method)

        {

            if (!method.IsVirtual)

            {

                string methodName = method.DeclaringType.FullName + “.” + method.Name;

 

                var message = new Message(SeverityType.Fatal, “MustBeVirtual”,

                    string.Format(“{0} must be virtual”, methodName), GetType().Name);

                MessageSource.MessageSink.Write(message);

 

                return false;

            }

 

            return true;

        }

    }

}

And then we make sure this check is applied to my NHibernate entities:

#if DEBUG

[assembly: RequireVirtualMethodsAndProperties(AttributeTargetTypes = "Northwind.Domain.Entities.*")]

#endif

Now, whenever i forget to mark my properties/methods as virtual, i get this:


EXEC : error MustBeVirtual: Northwind.Domain.Entities.Region.RemoveTerritory must be virtual
EXEC : error MustBeVirtual: Northwind.Domain.Entities.Region.AddTerritory must be virtual
EXEC : error MustBeVirtual: Northwind.Domain.Entities.Region.get_Territories must be virtual
EXEC : error MustBeVirtual: Northwind.Domain.Entities.Region.set_Description must be virtual
EXEC : error MustBeVirtual: Northwind.Domain.Entities.Region.get_Description must be virtual
EXEC : error MustBeVirtual: Northwind.Domain.Entities.Region.set_Id must be virtual
EXEC : error MustBeVirtual: Northwind.Domain.Entities.Region.get_Id must be virtual

...

Done building project "Northwind.csproj" -- FAILED.

And there we go :)

Share/Save/Bookmark

Posted in NHibernate, PostSharp | 1 Comment »

Creating Sanity Checks

Posted by Davy Brion on 24th May 2008

Ever been in a situation where you notice that one of the team-members used classes from an assembly that really shouldn’t be used in that part of the code? For instance, accessing the data layer from the presentation layer. It’s not always easy to keep an eye on improper assembly usage. You could keep an eye on the referenced assemblies manually. You could write an FxCop rule that checks for disallowed assembly references. There’s a lot of stuff you can do, but it’ll always be an after-the-facts check. By that time, the ‘illegal’ code is already there.

Wouldn’t it be cool if you could break the compilation whenever a developer tries to build code that has improper assembly usage? Actually, with PostSharp, we can do just that. You can simply create an aspect like this:

    [Serializable]

    [AttributeUsage(AttributeTargets.Assembly)]

    public class SanityCheck : OnMethodInvocationAspect

    {

        public override bool CompileTimeValidate(System.Reflection.MethodBase method)

        {

            var methodName = method.DeclaringType.FullName + “.” + method.Name;

 

            var message = new Message(SeverityType.Fatal, “ProhibitedMethodCall”, String.Format(

                “Sorry, but we can’t allow you to call {0} from the current assembly”, methodName),

                “SanityCheck”);

            MessageSource.MessageSink.Write(message);

 

            return false;

        }

    }

First of all, we declare that the SanityCheck attribute can only be applied on the assembly level. Notice that we inherit from OnMethodInvocationAspect. This aspect is applied on events, properties, and methods. So basically, whenever you call a property or method or try to hook to an event in an assembly that you’ve applied this attribute to, our SanityCheck aspect will run. Well, actually we won’t get that far. We override the virtual CompileTimeValidate method where we display a message and then we return false. Meaning that this compilation is invalid.

Suppose you’ve used the attribute in the AssemblyInfo.cs file of your presentation layer like this:

[assembly: SanityCheck(AttributeTargetAssemblies = "System.Data")]

If you try to compile the following code:

            DataTable table = new DataTable();

            table.NewRow();

You’ll see the following line in your build output:

EXEC : error ProhibitedMethodCall: Sorry, but we can't allow you to call System.Data.DataTable.NewRow from the current assembly

And most importantly:

========== Build: 0 succeeded or up-to-date, 1 failed, 0 skipped ==========

Obviously, this will only cause compile errors when you try to touch the prohibited parts within the assembly where you used the SanityCheck attribute. If you call a method in another assembly that is allowed to touch System.Data, it will not cause a compile error.

Share/Save/Bookmark

Posted in PostSharp, Software Development | 3 Comments »

WCF Exception Handling with PostSharp

Posted by Davy Brion on 23rd May 2008

In your WCF service methods, you should always provide proper exception handling. The exception handling code is almost always the same, so it’s usually just copy/pasted in each method. Here’s an example of a typical service method:

        public void PersistOrder(Order order)

        {

            try

            {

                Container.Resolve<IPersistOrderHandler>().Handle(order);

            }

            catch (BusinessException b)

            {

                throw new FaultException<BusinessExceptionInformation>(new BusinessExceptionInformation(b),

                    new FaultReason(b.Message));  

            }

            catch (FaultException)

            {

                throw;

            }

            catch (Exception e)

            {

                Container.Resolve<ILogger>().LogException(e);

                throw new FaultException(new FaultReason(e.Message));

            }

        }

That’s a lot of code, even though only ONE line will differ from the other service methods in this service. Let’s clean up this mess, shall we? Exception handling is a cross cutting concern so we might as well put it in one place (at least, one place for each different kind of exception handling that you need). We’ll use PostSharp to define an ApplyServiceExceptionHandling aspect:

    [Serializable]

    public class ApplyServiceExceptionHandling : OnExceptionAspect

    {

        public override void OnException(MethodExecutionEventArgs eventArgs)

        {

            if (eventArgs.Exception is BusinessException)

            {

                throw new FaultException<BusinessExceptionInformation>(

                    new BusinessExceptionInformation(eventArgs.Exception as BusinessException),

                    new FaultReason(eventArgs.Exception.Message));

            }

            else if (eventArgs.Exception is FaultException)

            {

                eventArgs.FlowBehavior = FlowBehavior.RethrowException;

            }

            else

            {

                Container.Resolve<ILogger>().LogException(eventArgs.Exception);

                throw new FaultException(new FaultReason(eventArgs.Exception.Message));

            }

        }

    }

And now we can modify the earlier service method to this:

        [ApplyServiceExceptionHandling]

        public void PersistOrder(Order order)

        {

            Container.Resolve<IPersistOrderHandler>().Handle(order);

        }

That’s a lot cleaner than the previous version right? Actually, this service has more than one service method so we’re better off moving the ApplyServiceExceptionHandling attribute to the class level instead of the method level:

    [ApplyServiceExceptionHandling]

    public class OrderManagementService : IOrderManagementService

now we can get rid of all of the exception handling code in each service method, but each one will have exception handling applied to it because the exception handling aspect is applied to the class, and thus, to each method of that class. And if you need to modify your exception handling (to handle TargetInvocationExceptions with BusinessExceptions as InnerExceptions for instance?) you’d only have to do this in one place.

Btw, i’m aware of WCF’s IErrorHandler interface (which allows you to handle exceptions in a general manner as well)… i just don’t like it :)

Share/Save/Bookmark

Posted in Aspect Oriented Programming, PostSharp, WCF | 1 Comment »

Adding Behavior With PostSharp

Posted by Davy Brion on 21st May 2008

A little while ago, i talked about adding behavior to components using Windsor’s Interceptors. I wanted to try something similar with PostSharp, which is a very powerful Aspect Oriented Programming framework for the .NET world. Discussing everything PostSharp can do (which is a lot) is way beyond the scope of this post, so we’ll just focus on what i’m trying to do, and how PostSharp helps us with that.

In the previous post, i used a logging example… i didn’t want logging code mixed up with my real code, so i used a Windsor Interceptor to intercept calls to classes that i had configured to be logged. The interceptor would then log before and after the method calls were executed. In this post we’ll do something very similar, but a bit more advanced. We’re gonna write some tracing logic that will write a trace statement when the method is entered, along with the parameter values for each of its parameters. Then after the original method has executed, we’ll write a trace statement to indicate that we have left the method, and we’ll display the return value of the method as well, if there is one. That kind of tracing information can be very valuable, but writing that code is extremely tedious work. Nobody wants to do that, right? I sure as hell don’t.

We’ll use the following simple class as an example:

    public class Calculator

    {

        public int Add(int firstValue, int secondValue)

        {

            return firstValue + secondValue;

        }

 

        public int Subtract(int firstValue, int secondValue)

        {

            return firstValue - secondValue;

        }

    }

This calculator has a pretty limited feature-set, but we don’t need anything more for this example. Ok, so how do we add the tracing code without actually adding it to this code? We can just compile the code, and then we can have PostSharp inject some extra code to the compiled code. Sounds pretty hard, right? It is. But PostSharp Laos is a small framework that runs on top of PostSharp and it takes care of the hard work for you. PostSharp Laos offers some base classes which make it really easy for you to write your aspects (if you don’t understand that term, you should have clicked on the aspect oriented programming link earlier in the post ;)). There are a couple of approaches you can choose between, and they all have their pro’s and con’s. A nice overview of the different kinds of aspects you can inherit from can be found here. For this example, we’ll use the OnMethodInvocationAspect base class. This basically intercepts calls to methods and allows you to add some logic.

So what would our tracing aspect look like? How about this:

    [Serializable]

    public class TraceAspect : OnMethodInvocationAspect

    {

        public override void OnInvocation(MethodInvocationEventArgs eventArgs)

        {

            string methodName = GetFullMethodName(eventArgs);

            WriteOutput(String.Format(“Entering {0}”, methodName));

            WriteParameterInfo(eventArgs.Delegate.Method.GetParameters(), eventArgs.GetArgumentArray());

            base.OnInvocation(eventArgs); // calls the original method

            WriteOutput(String.Format(“Leaving {0}”, methodName));

            WriteReturnValueInfo(eventArgs.Delegate.Method.ReturnType, eventArgs.ReturnValue);

        }

 

        private static void WriteParameterInfo(ParameterInfo[] parameterInfos, object[] parameters)

        {

            if (parameterInfos == null || parameterInfos.Length == 0)

            {

                return;

            }

 

            WriteOutput(“With the following parameters: “);

 

            for (int i = 0; i < parameterInfos.Length; i++)

            {

                WriteOutput(string.Format(“{0} = {1}”, parameterInfos[i].Name, parameters[i]));

            }

        }

 

        private static void WriteReturnValueInfo(Type returnType, object returnValue)

        {

            if (returnType == typeof(void))

            {

                return;

            }

 

            WriteOutput(string.Format(“Return Value: {0}”, returnValue));   

        }

 

        private static string GetFullMethodName(MethodInvocationEventArgs args)

        {

            return args.Delegate.Target.GetType().FullName + “.” + args.Delegate.Method.Name;

        }

 

        private static void WriteOutput(string line)

        {

            Trace.WriteLine(DateTime.Now.TimeOfDay + ” “ + line);

        }

    }

So now we have a trace aspect which shows some valuable information about method calls… it shows when the method is entered, which parameters were passed in, when we leave the method and what the return value is.

So how do we apply this aspect to our code? We modify the definition of the Calculator class like this:

    [TraceAspect]

    public class Calculator

And then we compile… Well actually you have to add something to your project file so PostSharp can modify the compiled code, but since that’s covered nicely in the PostSharp documentation we’ll skip that step in this post. Anyways, when you compile this, you’ll get the following build output:

------ Build started: Project: UsingPostSharp, Configuration: Debug Any CPU ------
C:\WINDOWS\Microsoft.NET\Framework\v3.5\Csc.exe /noconfig /nowarn:1701,1702 /errorreport:prompt /warn:4 /define:POSTSHARP;DEBUG;TRACE /reference:..\Libs\postsharp\PostSharp.Laos.dll /reference:..\Libs\postsharp\PostSharp.Public.dll /reference:"C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.5\System.Core.dll" /reference:"C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.5\System.Data.DataSetExtensions.dll" /reference:C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.Data.dll /reference:C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.dll /reference:C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.Xml.dll /reference:"C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.5\System.Xml.Linq.dll" /debug+ /debug:full /filealign:512 /optimize- /out:obj\Debug\UsingPostSharp.exe /target:exe Program.cs Properties\AssemblyInfo.cs TraceAspect.cs

Compile complete -- 0 errors, 0 warnings
“C:\mydocs\src\dbr\Experiments\UsingPostSharp\Libs\postsharp\PostSharp.exe” “C:\mydocs\src\dbr\Experiments\UsingPostSharp\Libs\postsharp\Default.psproj” “C:\mydocs\src\dbr\Experiments\UsingPostSharp\UsingPostSharp\obj\Debug\UsingPostSharp.exe” “/P:Output=obj\Debug\PostSharp\UsingPostSharp.exe ” “/P:ReferenceDirectory=C:\mydocs\src\dbr\Experiments\UsingPostSharp\UsingPostSharp ” “/P:Configuration=Debug ” “/P:Platform=AnyCPU ” “/P:SearchPath=bin\Debug\, ” “/P:IntermediateDirectory=obj\Debug\PostSharp ” “/P:CleanIntermediate=False ” “/P:MSBuildProjectFullPath=C:\mydocs\src\dbr\Experiments\UsingPostSharp\UsingPostSharp\UsingPostSharp.csproj ” “/P:SignAssembly=False ” “/P:PrivateKeyLocation= ”
PostSharp 1.0 [1.0.9.365] - Copyright (c) Gael Fraiteur, 2005-2008.

info PS0035: C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\ilasm.exe “C:\mydocs\src\dbr\Experiments\UsingPostSharp\UsingPostSharp\obj\Debug\PostSharp\UsingPostSharp.il” /QUIET /EXE /PDB “/RESOURCE=C:\mydocs\src\dbr\Experiments\UsingPostSharp\UsingPostSharp\obj\Debug\PostSharp\UsingPostSharp.res” “/OUTPUT=C:\mydocs\src\dbr\Experiments\UsingPostSharp\UsingPostSharp\obj\Debug\PostSharp\UsingPostSharp.exe” /SUBSYSTEM=3 /FLAGS=1 /BASE=4194304 /STACK=1048576 /ALIGNMENT=512 /MDV=v2.0.50727
UsingPostSharp -> C:\mydocs\src\dbr\Experiments\UsingPostSharp\UsingPostSharp\bin\Debug\UsingPostSharp.exe

========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========

If we use the calculator class like this:

            var calculator = new Calculator();

            calculator.Add(10, 15);

            calculator.Subtract(10, 15);

We’ll get the following trace output:

21:48:49.9414848 Entering UsingPostSharp.Calculator.~Add
21:48:49.9615136 With the following parameters:
21:48:49.9615136 firstValue = 10
21:48:49.9615136 secondValue = 15
21:48:49.9615136 Leaving UsingPostSharp.Calculator.~Add
21:48:49.9615136 Return Value: 25
21:48:49.9615136 Entering UsingPostSharp.Calculator.~Subtract
21:48:49.9615136 With the following parameters:
21:48:49.9615136 firstValue = 10
21:48:49.9615136 secondValue = 15
21:48:49.9615136 Leaving UsingPostSharp.Calculator.~Subtract
21:48:49.9615136 Return Value: -5

How nice is that? With some exception handling, better formatting and some clever indenting, this tracing aspect could be the only tracing code you’ll ever need from now on :)

So how does it work? Well, we can look at the compiled code in reflector for that. Here’s how the Add method looks like after PostSharp modified the code:

private int ~Add(int firstValue, int secondValue)
{
return (firstValue + secondValue);
}

[DebuggerNonUserCode, CompilerGenerated]
public int Add(int firstValue, int secondValue)
{
Delegate delegateInstance = new ~PostSharp~Laos~Implementation.~delegate~0(this.~Add);
object[] arguments = new object[] { firstValue, secondValue };
MethodInvocationEventArgs eventArgs = new MethodInvocationEventArgs(delegateInstance, arguments);
~PostSharp~Laos~Implementation.TraceAspect~1.OnInvocation(eventArgs);
return (int) eventArgs.ReturnValue;
}

As you can see, it’s renamed our Add method to ~Add, and it added another Add method which calls the OnInvocation method of our TraceAspect class with the correct MethodInvocationEventArgs. Pretty cool stuff IMO. Oh and btw, if you’re debugging this code in Visual Studio, you still only see your own code, with the original method name. While you step through it, the code runs just like it normally would, and the applied aspects are also executed. Very impressive.

If you use the OnMethodBoundaryAspect instead of the OnMethodInvocationAspect, you’ll see that the modified code doesn’t replace your method like it did here. But it will add a lot of extra code to it as well. All in all, there are a lot of possibilities here. If i’m not mistaken, i can now even use our TraceAspect and use it on classes in other assemblies, even if i don’t have access to them. We’ll look into that in a future post :)

Anyway, PostSharp is an extremely impressive piece of software which provides a whole lot of power and flexibility. You should definitely check it out and play around with it… i know i’m gonna be experimenting with it more often to see what kind of weird and cool stuff we can make it do :)

Share/Save/Bookmark

Posted in Aspect Oriented Programming, PostSharp, Software Development | 5 Comments »

More Entity Framework stuff

Posted by Davy Brion on 20th May 2008

Looks like everyone is joining the Entity Framework discussion… So, here are a few more interesting links:

More Entity Framework Thoughts
Entity Framework: Our Albatross
EF Long Term Plans
On Comparing Current Tools to Futureware

Definitely worth reading if you’re considering using Entity Framework on real projects.

Share/Save/Bookmark

Posted in Entity Framework | 2 Comments »

Entity Framework stuff

Posted by Davy Brion on 20th May 2008

I was gonna post these links in my weekly links overview, but since these are all related i figured it deserved it’s own post. It starts off with Danny Simmons (of the Entity Framework development team) comparing Entity Framework with some other data access technologies and explaining why he thinks you should use it. Rob Eisenberg has a short post detailing his thoughts on why you shouldn’t use Entity Framework. Ayende’s post doesn’t really comment on Danny’s post, but he does highlight a major source control issue with Entity Framework which should make anyone who’s thinking about using Entity Framework in a team environment think twice. And finally, Jeremy Miller comments on Danny’s post by mentioning another big downside of the Entity Framework.

Oh and did you know it won’t support lazy loading out of the box? Yes, really. Yes, this is the year 2008. No, no lazy loading. Which means that any code using your business objects is going to be littered with statements to check for empty collections/references and dealing with that. Yikes… Let’s hope the next version comes out before the next version of Visual Studio :)

Share/Save/Bookmark

Posted in Entity Framework | No Comments »