Posted by Davy Brion on 25th February 2009
Just wanted to apologize for my site being either down or extremely slow in the past couple of days… my hosting provider is having a lot of issues with the server i’m currently on, and they are looking into it. Hopefully, it’ll be resolved soon.
Posted in About The Blog | No Comments »
Posted by Davy Brion on 25th February 2009
This has got to be the weirdest memory leak i’ve ever investigated. We have this kick-ass Silverlight application, but unfortunately it suffered from very high memory usage that went up rather rapidly. So i attached windbg to the browser’s process, took a memory dump and checked out which objects were still available in the heap. Much to my surprise, pretty much everything we instantiated was retained in memory and never got removed from the heap.
So i started looking into the usual things: making sure disposable instances where disposed of properly, that evenhandlers were unregistered properly, etc. I went over the code and it seemed to be alright. Stepping through the code with a debugger verified that disposables were indeed disposed of, and that all event handlers were unregistered.
So why was pretty much everything kept in memory? Further research with windbg showed that every ProgressBar instance that was ever created (and we use a lot of them, basically every time we make a call to the application server) kept a reference to the UserControl it was placed on and thus, kept the UserControl and all the references it contained alive. In our case, that includes our presentation models and obviously all of the contained child UserControls.
The ProgressBar is defined like this:
<ProgressBar Height="40" Style="{StaticResource ourKickAssStyle}" VerticalAlignment="Center" Width="40" IsIndeterminate="True"/>
The key here is the usage of the IsIndeterminate property… setting this to true causes the ProgressBar to move continuously without respecting any current Value property. You know, basic stuff. The thing is… if i change the definition of the ProgressBar to this:
<ProgressBar Height="40" Style="{StaticResource ourKickAssStyle}" VerticalAlignment="Center" Width="40" />
The memory leak suddenly went away 
Obviously, this isn’t a solution because the ProgressBar now doesn’t really indicate any progress and we need our kick ass custom animation to retain the coolness of the application.
So for some reason, when you set the ProgressBar’s IsIndeterminate property to true, it actually keeps all of its references alive even when the ProgressBar control is removed from its parent control. Happy times.
We now have the following ugly method in one of our base UI classes:
private static void StopProgressBars(DependencyObject dependencyObject)
{
var count = VisualTreeHelper.GetChildrenCount(dependencyObject);
for (int i = 0; i < count; i++)
{
var child = VisualTreeHelper.GetChild(dependencyObject, i);
if (child != null)
{
var progressBar = child as ProgressBar;
if (progressBar != null)
{
progressBar.IsIndeterminate = false;
}
StopProgressBars(child);
}
}
}
Posted in Memory Management, Silverlight | 5 Comments »
Posted by Davy Brion on 24th February 2009
A coworker of mine pointed this out… if you’re doing Silverlight development, make sure you install Silverlight Spy!
Posted in Silverlight | No Comments »
Posted by Davy Brion on 22nd February 2009
After the uncommented code, and then the commented version of the code, you finally get to see the tests that verify that solution protects the code from the issue it was facing. I think all 3 posts (and the comments on them) sufficiently explain the problem and the solution so i won’t go through the trouble of explaining everything in this post. The tests however, might not be very clear to everyone. I’m only posting 3 tests, though there are more but then the post would just be way too long.
These tests use the following 2 fields:
private Broadcaster broadcaster;
private IClientProxyFactory clientFactory;
Which are set up before each test like this:
clientFactory = MockRepository.GenerateMock<IClientProxyFactory>();
broadcaster = new Broadcaster(clientFactory);
First of all, take a look at some of the utility methods that these tests use:
private List<IClientProxy> GetBroadcastersClients()
{
var clientsFieldInfo = typeof(Broadcaster).GetField("clients", BindingFlags.NonPublic | BindingFlags.Instance);
return (List<IClientProxy>)clientsFieldInfo.GetValue(broadcaster);
}
private Exception GetExceptionThrownBy(Action yourCode)
{
try { yourCode(); } catch (Exception e) { return e; }
return null;
}
private void RegisterClientWithBroadcaster(IClientProxy client)
{
clientFactory.Stub(f => f.CreateClientProxyForCurrentContext(null))
.IgnoreArguments().Return(client).Repeat.Once();
broadcaster.Register();
}
private IClientProxy RegisterClientWithImplementationForSend(Action implementation)
{
var client = MockRepository.GenerateMock<IClientProxy>();
client.Stub(c => c.SendNotificationAsynchronously(null))
.IgnoreArguments().WhenCalled(obj => implementation());
RegisterClientWithBroadcaster(client);
return client;
}
private IClientProxy RegisterClientWithEmptySendImplementation()
{
return RegisterClientWithImplementationForSend(() => { });
}
private void RegisterClientsWithImplementationForSend(int number, Action implementation)
{
var clients = new IClientProxy[number];
for (int i = 0; i < number; i++)
{
clients[i] = RegisterClientWithImplementationForSend(implementation);
}
}
And then the actual tests:
[Test]
public void RegisterClientWhileBroadcasting_ClientIsAddedAndBroadcastingDidntThrowException()
{
RegisterClientsWithImplementationForSend(5, () => Thread.Sleep(50));
Exception exceptionFromBroadcastThread = null;
var broadcastThread =
new Thread(() => exceptionFromBroadcastThread = GetExceptionThrownBy(() => broadcaster.Broadcast(null)));
broadcastThread.Start();
Thread.Sleep(50);
var newClient = RegisterClientWithEmptySendImplementation();
broadcastThread.Join();
Assert.IsNull(exceptionFromBroadcastThread);
Assert.That(GetBroadcastersClients().Contains(newClient));
}
[Test]
public void ClientFaultedWhileBroadcasting_FaultedClientIsRemovedFromClientsList()
{
RegisterClientsWithImplementationForSend(2, () => { });
var faultyClient = MockRepository.GenerateMock<IClientProxy>();
faultyClient.Stub(c => c.SendNotificationAsynchronously(null))
.IgnoreArguments().WhenCalled(obj => faultyClient.Raise(c => c.Faulted += null, faultyClient, EventArgs.Empty));
RegisterClientWithBroadcaster(faultyClient);
RegisterClientsWithImplementationForSend(2, () => { });
broadcaster.Broadcast(null);
Assert.IsFalse(GetBroadcastersClients().Contains(faultyClient));
}
[Test]
public void ClientFaultedInSeparateThreadWhileBroadcasting_FaultedClientIsRemovedWithoutExceptionDuringBroadcasting()
{
var faultyClient = RegisterClientWithEmptySendImplementation();
RegisterClientsWithImplementationForSend(10, () => Thread.Sleep(50));
Exception exceptionFromBroadcastThread = null;
var broadcastThread =
new Thread(() => exceptionFromBroadcastThread = GetExceptionThrownBy(() => broadcaster.Broadcast(null)));
broadcastThread.Start();
Thread.Sleep(150);
faultyClient.Raise(c => c.Faulted += null, faultyClient, EventArgs.Empty);
broadcastThread.Join();
Assert.IsNull(exceptionFromBroadcastThread);
Assert.IsFalse(GetBroadcastersClients().Contains(faultyClient));
}
Note: i’m not sure if this is actually the best way to test this code… there will probably be better solutions for testing threading issues.
Posted in Software Development | 2 Comments »
Posted by Davy Brion on 21st February 2009
Before i post the tests and a further explanation of the code i listed in the readable code challenge, i wanted to post only the commented version of this code:
public class Broadcaster : IBroadcaster
{
private readonly object monitor = new object();
// This reference will be overwritten with a new instance of a clients List whenever
// we need to add or remove a client from the list. The reason for this is because
// we need to be able to loop through the clients list, but during this loop clients
// might try to register or might be removed from the clients list.
// Using the same instance of the clients List and synchronizing all access with the
// monitor object would not be sufficient because if both the loop and the removal
// would lock on monitor, and the loop and removal would happen on the same
// thread then one of the operations won't block, because that thread already has
// acquired the lock. At that point we get a concurrency exception because clients'
// Enumerator would have been modified while looping through it.
private List<IClientProxy> clients;
private readonly IClientProxyFactory clientProxyFactory;
public Broadcaster(IClientProxyFactory clientProxyFactory)
{
this.clientProxyFactory = clientProxyFactory;
clients = new List<IClientProxy>();
}
public void Register()
{
var client = clientProxyFactory.CreateClientProxyForCurrentContext("Nokeos/IBroadcastOverWcf/Receive");
client.Faulted += Client_Faulted;
AddClientToRegisteredClients(client);
}
private void AddClientToRegisteredClients(IClientProxy client)
{
lock (monitor)
{
// we create a new List instance based on the previous list plus the new client
// and then we assign the new list to the clients reference that the rest of
// this class uses. This happens behind a lock to make sure that the clients
// reference isn't overwritten by the RemoveClientFromRegisteredClients method
// simultaneously
clients = new List<IClientProxy>(clients) { client };
}
}
public void Broadcast(Notification notification)
{
// When we enter the foreach loop, we get a reference to the enumerator of
// the _current_ clients reference (the reference might be overwritten while
// we loop, but our enumerator will never be modified) which means we don't
// need to use a lock here
foreach (var client in clients)
{
// if the send operation fails, the client's Faulted event will be
// triggered which causes removal of the client from our clients list.
// this can happen either while we are in this loop, or afterwards in
// a background thread but in both cases, we don't need to worry about it
// here
client.SendNotificationAsynchronously(notification);
}
}
private void Client_Faulted(object sender, System.EventArgs e)
{
var client = (IClientProxy)sender;
client.Faulted -= Client_Faulted;
RemoveClientFromRegisteredClients(client);
}
private void RemoveClientFromRegisteredClients(IClientProxy client)
{
lock (monitor)
{
// we create a new List instance based on the previous list minus the client
// that needs to be removed and then we assign the new list to the clients
// reference. This happens behind a lock to avoid a simultaneuos overwrite
// of the clients reference by the AddClientToRegisteredClients method
var clientList = new List<IClientProxy>(clients);
clientList.Remove(client);
clients = clientList;
}
}
}
I hope the comments clarify the problem sufficiently. If not, tell me what’s not clear to you because it might mean i need to clarify the comments more. Also, suggestions on how to make this code more readable and reducing the need for comments would be very welcome
I’ll post the tests tomorrow, and then you can all decide what you think is more communicative: the comments, or the tests.
Posted in Software Development | 12 Comments »
Posted by Davy Brion on 19th February 2009
I think this might be an interesting challenge for all of you. The code that will be listed below contains a non-obvious solution. The reason why i went with the non-obvious solution, or what exactly the non-obvious part is, will not be mentioned in this post. What i want to know is: do you guys truly understand this code without any sort of “why i did this” comment in the code? I think the code is very readable, and pretty communicative. So according to many coding purists, it should not contain any comments. I am particularly interested in finding out if these coding purists think that this code should contain a “why i did it like this” comment or not.
Now, it certainly is possible that there is a more communicative way to write the same code. If so, i would be very interested to hear about any possible improvements you can come up with
I will do a follow-up post about this soon. Perhaps tomorrow, or in a few days, depending on how clear or unclear this code turns out to be.
So, this is the code:
public class Broadcaster : IBroadcaster
{
private readonly object monitor = new object();
private List<IClientProxy> clients;
private readonly IClientProxyFactory clientProxyFactory;
public Broadcaster(IClientProxyFactory clientProxyFactory)
{
this.clientProxyFactory = clientProxyFactory;
clients = new List<IClientProxy>();
}
public void Register()
{
var client = clientProxyFactory.CreateClientProxyForCurrentContext("MyCoolNamespace/IBroadcastOverWcf/Receive");
client.Faulted += Client_Faulted;
AddClientToRegisteredClients(client);
}
private void AddClientToRegisteredClients(IClientProxy client)
{
lock (monitor)
{
clients = new List<IClientProxy>(clients) { client };
}
}
public void Broadcast(Notification notification)
{
foreach (var client in clients)
{
client.SendNotificationAsynchronously(notification);
}
}
private void Client_Faulted(object sender, System.EventArgs e)
{
var client = (IClientProxy)sender;
client.Faulted -= Client_Faulted;
RemoveClientFromRegisteredClients(client);
}
private void RemoveClientFromRegisteredClients(IClientProxy client)
{
lock (monitor)
{
var clientList = new List<IClientProxy>(clients);
clientList.Remove(client);
clients = clientList;
}
}
}
Update: you can find the commented version of this code here
Posted in Software Development | 33 Comments »
Posted by Davy Brion on 16th February 2009
I always figured writing a Domain Specific Language (DSL) would be such a dauntingly complex task that the chances of me being able to pull it off would be slim to none. With that in mind, i was pretty interested in reading Ayende Rahien’s Building Domain Specific Languages in Boo to find out just how difficult it would be.
In the first chapter of the book, Ayende mentions his success criteria for the book:
I hope that by the end of this book, you’ll feel that you are able to tackle problems in your
domains and solve them by building Domain Specific Languages to solve those problems in
an elegant manner.
Quite a goal, and i was very interested to see if he would be able to pull it off. Let’s hold off with answering that question for now, and let’s go over the content of the book.
In the first chapter, Ayende basically brings you up to speed on the different kinds of DSL that you can create, covering pro’s and con’s of each approach. Most importantly, he provides some answers as to why you would want to create your own DSL and discusses why he chooses to create DSL’s on top of the Boo language.
Chapter 2 gives a brief overview of the Boo language. I didn’t know Boo yet, so this chapter was very useful and interesting for me. You learn some basics of the language, and you’ll also find out why it’s such a good candidate to build your DSL on top off. The only downside about this chapter is that i had to refrain myself from running off to go play around with Boo. Boo seems very interesting and extremely powerful, and it definitely deserves more exposure within the .NET world.
In the next 2 chapters, Ayende walks us through the implementation of a couple of DSLs (each has a different purpose), while continuously offering a lot of great advice and discussing many of the different decisions you can make while you’re building your DSL, including how you can integrate your DSL with your actual business domain API. This is very nicely done, in a manner that makes you feel like you’re sitting right there with him as he is considering each possible direction to take. After these 2 chapters, you already have a pretty good idea of what exactly is possible to do in your very own language. We’re also introduced to Ayende’s open source Rhino DSL project, which already does some of the heavy lifting of actually making your DSL executable.
Chapter 5 deals with how you can actually integrate your DSL into your application. A DSL can be very nice, but if it doesn’t integrate nicely with your application, it’s pretty useless isn’t it? That’s what this chapter is for. Topics like storing of the DSL scripts, order of execution, performance implications, security considerations, dealing with errors and administration of scripts in general are all covered pretty nicely here.
Up until this point, most of the stuff in this book has been pretty easy to grasp. Chapter 6 however might occasionally leave your head spinning. This is where we get into making sure the compiler behaves the way we want it to. Think about that. As developers, we’re used to making sure that we comply with what the compiler expects from us. Now we actually get to turn this around. This chapter is by far the most difficult one in the book, but it is definitely important that you take the time to understand everything this chapter covers. It’s basically about extending the capabilities of your language (instructing the Boo compiler on how to compile your DSL, introducing new keywords, or even modifying the behavior of existing language keywords) to provide the best possible experience for the consumers of your DSL. Again, some parts might leave your head spinning. Read them again and again until you get them because it’s not only very important to understand, it really is fascinating stuff.
The next chapter deals with the infrastructure you need to support your DSL, based on Rhino DSL. You don’t need to use Rhino DSL, but it certainly makes a lot of the tasks involved easier to handle.
In chapter 8, Ayende covers the importance of testing your DSL, and how you can do it. It not only covers how to create a testable DSL, it also shows you can extend the xUnit.NET testing framework to make it easier to write unit tests for your DSL scripts.
By now, you already know a lot about how to build a high quality DSL and how to integrate it with your application. But there is more to it than just creating a fancy DSL and making it work. You need to make sure that the DSL is maintainable in the long term, and that it is easily accessible and comprehensible for the intended users of the language. The next 3 chapters deal with all of this. Topics covered are multiple versioning strategies, properly documenting your DSL, and even creating a professional UI on top of your DSL. A UI for a DSL you ask? Well, depending on the intended users, you might want to create a nice GUI on top of it, or you might want to create a bit of an IDE experience (complete with syntax highlighting and code completion). All of these possibilities are also covered nicely in this book. After that, Ayende covers a bunch of typical DSL implementation patterns that you’ll probably find very useful when creating your own DSL.
For those of you who are familiar with Ayende’s blog, you’ll be happy to know that the writing style is very similar. It’s very engaging, it keeps you interested in the topic at hand throughout the book, and he often makes you marble at his ability to come up with simple and elegant solutions to difficult problems. I used to be skeptical towards the actual value of creating your own DSL, but now i know that it is not only doable, there really are a lot of cases where it makes sense to create a DSL, more than i’d ever imagined.
I think Ayende achieved his goal. I do think that i can now create a DSL, and i’m sure many of you would agree with me after reading this book. In most cases, it’s pretty easy with this book by your side. And for those cases where it gets hard, this book thoroughly covers all of your options. Do yourself a favor and read this book, you won’t only learn a lot about DSL’s, but also about problem solving in general.
Posted in Books | 1 Comment »
Posted by Davy Brion on 15th February 2009
A lot of developers deal with performance differently. Some care too much, trying to optimize every single piece of code they write. Some care too little, not thinking about performance at all until it is proven to be a problem. I think both of these approaches suck and usually try to find a healthy balance between them.
Below is a list of rules of thumb that i always try to keep in mind. Keep in mind that these are just rules of thumb… they are not rules that never should be broken, nor are they applicable to each and every situation. In general though, i do believe that if you keep these in mind you can avoid serious performance issues and the need to do heavy architectural refactoring in the long run. So without further ado, these are the performance-related things that i do care about while i’m writing my code:
And that’s pretty much it… outside of the stuff listed above, i typically don’t care about performance. I try to keep my code clean and focused, and rely on profiling to identify performance hotspots. When the profiler identifies problems, they are often more easy to solve if you’ve kept your code clean than it would be if you had tried to prematurely optimize in the wrong places.
Posted in Performance | 10 Comments »
Posted by Davy Brion on 10th February 2009
You gotta do Agile Development. No wait, these days you gotta do Lean. Oh, and don’t forget to do Test Driven Development (TDD). No wait, these days you gotta do Behavior Driven Development (BDD). Oh wow, you’re not doing Domain Driven Design (DDD)? Dude, you suck!
If someone asked me to summarize the majority of the content that the software development community has put forward in the last couple of weeks/months, both on blogs and mailinglists or any other medium that enables online discussion, the paragraph above would be it. Sad, isn’t it? I often get the impression that a large part of this community has simply lost the ability to put things into perspective.
Each and every one of the popular development styles are often touted to solve all of your development-related problems. Well, to be fair, not all of the proponents of these development styles will make these claims. But a large majority of them will. Yet, when i read these people’s statements, i can’t help but wonder if their inability to put things into perspective is the reason why they’re having so many development-related problems in the first place. Blindly following a certain development style and going to far with it without truly understanding the pro’s and con’s is a recipe for disaster. I don’t think it’s too far fetched to claim that this industry has about 40 years of historical data to back up that statement.
The thing about all of these styles is that they all have some merit (some arguably more than others), but none of them is a silver bullet. Not one of them will truly solve all of the problems you might face in software development. You know what does help in most cases? A little bit of common sense. You don’t really need to follow everything the ‘cool kids’ in the software development community say. Half of them aren’t truthful and another 30% (give or take) don’t really know what the hell they’re talking about and just want be considered ‘cool’ as well. It’s kinda like high school really.
Are you trying to improve the quality of the software you deliver? By all means, investigate each and every one of the popular approaches. Start applying parts of them that may start improving your situation in the short term. Take small steps. Don’t go overboard. Keep doing this as you gradually try to remove whatever it is that is making things more difficult than they need to be. In the end, you might end up with a mix of various pieces and parts of various development styles. And you know what? That’s ok. There’s nothing wrong with that. Nobody ever said that you have to follow a certain development style to the letter if you really want to solve your problems. And if the approach i just outlined solved your problems (or parts of them), then really, what is the problem?
Posted in Opinions | 5 Comments »
Posted by Davy Brion on 9th February 2009
The purpose of this post is just to quickly go over what you need to do to get NHibernate’s 2nd Level Cache working in your application. If you want to read how the 1st and 2nd Level Caches work, please read Gabriel Schenker’s excellent and thorough post about it.
Anyways, the first thing you need to do, is to enable the 2nd level cache. Add the following 2 properties to your hibernate.cfg.xml file:
<property name="cache.use_second_level_cache">true</property>
<property name="cache.use_query_cache" >true</property>
The first one (obviously) enables the 2nd level cache, while the second one enables query caching. That basically means that you can (optionally) cache the results of specific queries. Note that this doesn’t mean that the results of all queries will be cached, only the ones where you specify that the results can be cached.
Next, you need to choose a CacheProvider. There are various options available, although i generally just use SysCache (which makes use of the ASP.NET Cache). You can download the CacheProviders here.
Once you’ve picked out a CacheProvider, you need to add a property for it to your hibernate.cfg.xml file as well:
<property name="cache.provider_class">NHibernate.Caches.SysCache.SysCacheProvider, NHibernate.Caches.SysCache</property>
Let’s first start with caching the results of a query. Suppose we have the following query:
var products = session.CreateCriteria(typeof(Product))
.Add(Restrictions.Eq("Category.Id", categoryId))
.List<Product>();
If we want NHibernate to cache the results of this query, we can make that happen like this:
var products = session.CreateCriteria(typeof(Product))
.SetCacheable(true)
.Add(Restrictions.Eq("Category.Id", categoryId))
.List<Product>();
When we execute this query, NHibernate will cache the results of this query. It is very important to know that it won’t actually cache all of the values of each row. Instead, when the results of queries are cached, only the identifiers of the returned rows are cached.
So what happens when we execute the query the first time with categoryId containing the value 1? It sends the correct SQL statement to the database, creates all of the entities, but it only stores the identifiers of those entities in the cache. The second time you execute this query with categoryId containing the value 1, it will retrieve the previously cached identifiers but then it will go to the database to fetch each row that corresponds with the cached identifiers.
Obviously, this is bad. What good is caching if it’s actually making us go to the database more often than without caching? That is where entity caching comes in. In this case, our query returns Product entities, but because the Product entity hasn’t been configured for caching, only the identifiers are cached. If we enable caching for Product entities, the resulting identifiers of the query will be cached, as well as the actual entities. In this case, the second time this query is executed with a categoryId with value 1, we won’t hit the database at all because both the resulting identifiers as well as the entities are stored in the cache.
To enable caching on the entity level, add the following property right below the class definition in the Product.hbm.xml file:
<class name="Product" table="Products">
<cache usage="read-write"/>
This tells NHibernate to store the data of Product entities in the 2nd level cache, and that any updates that we make to Product entities need to be synchronized in both the database and the cache.
That’s pretty much all you need to do to get the 2nd Level Cache working. But please keep in mind that there is a lot more to caching than what i showed in this post. Reading Gabriel’s post on caching is an absolute must IMO. Caching is a powerful feature, but with great power comes great responsibility. Learn how to use it wisely
Posted in NHibernate | 9 Comments »