The Inquisitive Coder – Davy Brion’s Blog

Trying to walk that thin line between intelligence and ignorance

Save Some Time With The Right Testrunner

Posted by Davy Brion on July 1st, 2009

If you’re running your tests multiple times throughout the day, it’s in your best interest to keep those testruns as fast as possible. We have a solution with 2 test projects. The first testproject contains 2700 regular tests that don’t use any external resources so they should be very fast. The second testproject contains 1200 database tests (you do write tests for your queries, right?) which is a little bit slower, as is to be expected.

The problem is that with Resharper’s testrunner, running the database tests actually takes less time than running the ‘fast’ tests. The 1200 database tests take 1m14 seconds to run. The 2700 in memory tests take 1m20 seconds to run… so that’s almost 3 minutes of waiting time whenever you run the entire testsuite.

Just out of curiosity, i ran all of the tests with TestDriven.NET to see how much faster it would be… The 1200 database tests only took about 45 seconds, and the 2700 in memory tests only took 28 seconds to run. Running the entire testsuite with TestDriven.NET actually takes less than half the time it takes with Resharper’s testrunner…

  • Share/Save/Bookmark

Posted in Test Driven Development | 3 Comments »

Avoid Using NHibernate With NUnit 2.4.6

Posted by Davy Brion on June 24th, 2009

We just spent about 2 hours trying to find out why our NHibernate tests were about 10x slower on our build server than they were on our local machines. I had noticed lately that the build for one of our projects was taking longer and longer but i hadn’t really timed the difference. This project has about 1200 tests that use NHibernate and they run in about 45-60 seconds on my local machine. It turns out they took around 15 minutes on the buildserver when running them through TeamCity.

I logged into the buildserver and ran the tests manually using nunit’s console runner (with an NUnit-2.4.7 build that i happened to have installed somewhere on the machine) and they only took about 45 seconds. After a lot of guesswork and screwing around, it turned out that we never modified our base build script (why yes, i do believe in build script inheritance) to use a newer version of NUnit. We set up the buildserver about 1 year ago, and at that time, the latest stable NUnit version that TeamCity supported was NUnit 2.4.6. Our base build script was still referring to NUnit 2.4.6, which apparently sets log4net to use debug level logging. Now, NHibernate logs a huge amount of information at the debug level, so this turned out to slow down all of our builds that had NHibernate tests.

We changed the the 2.4.6 version in our script to 2.4.7 and the build time of this particular project decreased from around 50 minutes to about 35 minutes. Yes, that’s still a lot but this is a huge project with a lot of legacy tests and the entire build process is pretty complex. Other projects went from build times from around 7 minutes to about 2 minutes.

That’s a pretty nice improvement for simply changing a “6″ to a “7″ ;)

  • Share/Save/Bookmark

Posted in NHibernate, Performance, Test Driven Development | 3 Comments »

Surface Development For Belgian TV

Posted by Davy Brion on June 22nd, 2009

Some of my coworkers recently developed 2 Surface applications for VTM, one of the commercial Belgian TV channels. This is one of the few high-profile usages of Microsoft’s Surface in Belgium so far. Be sure to check out the videos on our company website.

  • Share/Save/Bookmark

Posted in Software Development | No Comments »

And We’re Back

Posted by Davy Brion on June 22nd, 2009

I got back from the Dominican Republic yesterday morning so i just wanted to post a short ‘review’ of the trip. Getting there took a looong time, because our plane first stopped in Cuba where we had to wait for another hour before we flew into Punta Cana. Unfortunately, we weren’t allowed to get off the plane for a short time which kinda sucks after 10 hours of flying. The little bit of Cuba i did get to see (through the window) did look pretty nice though. So after 13 hours we finally landed at Punta Cana’s airport and then we had to take a 2 hour bus ride to get to our hotel.

The hotel was great. Big, nice rooms. Lots of restaurants (4 plus a general buffet as well). Great pool, including a nice bar. Fantastic beach, also with a bar obviously. Plenty of activities and a few shops as well. If you really want to, you could have a pretty good time without even leaving the resort.

But that would be a bit of a shame because the country has so much more to offer than beautiful resorts. I went on two excursions while i was there. The first one was a snorkeling trip which was great. It was basically a whole day on a boat going along the Caribbean coastline, which in itself is already a sight to behold. Then there were 2 snorkeling sessions, one before noon, and one in the afternoon. For lunch, we went to the Saona island which you may know better as the Bounty Island (used in the commercials) to have a barbecue plus some relaxing on the beach. We also spent some time in a ‘natural swimming pool’ which is located within the sea on a huge sandbank (which was the result of a hurricane a few years back, as i’ve been told). It really is mindblowing to stand up in the middle of the ocean, with the water only reaching up to your waist.

The second excursion was a truck safari from the southern part of the island to the north side. I’d definitely recommend doing this because the sights you see along the way are unbelievably beautiful. And you don’t just spend the whole time in the truck either. We stopped along a sugar cane field and tried real sugar from a sugar cane. We also visited some people in the country side where we tried fresh coffee (from beans that had finished roasting in the sun just a few days ago), fresh cocoa (both in powder form after non-industrial processing as well as directly from the fruit), fresh Creole pine apple, coconuts, Mama Juana and other things as well. We had lunch on some ranch in the mountains (seeing large palm trees on top of high mountains is pretty cool as well) and after that we did some horseback riding. We also visited a tiny school (with only 2 classrooms) and a big church in Higuey which is apparently pretty famous. In the afternoon we stopped along a beach in the north side of the island (with the Atlantic ocean instead of the Carribean sea). We definitely saw a lot of beautiful things during this excursion. Funny side story: some British tourists that were also on the truck brought notepads and pens for the poor children we’d meet along the way. During one of our walks, they gave them to two kids who were sitting in front of their house. What these tourists didn’t realize was that those kids were sitting in front of a very nice house which actually had a beautiful pool in the backyard. Good thing these poor kids now also have notepads and pens ;)

Combining these excursions with a lot of relaxing at the beach definitely makes for a great trip. I’m not really a going-to-the-beach-kinda-guy but the Bahayibe beach and the Carribean Sea are just too nice to pass up on. And the water is just incredible… it’s crystal clear, it’s very warm, and occasionally you’ll see a bunch of colorful fish swim by or even around you.

The one downside to the trip was that there was hardly anything to do at night where we were. The northern side apparently has a lot more going on at night (it’s only normal with all the mass-tourism over there) but on the southern side there’s really not much to do outside of what’s going on in your resort. I was there with my best friend so we were hoping for some interesting bars to go to at night, but we mostly stuck to the lobby bar in the resort for a lack of anything better. You could just go to local towns and visit the bars over there but that’s not really recommended unless you’re with a reasonably-sized group to avoid trouble.

But apart from that, everything was great and i’d definitely recommend it to anyone. And after all of that, we spent another 9 hours on a plane to land in Brussels with pouring rain and pretty chilly weather *Sigh*.

I still have to go through a couple hundred blog posts that i missed, a shitload of email and some other stuff i have to do as well. I’ll start posting and coding again in the next couple of days though ;)

  • Share/Save/Bookmark

Posted in Off Topic | 1 Comment »

Time For A Much Needed Break

Posted by Davy Brion on June 7th, 2009

As you have probably noticed, i haven’t been very active lately. Mostly because of some situations in my personal life lately which took up a lot of my spare time. Because of this, i haven’t been posting as much as i used to, nor did i contribute anything to NHibernate in the past couple of weeks. Luckily for me, i’m leaving for a trip to the Dominican Republic in a couple of days where i intend to explore the limits of the concept “relaxation”. As much as i love software development and computers, i’m really looking forward to not even touching a computer (or any electronic device for that matter) for about 10 days :) . Who could think of computers when you’re hanging out a place that looks like this:

paradise

I’ll see you all in about 2 weeks :)

  • Share/Save/Bookmark

Posted in Off Topic | 10 Comments »

How Can This Test Fail?

Posted by Davy Brion on June 3rd, 2009

We have a certain class which sometimes makes some tests fail… the code is old (consider it legacy) but it’s not that bad… but we just can’t figure out why it sometimes causes tests the fail.

Here’s an example of a test that occasionally fails:

        [Test]

        public void TestGet_CacheableObjectReturned()

        {

            TestCacheableObject cacheableObject = new TestCacheableObject("test", 60000);

 

            CacheProvider.GetInstance().Put(cacheableObject);

 

            try

            {

                Assert.AreSame(cacheableObject, CacheProvider.GetInstance().Get("test"));

            }

            finally

            {

                CacheProvider.GetInstance().Remove("test");

            }

        }

If it fails, it’s on the following line:

                Assert.AreSame(cacheableObject, CacheProvider.GetInstance().Get("test"));

Sometimes, the Get method of the CacheProvider returns null in this test, even though the cacheable object has just been added to the CacheProvider.

Here’s the code of the CacheProvider:

    public class CacheProvider

    {

        private static object monitor = new object();

 

        private static CacheProvider _instance;

 

        private IDictionary<object, CacheableObject> _cacheDictionary;

 

        private CacheProvider()

        {

            _cacheDictionary = new Dictionary<object, CacheableObject>();

        }

 

        public static CacheProvider GetInstance()

        {

            lock (monitor)

            {

                if (_instance == null)

                {

                    _instance = new CacheProvider();

                }

            }

 

            return _instance;

        }

 

        public void Put(CacheableObject value)

        {

            if (ContainsKey(value.Key))

            {

                Remove(value.Key);

            }

 

            value.Provider = this;

            _cacheDictionary.Add(value.Key, value);

            value.StartTicking();

        }

 

        public CacheableObject Get(object key)

        {

            if (_cacheDictionary.ContainsKey(key))

            {

                return _cacheDictionary[key];

            }

 

            return null;

        }

 

        public void Remove(object key)

        {

            _cacheDictionary.Remove(key);

        }

 

        public bool ContainsKey(object key)

        {

            return _cacheDictionary.ContainsKey(key);

        }

    }

This is the code of the CacheableObject class:

    public abstract class CacheableObject

    {

        private object _key;

        private CacheProvider _cacheProvider;

 

        private Timer _timer;

 

        protected abstract double GetTimeToLive();

 

        public CacheableObject(object key)

        {

            _key = key;

        }

 

        public void StartTicking()

        {

            _timer = new Timer(GetTimeToLive());

            _timer.Elapsed += Timer_Elapsed;

            _timer.Start();

        }

 

        private void Timer_Elapsed(object sender, ElapsedEventArgs e)

        {

            _timer.Stop();

            Provider.Remove(this.Key);

        }

 

        public object Key

        {

            get

            {

                return _key;

            }

        }

 

        public CacheProvider Provider

        {

            get

            {

                return _cacheProvider;

            }

            set

            {

                _cacheProvider = value;

            }

        }

    }

And this is the code of the TestCacheableObject that is used in the failing test:

        public class TestCacheableObject : CacheableObject

        {

            private double _ttl;

 

            public TestCacheableObject(string key, double ttl)

                : base(key)

            {

                _ttl = ttl;

            }

 

            protected override double GetTimeToLive()

            {

                return _ttl;

            }

        }

I can’t for the life of me figure out why the test above fails sometimes. It happens very occassionaly, sometimes it even takes weeks or months before it fails again. It’s not just the test above, it’s basically every test that uses the CacheProvider’s ContainsKey method (either directly or indirectly) that fails sometimes.

Important note to consider: when these tests fail, they take about 10ms…

If anyone has a clue what could possibly cause this, i’d be very willing to hear it :)

  • Share/Save/Bookmark

Posted in Test Driven Development | 24 Comments »

What Microsoft Should Do For .NET Open Source

Posted by Davy Brion on May 25th, 2009

I recently read Rob Conery’s “What Should Microsoft Do For .NET Open Source” post where he invites people to answer the question he poses in his post. Obviously, i could not resist posting my thoughts and views on this.

Before i start, i would like to state that i appreciate all of Microsoft’s recent efforts to move to a more open development model. I think they’ve done a pretty good job already but nothing’s perfect and there’s always room for improvement. So let’s just get to the list of things that i would love to see from Microsoft with regards to dealing with Open Source Software in the .NET world.

First of all, i’d like to see a completely transparent development model. I love the fact that ASP.NET MVC is released under an Open Source License, but it would be better if everyone could track the development. With that i mean access to the internal bug/issue tracker and source code repository. I want to know what problems are known, and how they are being dealt with. I want to see the progress in the code. I want the ability to either run off the trunk, or to merge specific pieces of the trunk into a released version if that enables me to avoid problems that i might run into with an officially released version. Getting the source code to a released version is (IMO) only a small part of what Open Source development is all about.

I also want them to be open to outside contributions. I truly hate running into annoying little bugs that won’t be fixed until the next major version (which sometimes takes a year or 2 to come out) but which could be avoided with a simple patch. Allow people to contribute patches and you’d be amazed at the amount of goodwill and support you’ll create among a vocal group of users. Obviously, this only works when the development model in general is as open as i mentioned earlier. Nobody is going to submit patches based on a released version if the actual trunk might already contain fixes or has been modified heavily after the latest release. And being able to patch a released version for your own use isn’t always a viable option to most people since you really don’t want to deal with having to merge the patch when newer versions are released (and all the regression testing that comes with it).

Microsoft should also play nice with other Open Source projects. Sure, they included JQuery with ASP.NET MVC which is a good move. But there are a lot of examples where they developed some library or framework for which a generally accepted Open Source alternative was already present. I still cringe whenever i’m forced to use MSTest (which to this day is still a pretty crappy testing framework IMHO) while they could’ve just as easily supported NUnit’s development. And if you’re not happy with NUnit, consider the state of MSTest before you complain about my choice of NUnit over say… MBUnit or any of the other testing frameworks. Does anyone remember NDoc? Was Sandcastle really necessary or would it have made more sense to just invest some effort into NDoc instead of starting from scratch? My biggest issue with this is that Microsoft’s alternatives can often be used without resistance in certain workplaces, just because it’s from Microsoft. It doesn’t even matter if there are actually better Open Source alternatives available… In Microsoft we trust, thus Microsoft you’ll use. Not always the smartest choice, as i’m sure many of you will agree with.

Would it really be so bad for Microsoft if they would just assign some developers to help out with projects that already exist? We now generally have to wait until the third version of any Microsoft product before we really get great quality, while in some cases, a high-quality Open Source alternative was already available from the start. This often leads to about 3 to 4 years of crappy releases that many people will adopt solely because it’s from Microsoft and because “it’ll improve in the next versions”. Even if you don’t want to assign developers to Open Source projects, it might already be very useful to provide helpful things such as servers, bandwidth, anything that could help out. Hell, references in official Microsoft documentation on MSDN would already be a huge step up.

There also needs to be some kind of PR cleanup campaign for all of the FUD that Microsoft has spread about Open Source software in general in the past few years. While it can be argued that that FUD was mainly targeted at GPL software, to many managers in big companies the message came across as “Do not use Open Source because it’s a legal minefield”. That is a misconception that many .NET developers are still fighting on a daily basis which isn’t good for anyone.

Then there is the issue of support. I’m not going to repeat myself so i’ll just link to a previous post of mine which covers that topic: Support Of Commercial Software vs Open Source Software.

And as for the host of legal aspects that Microsoft is probably afraid of, i would suggest looking into the reasons why other (large) companies are able to be cooperative citizens within the Open Source world without legal troubles. Hell, commercial software can be a legal minefield as well so is the Open Source world truly worse? Maybe they should ask companies like IBM… surely they have plenty of experience in both cases.

Do i expect or even want Microsoft to open source everything in the .NET world? No. But i would definitely appreciate it if Microsoft would play nice with other Open Source projects, be a more active participant and be open to outside participation as well. I’m sure i won’t be the only one who thinks so.

  • Share/Save/Bookmark

Posted in Opinions | 7 Comments »

Comment Spam

Posted by Davy Brion on May 24th, 2009

I used to go over the comments that were marked as spam on a regular basis to check for false positives. This was pretty doable in the past since i got between 30-50 spam comments on a daily basis and you could usually scroll through them pretty quickly to identify real comments. Lately though, i’ve been getting several hundred of spam comments per day and i really don’t have the time to go through all of that. Unlike some other bloggers, i refuse to disable commenting in general due to spam so some legit comments might accidentally be flagged as spam and won’t show up. So, if you leave a comment and it’s not showing up immediately, just send me an email (preferably from the same address you filled in for the comment) and i’ll try to rescue it from the spam filter. I do delete spammed comments pretty regularly so don’t wait around if your comment isn’t displayed immediately :)

  • Share/Save/Bookmark

Posted in About The Blog | 2 Comments »

Continuous Integration On A Real, Big Project

Posted by Davy Brion on May 24th, 2009

Some of you may remember a post of mine where i showed the complete lack of attention that was being paid to the Continuous Integration of one of our projects. This particular project is pretty big. The development of this project has gone on for years, and will keep going for years to come as it is a strategically important project for us. The actual system is used internally by our company as well as a growing list of customers.

There are essentially 2 big problems with this project. One is a mountain of legacy code with so much technical debt (due to the evils of code generation) it sorta resembles the current economic recession (as in: it’ll take a long time to get everything sorted out). The other problem is more a matter of organization. We’re a pretty small company and while we have great ideas for products, we simply can’t assign a steady, stable team of developers to work on any of these projects on a continuous basis since we all often need to work on stuff that is simply more lucrative at that particular point in time. The result is that this particular project typically has an ever rotating group of developers working on it, usually for short periods of time. Some people work almost full-time on it, but that list is pretty limited.

Not exactly the ideal situation for a large project to apply CI and other agile development practices to, right? Luckily for us, we’re quite stubborn and we try to make the best out of every situation. So back when this project’s CI success rate was only showing a lack of success, we all agreed to follow the CI rules and at this point, i’m pretty proud to be able to show you guys the following picture:

image001

Note: we moved to Team City in July 2008 so i can’t show you any earlier data than that.

Anyways, as you can see, after the dismal months of February and March, the success rate of the CI build gradually started improving again. The average time to fix failing tests also decreased sharply. We still have failing tests from time to time, but at least now they’re all dealt with in a timely fashion. We still have build failures, though those have decreased a lot as well and are always dealt with pretty soon now. I’m not sure if this is because of my ‘Continuous Bitching’ whenever the build fails, or that everyone bought into the concept of CI again (for my own sake, i’ll just assume that it’s the latter instead of the former) but i’m pretty happy with the results that we’re getting.

Also, take a look at the number of tests (we’re at 16000+ now) and the duration of the build. The build time is about 48 minutes on average now, which is obviously way above the recommended 10 minutes. I’d love to see this go down to about 20 minutes (which i’d find very acceptable considering the size of the project) but that’s gonna take a long while. Of those 16000 tests, there are about 13000 tests that cover the legacy code and they all use the database. And since those 13000 tests use a generated data layer, we can’t just let it run on an in-memory database nor can we mock the database in those tests because all of that generated code, and pretty much everything that was written on top of it, is coupled more tightly than Siamese Twins. We also lose a couple of minutes of build time due to our Genesis processing but that is simply something that we can’t go without anymore so we don’t really mind the extra build time of that part.

So there you have it… the reason i wanted to post this is because when the topic of CI comes up, you always read about ‘instant feedback’ and really quick builds and things like that. It’s simply not always like that in the real world. But with a bit of effort and focus, you can get many of the benefits that are usually attributed to CI, even on huge projects with lots of legacy code.

  • Share/Save/Bookmark

Posted in Continuous Integration | 7 Comments »

Keep An Eye On Those Indexes

Posted by Davy Brion on May 21st, 2009

We have a multi-tenant application, where each tenant has its own database. We recently were informed about a particular performance problem that one tenant (which we’ll refer to as Tenant A) was experiencing in every screen where data of a certain type needed to be shown. None of the other tenants experienced this problem though.

We tracked down the query that was causing the bad performance and ran it on the database of Tenant B. Tenant B actually had a lot more data in the main table that was used in the query and the query executed immediately whereas it took about 25 seconds to complete for Tenant A. So the query runs fast on another database that actually has more data… at this point i was convinced that it had to be related to indexes.

Turns out that someone recently ran an import process to import a bunch of data in Tenant A’s database. I know very little about databases, but one thing i’ve seen time and time again (with both Oracle and SQL Server) is that you really need to make sure that your indexes are in good shape after any process that performs a lot of inserts (or removals). A couple of years ago, i had a very intensive nightly import process for a particular project that used an Oracle database. As time went on, the application’s queries became painfully (unacceptably even) slow. I managed to restore the performance of those queries by simply instructing Oracle to recalculate all of the statistics of the indexes of tables that were affected heavily during the nightly import.

With that in mind, we simply rebuilt the indexes for Tenant A’s database, and the same query that took 25 seconds completed almost instantly from then on. Now, we did had a weekly job running on that database server to keep the indexes in a healthy shape but that job didn’t really do a good umm… job of it, apparently.

Lessons learned: make sure that you:

  • Have a proper maintenance job set up which keeps your indexes healthy and schedule it to run regularly
  • Run that job manually if you need to perform a manual import process
  • Execute that job in an automated fashion whenever an intensive automated import process has completed

Oh, and consult with your DBA’s or at least people who know what they’re doing when it comes to your particular database on how to keep those indexes healthy. In this case, we rebuilt them. In other cases it’s sufficient to recalculate the statistics… i’m not sure which way is the best but you should at least keep an eye on this possible problem :)

  • Share/Save/Bookmark

Posted in Performance | 1 Comment »