The Inquisitive Coder - Davy Brion’s Blog

Trying to walk that thin line between intelligence and ignorance

Archive for November, 2008

Genesis: Bridging The Gap Between Requirements And Code

Posted by Davy Brion on 28th November 2008

I just read the following post by Robert C. Martin. In it, he says the following:

If you can enumerate the states, and events, then you know the number of paths through the system. So if Given/When/Then statements are truly nothing more than state transitions, all we need to do is enumerate the number of GIVENs and the number of WHENs. The number of scenarios will simply be the product of the two.

To my knowledge, (which is clearly inadequate) this is not something we’ve ever tried before at the level of a business requirements document. But even if we have, the BDD mindset may make it easier to apply. Indeed, if we can formally enumerate all the Givens and Whens, then a tool could determine whether our requirements document has executed every path, and could find those paths that we had missed.

The company i work for (Item Solutions) actually has a tool for this: Genesis. After reading Robert’s post and discussing it here at the office, we decided to introduce this tool to a wider audience.

Genesis is more than just a tool. It is a methodology that we’ve been using for our internal projects for over two years now and that we are starting to commercialize. Before i start explaining how it all works, i’d like to show you how we can monitor the coverage of the requirements:

This is the global overview of one of our projects where we use the Genesis methodology. Each circle represents a functional module of the application we’re working on. Some of these circles are entirely green, which means that the requirements of that functional module are completely covered. Some of the circles still have a red part in them. This shows us that some of the requirements for that part have not been implemented yet.

The Genesis web application allows you to click on each circle to drill down into that functional module. In the screenshot, you can see that the circle for the Project Management module still has a red part in it. As we drill down into the module and the parts it consists of, we can get to the following view:

As we can see here, we have a couple of use cases within this part of the application that are entirely green. We also have two that are entirely red, which means we haven’t done anything for those use cases yet. And then we have a couple which are mainly green, but still with a bit of red in it. If i click on one of those use cases, this is what shows up on my screen:

As you can see, there are a couple of business requirements which are green, and below them we list the automated tests which cover this specific requirement. You can also see a red requirement, and the tool tells us that are no tests that cover that specific requirement.

That’s just a small overview of how we can monitor the development progress of our projects (there’s actually a lot more that we can see, but i might cover that in future posts about Genesis).

By now, you’re probably wondering: so how does it all work? Obviously, i can’t give away too many implementation details, but i can tell you how we use the tool and the methodology.

It all starts with a requirements document obviously. The requirements of the use case i showed earlier are written down like this:

Every time our continuous integration build runs, Genesis analyzes all of the test results and all of the data in our requirements document. In the Genesis tool, we can click on each business requirement to generate a specific code attribute for that requirement. When we write our tests, we put those attributes on top of the test methods. When Genesis analyzes the results and requirements, it processes the links between the requirements and the tests. If all of the tests that are linked to a requirement succeed, then the requirement shows up in green. If there are no tests for a requirement, or if existing tests for a requirement fail, the requirement shows up in red with some information about the problem. Genesis will either tell us that the tests for that requirement are no longer working, or that they are missing. Not only that, it will also notify us if the requirement changed since the test was written.

All of this gives us a tremendous amount of information and feedback, which is easily consulted by developers, analysts, project managers, clients, and pretty much everyone who’s been given access to the data. It’s also possible for a developer to ‘comment’ on the requirements… those comments will also be shown by the tool so the analyst and/or the project lead immediately know that something might be wrong with the requirements. This actually enables an effective line of communication when team members aren’t sitting together in the same physical location. We actually use this methodology with remote team members as well, and we’ve had less communication-related delays ever since we started doing this.

Finally, i would like to stress that this methodology can be a supplement to your existing process, instead of an “all-or-nothing” approach. It imposes no restrictions on the way you write your automated tests. You can place the requirement attributes on your unit tests, on integration tests, or whatever other kind of tests you write. All we need is data from the test results, which we then match to the data in the requirements document.

Also, i’d like to note that we currently only support .NET and Java but the system is open for extension so we can add support for other development environments as well.

Finally, you can get some more information about Genesis here.

Share/Save/Bookmark

Posted in Software Development, Test Driven Development | 5 Comments »

ORM Is NOT Inherently Evil

Posted by Davy Brion on 25th November 2008

There were some comments on a previous post about how the problem described in that post is somehow typical for applications that use ORMs. I really could not disagree more.

It seems that even today, in 2008 mind you, we still have a lot of people who are convinced that ORM usage can never be as efficient as the more classic data access approaches. Now, i don’t even want to get into the whole debate about where business logic belongs (but if you think it belongs in the database you no doubt have better things to do than reading this blog), but one thing that does bother me tremendously is that a lot of people discard ORMs because they simply don’t know how to use it properly.

An ORM is a tool. Nothing more, nothing less. Well, it is a pretty powerful tool and, as with any other powerful tool, improper usage of said tool can really cause a lot of problems. Should we discard the tool because a lot of people never took the time to figure out how to use it properly? That would be kinda stupid, no?

It seems to me that a lot of people seem to have this misconception that using an ORM essentially leads to data-fetching in tremendously inefficient manners. They see the tutorials where only the ‘get-by-id’ functionality and the lazy loading features are shown and they somehow think that’s all there is to it. They hear all the horror stories about projects that performed terribly because the developers used an ORM and they blame the tool, not the developers. Nevermind the fact that there are plenty of projects that use more classic data access approaches who perform like shit as well.

So let’s try to get a few of these misconceptions out of the way, shall we?

  • You can create highly efficient queries with an ORM tool, and you can actually do so in a manner which enables high developer productivity
  • ORM’s are not slow by definition. Using them wrong (just like with any other data access technology) can be tremendously slow however. Who’s at fault?
  • ORM’s do not use a shitload of memory. Improper usage of them however can lead to excessive memory usage. Blame the developer, not the tool.
  • ORM’s do not lead to lazy developers, who are doing lazy coding by relying on lazy loading. Bad developers lead to lazy coding by relying on lazy loading.

So, for those who think that ORM’s can never work ‘right’, i have only one question: are you absolutely sure you know what you’re talking about?

Share/Save/Bookmark

Posted in Opinions, Rants | 8 Comments »

Populating Entities With Associations From Stored Procedures With NHibernate

Posted by Davy Brion on 24th November 2008

In response to my last post where i showed how you could fill entities with the resultset of a stored procedure, i was asked if it was also possible to fill entities and their associations if the stored procedure returned all of the necessary data. I looked into it, and it’s possible, although it did take me some time to figure out how to actually do it.

First of all, here’s the modified stored procedure:

ALTER PROCEDURE [dbo].[GetProductsByCategoryId]

    @CategoryId int

AS

BEGIN

    SET NOCOUNT ON;

 

    SELECT [Products].[ProductID] as "Product.ProductID"

          ,[Products].[ProductName] as "Product.ProductName"

          ,[Products].[SupplierID] as "Product.SupplierID"

          ,[Products].[CategoryID] as "Product.CategoryID"

          ,[Products].[QuantityPerUnit] as "Product.QuantityPerUnit"

          ,[Products].[UnitPrice] as "Product.UnitPrice"

          ,[Products].[UnitsInStock] as "Product.UnitsInStock"

          ,[Products].[UnitsOnOrder] as "Product.UnitsOnOrder"

          ,[Products].[ReorderLevel] as "Product.ReorderLevel"

          ,[Products].[Discontinued] as "Product.Discontinued"

          ,[Categories].[CategoryID] as "Category.CategoryID"

          ,[Categories].[CategoryName] as "Category.CategoryName"

          ,[Categories].[Description] as "Category.Description"

      FROM [Northwind].[dbo].[Products]

            inner join [Northwind].[dbo].[Categories]

                on [Products].[CategoryID] = [Categories].[CategoryID]

     WHERE [Products].[CategoryId] = @CategoryId

END

As you can see, this returns all of the columns of the Products table, as well as the columns of the Categories table. The goal is to let NHibernate execute this stored procedure, and use the returning data to give us a list of Product entities with a Category reference which is already set up with the proper data.

The mapping of the named query now looks like this:

  <sql-query name="GetProductsByCategoryId">

    <return alias="Product" class="Product">

      <return-property column="Product.ProductID" name="Id" />

      <return-property column="Product.ProductName" name="Name" />

      <return-property column="Product.CategoryId" name="Category" />

      <return-property column="Product.SupplierID" name="Supplier" />

      <return-property column="Product.QuantityPerUnit" name="QuantityPerUnit" />

      <return-property column="Product.UnitPrice" name="UnitPrice" />

      <return-property column="Product.UnitsInStock" name="UnitsInStock" />

      <return-property column="Product.UnitsOnOrder" name="UnitsOnOrder" />

      <return-property column="Product.ReorderLevel" name="ReorderLevel" />

      <return-property column="Product.Discontinued" name="Discontinued" />

    </return>

    <return-join alias="Category" property="Product.Category">

      <return-property column="Category.CategoryId" name="Id" />

      <return-property column="Category.CategoryName" name="Name" />

      <return-property column="Category.Description" name="Description" />

    </return-join>

    exec dbo.GetProductsByCategoryId :CategoryId

  </sql-query>

We map each column of the Product table to its correct property of the Product class. Notice that we defined the ‘Product’ alias for this part of the data. Then we use the return-join element to map the joined properties to the ‘Product.Category’ property. This might look a bit weird at first. You have to specify the alias of the owning object (which in our case is the ‘Product’ alias), and then you need to specify the name of the property of the owning object upon which the other part of the data should be mapped (in our case, the ‘Category’ property of the ‘Product’ object).

Now we can retrieve the data like this:

            IQuery query = Session.GetNamedQuery("GetProductsByCategoryId");

            query.SetInt32("CategoryId", 1);

            IList results = query.List();

I first tried to use the IQuery’s generic List of T method which i had hoped would give me a generic list of Product entities. But i couldn’t get that working. So i tried the regular List method, and it turns out that NHibernate doesn’t just give me a list of Product entities… it gives me a list where each item in the list is an object array where the first item in the array is the Product entity, and the second item is the Category. Each Product entity’s Category property references the correct Category instance though. So you can get the product instances like this:

            IEnumerable<Product> products = results.Cast<Object[]>().Select(i => (Product)i[0]);

There’s probably an easier way to just get the list of Product entities from the named query, but i haven’t found it yet :)

Share/Save/Bookmark

Posted in NHibernate | 1 Comment »

NHibernate And The Second Level Cache

Posted by Davy Brion on 23rd November 2008

As i had mentioned earlier, one of the NHibernate related posts i had on my TODO list was “Getting Up To Speed With NHibernate’s Second Level Cache”.

Luckily, Gabriel Schenker beat me to it and wrote a fantastic post about it. It is by far the best documentation on the 2nd level cache that you’ll find for now, so be sure to check that out if you’re looking into using some of NHibernate’s more advanced caching features.

Share/Save/Bookmark

Posted in NHibernate | No Comments »

Populating Entities From Stored Procedures With NHibernate

Posted by Davy Brion on 23rd November 2008

A short while ago we needed to fetch the data for some entities through a stored procedure for performance reasons. We already use NHibernate in the typical way to fetch and modify the data of this entity type, but we just wanted something so we could also use the resultset of the stored procedure to populate the entities. One of my team members spent some time figuring out how to get the data returned by the stored procedure into the entities without actually having to write the code ourselves. Turns out this was pretty easy to do. Let’s go over the solution with a very simple example.

The stored procedure i’ll use for the example is extremely simple, and you’d never need to use this technique for such a stupid procedure. But in the situation we faced at work, the stored procedure was obviously a lot more complicated. So the stored procedure for this example is just this:

ALTER PROCEDURE [dbo].[GetProductsByCategoryId]

    @CategoryId int

AS

BEGIN

    SET NOCOUNT ON;

 

    SELECT [ProductID]

          ,[ProductName]

          ,[SupplierID]

          ,[CategoryID]

          ,[QuantityPerUnit]

          ,[UnitPrice]

          ,[UnitsInStock]

          ,[UnitsOnOrder]

          ,[ReorderLevel]

          ,[Discontinued]

      FROM [Northwind].[dbo].[Products]

     WHERE [CategoryId] = @CategoryId

END

This just returns the product rows for the given CategoryId parameter. Again, you’d never do this in real life but this simple procedure is just used as an example.

Now, the structure of the resultset that this procedure returns is identical to the structure that the Product entity is mapped to. This makes it really easy to get this data into the Product entities. Just add a named query to your mapping like this:

  <sql-query name="GetProductsByCategoryId">

    <return class="Product" />

    exec dbo.GetProductsByCategoryId :CategoryId

  </sql-query>

And this is all you need to do in code to get your list of entities from this stored procedure:

            IQuery query = Session.GetNamedQuery("GetProductsByCategoryId");

            query.SetInt32("CategoryId", 1);

            IList<Product> products = query.List<Product>();

Is that easy or what?

Now, suppose that the stored procedure returns more columns than you’ve got mapped to the entity. You can still use this approach as well, but then you’ll need to specify which return values map to which properties in the entity like this:

  <sql-query name="GetProductsByCategoryId">

    <return class="Product">

      <return-property column="ProductID" name="Id" />

      <return-property column="ProductName" name="Name" />

      <return-property column="SupplierID" name="Supplier" />

      <return-property column="CategoryID" name="Category" />

      <return-property column="QuantityPerUnit" name="QuantityPerUnit" />

      <return-property column="UnitPrice" name="UnitPrice" />

      <return-property column="UnitsInStock" name="UnitsInStock" />

      <return-property column="UnitsOnOrder" name="UnitsOnOrder" />

      <return-property column="ReorderLevel" name="ReorderLevel" />

      <return-property column="Discontinued" name="Discontinued" />

    </return>

    exec dbo.GetProductsByCategoryId :CategoryId

  </sql-query>

Notice how the CategoryID and SupplierID columns are mapped to Category and Supplier properties, which in Product’s mapping are mapped as Category and Supplier many-to-one types, so basically references of type Category and Supplier respectively. NHibernate basically just takes care of all of the dirty work.

Share/Save/Bookmark

Posted in NHibernate | 10 Comments »

Joining The NHibernate Developers

Posted by Davy Brion on 23rd November 2008

Big weekend so far… Fabio Maulo invited me to join the NHibernate development team, which i gladly accepted :)

It’s a great project and a great team as well. I’m very proud to be a part of that from now on. I’m so psyched about this that i don’t even know what to say :)

Share/Save/Bookmark

Posted in NHibernate | 11 Comments »

Joining Elegant Code

Posted by Davy Brion on 23rd November 2008

As David Starr already mentioned, i am now part of the Elegant Code team. It’s an honor to be invited and i’m obviously very happy with this. All of my posts will still be posted to my own blog, as well as on elegantcode.com. And the NHibernate posts will also continue to be cross posted to NHForge.

Now if you’ll excuse me, i’ve got to start writing some posts ;)

Share/Save/Bookmark

Posted in About The Blog | 2 Comments »

NHibernate Interest Increasing?

Posted by Davy Brion on 22nd November 2008

With the recent announcements regarding LINQ To SQL, and the Entity Framework team continuing to display how they really don’t ‘get’ it , i’ve noticed an increase in popularity of my NHibernate related posts lately. I’ve also noticed an increase in traffic from people who are searching online with NHibernate related keywords. Obviously, it could just be a coincidence. But it would be pretty cool if people are now giving NHibernate more proper consideration when investigating what data access technologies to build your applications with.

How about you? Has the LINQ To SQL announcement or the ‘progress’ of the Entity Framework had any impact on which technology you would consider for your applications?

Share/Save/Bookmark

Posted in NHibernate | 3 Comments »

Why On Earth Would A Developer Do This?

Posted by Davy Brion on 22nd November 2008

I just saw the following piece of code:

        public void Execute()

        {

            ArrayList empIds = PayrollDatabase.GetAllEmployeeIds();

 

            foreach (int empId in empIds)

            {

                Employee employee = PayrollDatabase.GetEmployee(empId);

 

                if (employee.IsPayDate(payDate))

                {

                    DateTime startDate = employee.GetPayPeriodStartDate(payDate);

                    Paycheck pc = new Paycheck(startDate, payDate);

                    paychecks[empId] = pc;

                    employee.Payday(pc);

                }

            }

        }

The code is old, pre .NET 2.0. But no, the usage of the ArrayList isn’t what bothers me here. It first gets all of the Id values for all of the employees in the database. Then it loops through the list of Id values, and fetches each employee from the database. Whenever i see this kind of code, it makes me want to kick the person who wrote it.

In case you don’t know what is so bad about this, think about this: if you have 5 employees you first send one select statement to the database to get their id values. Then you send another 5 select statements to the database to get the data of each employee. Oh well, 6 queries isn’t that big of a deal right? It sure as hell is if you could’ve just as easily gotten all of the required data with one query (and only one expensive roundtrip)! Imagine you have to calculate the paychecks of 100 employees instead of 5. How about 1000 employees?

What i really don’t understand about this is that this kind of code gets written every day. Do these people simply don’t care or do they genuinely not know how bad this is? If they don’t know, that’s truly sad. If they don’t care, that’s even worse because a developer who knows how bad this is but still writes it, obviously takes no pride at all in his/her work and (IMO) has little respect for the code, the team, the company and the client.

In case you’re wondering where i saw this code… it’s in Robert C. Martin’s Agile Principles, Patterns, and Practices in C# book. Yes, the Robert C. Martin, aka Uncle Bob. I probably shouldn’t criticize such a well known figure in the OO world, but seriously Bob, what were you smoking? The purpose of the whole book is to teach people how to write good code and there is tremendous amount of valuable information in there. But putting code like this in your examples is really inexcusable.

Share/Save/Bookmark

Posted in Opinions, Performance | 34 Comments »

200 Posts

Posted by Davy Brion on 20th November 2008

This is my 200th post. After my first 100 posts, i posted a list of my 10 most popular posts at the time. Well, popular is probably not the correct term. They were the 10 posts that had gotten the most visits anyway. So i figure i might as well make a tradition out of it. Here’s the list of the top 10 most ‘popular’ posts between the 100th and the 199th post:

  1. Career Advice For Young Developers

    This one was pretty big… it scored pretty good on reddit.com and also made it to the frontpage of delicious.com. In total it got over 15000 views, which is waaaay ahead of my other posts.

  2. How To Write Testable ASP.NET WebForms

    Basically my MVP approach for ASP.NET WebForms. Still using this approach, and am pretty happy with the results we’ve seen so far.

  3. Data Access With NHibernate

    My default Repository implementation to use with NHibernate. Still using a slightly modified version of this at work and again, pretty happy with how it’s worked out so far.

  4. Batching WCF Calls

    My way of eliminating chatty WCF interfaces while still being able to offer fine-grained functionality. Ayende even linked to this post, which at the time was the first real ‘highlight’ in my blogging career.

  5. The Request/Response Service Layer

    Basically a more formalized version of the Batching WCF Calls post. We’re using this approach in all of our new projects now, and it’s working pretty good.

  6. 6 Years As A Professional Software Developer

    Pretty much just a look back on the first 6 years of my career. I definitely didn’t expect this post to get as much attention as it did though.

  7. Link Blogs… Do Yourself A Favor

    My tribute to some of the great .NET-related link blogs. I get quite a bit of visitors from these guys, so thanks again! :)

  8. My Guide To Effective Test Driven Development

    There aren’t a lot of people doing Test Driven Development, and quite a bit of those just aren’t doing it as well they could be. This ‘guide’ was pretty much my attempt to list a couple of important things to always keep in mind.

  9. Software Development Books: Investing In Yourself

    This was my attempt to get developers to read more books about software development. The post got a lot of attention, but unfortunately it didn’t have the desired effect i had hoped it would have on my coworkers. I ordered a bunch of great books at work, and so far none of them has actually been read by any of my coworkers…

  10. .NET Memory Management

    A lot of people take .NET’s garbage collection for granted, and there aren’t too many people who try to use it effectively so i wanted to write a bit of guidance about this.

And that’s the list :)

Another post that just keeps on attracting visitors are my NHibernate Mapping Examples, but since that one was already included in the list of the first 100 posts, i had to leave it out. But it recently got the attention of the great Scott Hanselman, who even linked to the post on two occassions. Having one of Microsoft’s best people link to your posts (an NHibernate post even) is pretty much as good as it gets. Well, so far anyways ;)

Share/Save/Bookmark

Posted in About The Blog | No Comments »