Using TeamCity’s NUnit support, while keeping the output around
Posted by Davy Brion on July 19th, 2008
I’ve been playing around with TeamCity a lot lately. It’s really amazing how easy to use and powerful it is. Definitely my favorite CI server for the time being… i’m even running it at home now for my personal projects.
There was one issue at work that was somewhat hard to fix though. TeamCity has fantastic integrated support for NUnit, but unfortunately it doesn’t write NUnit’s output to XML files like nunit-console.exe does. After a bit of browsing, i found a post on the TeamCity forums that discussed a workaround on how to get the output. Unfortunately the workaround is a bit cumbersome as it requires you to you create an XML file which contains the arguments that TeamCity would pass to its NUnitLauncher task.
I believe in ’script reuse’ as much as i believe in ‘code reuse’, so every project’s build script merely imports a generic build script and then it just overwrites some variables that the generic script uses and then it kicks off the usual build process. Since all of our projects will now be built by TeamCity Build Agents, and we definitely need to have NUnit’s results xml file i wanted to automate this whole thing as much as possible.
So instead of having to create (and thus, maintain) a cumbersome xml file for each project, i wrote an MSBuild task that would generate the xml file containing the arguments on the fly during the build, and would then pass those xml arguments to TeamCity’s NUnitLauncher. This way we get to keep all of TeamCity’s NUnit integration goodness, while still keeping the NUnit result files around as well.
So now in my MSBuild file, i can just do this:
<CreateItem Include="**\Bin\Debug\*Tests*.dll" >
<Output TaskParameter="Include" ItemName="TestAssemblies" />
</CreateItem>
<BuildTeamCityNUnitArguments HaltOnError="true" HaltOnFirstTestFailure="true"
HaltOnFailureAtEnd="true" TestAssemblies="@(TestAssemblies)"
NUnitResultsOutputFolder="TestResults"
PathOfNUnitArgumentsXmlFile="nunitarguments.xml" />
<Exec Command="$(teamcity_dotnet_nunitlauncher) @@ nunitarguments.xml" />
As you can see, there is nothing project-specific in there so this just integrates nicely into each build that we need. The only limitation to this approach is that TeamCity’s NUnitLauncher will write one NUnit result file for each assembly containing tests. Apparently there is no way in the current version of TeamCity to get it to combine those results. We use an extra tool to analyze the NUnit output after the tests have run, and unfortunately it requires all of the results to be in one file. I looked around for a way to merge the output of several NUnit result files, but i didn’t find something that was already available. So i wrote another MSBuild task that would merge the output into one xml file:
<CreateItem Include="TestResults\*.xml" >
<Output TaskParameter="Include" ItemName="NUnitOutputXmlFiles"/>
</CreateItem>
<NUnitMergeOutput NUnitOutputXmlFiles="@(NUnitOutputXmlFiles)"
PathOfMergedXmlFile="TestResults.xml" />
Now we finally have all of TeamCity’s goodness while we still get to run our post-test analysis on the NUnit result file that contains the testresults of all the test assemblies 
You can find the code of the MSBuild tasks here. They are BSD-licensed so feel free to use them if you need them.
July 21st, 2008 at 1:19 pm
Hi,
I am the originator of the TeamCity forum thread. Your workaround is excellent. Thanks
Regards,
feffe
November 30th, 2008 at 1:35 am
[...] that there are problems getting the test results in one nicely merged output file. So I found a workaround developed by Davy Brion that I thought I’d [...]
March 20th, 2009 at 6:07 am
Good stuff. Do you know how to specify include/exclude categories for tests? I don’t see it in your example and I don’t see much info as to what the xml file is supposed to look like when it has include-category and exclude-category specified.
March 20th, 2009 at 7:39 am
@David
sorry, no idea on that
March 20th, 2009 at 8:09 am
Hi,
Could you please help me in the following issues, i have no knowledge about Team city.
1.I have installed the Teamcity 4.0.2 in my machine and i treated
my machine as server. I have installed one build agent on my machine and one more build agent on another machine.
2. I have selected the sln2005 as a build runner and i entered the test assemblies details for nunit tests settings.When i want to execute the testcase on my machines, its working fine. If i want to run the test cases on another build agent which is installed on another machine, i am not seeing the Tests option button. Could Please help me in this.
3.I have app.config file is there, how can we edit the app.config file before test execution starts.
4.How we build and executes the test cases with the help of Msbuild?
5.How can we run our build with out installing the VCS on build agents.
Please help me, its very urgent
March 20th, 2009 at 8:33 am
use msbuild as the build runner, and use a simple msbuild script like this one:
http://davybrion.com/blog/2008/07/basic-msbuild-script/
if you want to modify the app.config file before you start your test run, you just need to add another build step in your msbuild script between compilation and the test run. Plenty of examples on this can be found on the net.
March 24th, 2009 at 1:47 pm
Thanks a lot, its working fine. Could you please let me know, how can i capture the nunit test results.
When i use your sample script, which is mentioned in the top of this page, i am getting the following error.
\Tests\DataIntegrity\build.xml(11, 2): error MSB4036: The “BuildTeamCityNUnitArguments” task was not found. Check the following: 1.) The name of the task in the project file is the same as the name of the task class. 2.) The task class is “public” and implements the Microsoft.Build.Framework.ITask interface. 3.) The task is correctly declared with in the project file, or in the *.tasks files located in the “C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727″ directory
March 24th, 2009 at 1:51 pm
that exception clearly states that it is not working fine… it’s not working at all because it can’t even find the build task
you have to actually download the buildtask and import it with the correct location of the buildtask
March 24th, 2009 at 2:01 pm
Davvy,
Can we implement the msbuild script for execution of the nunit test cases only. Is it possible?
March 25th, 2009 at 2:37 pm
Thanks Davvy, now its working fine for me. I have one small doubt.
Currently we are using ClearCase as VCS and i created the msbuild file in one of my view. When i want to tried to build & execution of the test cases on my machine and i didn’t see any problem.
I have installed the build agents on 2 more machines, which dont have the clearcase, when i tried to build & execute the test cases on remote build agents i got the following error. I have no idea why i am getting the error, and my view is in share mode only.
Cannot find build file by path specified in build configuration settings: ‘\\WINK-SHK\Tests\ka11_Tesing_TCity_view\tdnetdp\Tests\DataIntegrity\build.xml’ (absolute path on agent). Please check that specified path is correct
Do i need install the clearcase on these machines ?
March 25th, 2009 at 2:49 pm
i dunno… i hardly know anything about ClearCase and i definitely don’t know everything there is to know about TeamCity either
i would recommend reading the manuals of the products you are using and/or finding someone who knows more about ClearCase…
April 2nd, 2009 at 11:31 am
[...] have one post where i describe a workaround for a certain problem that you might encounter while working with [...]
April 2nd, 2009 at 11:32 am
@Kiran (and any of your coworkers who keep hassling me with a problem that i can’t help you with)
please read this:
http://davybrion.com/blog/2009/04/i-am-not-a-helpdesk/
April 2nd, 2009 at 2:39 pm
I am really sorry
November 23rd, 2009 at 8:47 pm
Davy,
Great stuff! I’m using your MSBuild script and BuildTeamCityNUnitArguments target to create the NUnit arguments file. I also wrote my own results parser that does the work to record the results to my own test management system, but I’m having a problem with recording the results if a test failed. When a test fails, the build fails and the target to record the results does not get run (it’s a post-unit test target). Any suggestions?
Again, great stuff, thanks!!
Dave
November 23rd, 2009 at 8:52 pm
@Dave
we do the exact same thing… there’s an option in your teamcity build configuration where you can enable/disable whether or not a build should be considered a failure if a test fails. Turn that off, and the rest of your targets will still be executed. As a final target, we have something that parses the testresults to check for errors, and if it finds one (or more) it fails the build… the obvious benefit being that everything else is still executed after a failed testrun