Abstract AST

‘To start this article let us cite the definition of Continuous Integration from Martin Fowler’s site:

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. This article is a quick overview of Continuous Integration summarizing the technique and its current usage.

I advice you to read the whole article about Continuous Integration before you go on.

For the sake of CI you need the following tools:

1 – A Build Server

TeamCity is very popular CI server from JetBrains, the  creator of ReSharper, IntelliJ, etc. I have had also used CruiseControl.NET, a free CI server from ThoughtWorks, but I liked TeamCity much more because of its friendly UI and distributed agents principle.

2 – Build Tool

MSBuild is an xml-based build platform from Microsoft that comes with the .NET framework. A build platform is some kind of DSL, that helps you to compile your code and do all other related stuff in one action. You could compile source code, execute tests, generate documentation, generate a deployable zip file, publish your web site to the server and almost everything else you may need to do with your code.

Technically, you could write a batch file as you your build script and I have no doubt it will work and do every thing you wish. It just won’t be that convenient.

NAnt is another popular build platform ported from Ant for the Java platform.

Recently I have heard a lot about using Rake, the Ruby build tool, in the .NET world. Psake  is another build automation tool written in PowerShell .

In this and few next posts I will configuring TeamCity to do a nightly build for Stutali , the task list generation tool, I introduced last in this article.The task list generation task will be also included in the nightly build to process its own code and generate the task list.

Building the Build Script

The first step in configuring the build server is to add a build script to the project.  Because TeamCity supports sln files directly, a build script is technically not necessary if you just want to compile the project. Anyway if you want to perform some further building steps like processing  source code or deployment you will need the script.

As already mentioned, MSBuild is the  technology that will be used in this article. The first and most important task is the bare compilation. For this task MSBuild needs only to know  where the sln file is.

<PropertyGroup>
  <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
  <ProjectDir>src</ProjectDir>
  <ProjectName>TasklistGenerator</ProjectName >
  <ProjectFile>$(ProjectName).sln</ProjectFile >
</PropertyGroup>

<Target Name="Build" DependsOnTargets="Clean">
    <MSBuild Projects="$(ProjectFile)" Properties="Configuration=Debug" />
</Target>
  

The Property group section is the place where you define you constants. This could include the project name, tools path and every thing you may use at different places.

TeamCity

After testing the build script and pushing the code to the repository, we can start configuring TeamCity to do the CI work. At first we have to create a new project in the admin panel. This task is as easy as clicking through the wizard steps, filling out some text boxes and ready you are. TeamCity makes really pleasant. Out of the box is Git not supported. Anyway JetBrains has developed a plug-in to enable Git repositories. You have just to download this file from JetBrains and it to the place described.

For the source code repository you should have been able to choose Git from the list after getting the Git plugin installed properly.

In the build triggering section you can specify when you want the build process to be started. Plausible options are either when the repository changes, i.e. when you push modification, or scheduled on a regular base, usually in the nightly, the so called nightly build.

Artifacts

Artifacts are any data items generated during the build process. Those could be binaries, documentation, reports, etc. Because of distributed nature of TeamCity, the actual compiling process doesn’t have to happen physically on the save machine as the TeamCity server. Thus, the artifacts you want to keep have to be specified explicitly, so that they be copied from the machine of the build agent and kept on the server.

Suppose for now the build script generates an html file containing the task list parsed from the source code and save it to CHECKOUT_FOLDER\doc\tasklist.html.  If you want TeamCity to copy this file and make available for download in the artifacts tab of the build report, you have to specify it in the artifacts section

If you want to add an alias to the copied artifacts, just use the => syntax

doc\tasklist.html => tasks.html

You can also copy a whole folder of artifacts.

Now we are ready to go. Every thing is configured and we can trigger a test build. Just click the “run” that you see on almost every project page. If every thing went well you should see a new green line in the result page of the project.

Outlook

It was just the start. What you can do in the CI process is only limited by your imagination and time. You could generate code metric reports, what we will do in the next posts, generate release binaries, push them to the download server, generate documentation,  push the new ASP.Net files to the server and precompile them, etc.

For the next posts, we will see how to integrate NDepend, NCover, FxCop and code documentation with the build process.

See you later!

A Task list is the list, that VS generate for your code files including all special marker items like TODO, HACK, etc. Visual studio lists all those items in the tasks window in grid containing the description, file and line number.  Clicking an items in the tasks windows cause the editor to jump to this line in the code.  Unfortunately, visual studio show only tasks for opened file. Additionally, I thinks the number of task is a metric for code maturity and should be integrated in the continues integration process.

The Project

After some googling with both Google and Bing I decided to do it myself. The only similar program I found, was the Task Scanner plug-in from the Hudson project.  Unfortunately, this plug-in is written in Java and I didn’t want to assume the presence of a Java runtime on the build server. Besides, I want to integrate this job as build task (MSBuild build for now but maybe also for NAnt, etc..). For all other purposes a command line interface is essential. All those requirements make the amount of code I could reuse very minimal. So I went for the own solution and started summarizing the requirements.

Wish List

If you prefer the scrum terminology, you can call this wish list the product backlog. It contains all items that COULD be implemented.

Important feature for me are:

• Recognizing all lines including // TODO token
• Flexibility of defining pre configured and custom tasks
• Assigning priorities for the the different types of tasks
• Returning for each task a TaskItem object with the description, file and line number.
• Grouping the tasks by file name or  priority.
• Integration with TeamCity
• Generating reports in both xml and html formats
• being able to define thresholds for the occurrence of the different type/priorities  to consider a build failed

To generate a task list to you MSBuild script all you need the the following xml snippet:

<UsingTask AssemblyFile="$(ToolPath)\TasklistGenerator\TasklistGenerator.dll" TaskName="Tasklist" />

<Target Name="tasklist" >
	<Message Text="==== Generate tas klist ===" />
	<CreateItem Include="src\**\*.cs">
      <Output TaskParameter="Include" ItemName="Sourcefiles" />
    </CreateItem>
    <Tasklist Files="@(Sourcefiles)" OutputFile="doc\tasks.html" />
</Target>

The Code

Currently, still the project in the pre alpha phase of the CTP (PACTP as I call it since two minutes ago). It Uses a Regex to scan all specified cs files and generates a simple xml representation. A html reporting facility is also in pre stone age look and feel  html (Did I mention I don’t do UI?)  I will try to find out how does usually build tasks, like NUnit, integrate with build server. not sure if there is a common format. The only interface available currently is the MSBuild task.

You can download the most recent source code from GitHub  at the following link http://github.com/mouk/Stutali/tree/master.

Because I have never developed an integration task, I don’t know what are the requirements exactly such a thing. More than ever, every idea or notes will be highly appreciated.

xUnit.net is a unit testing framework for the .Net platform designed with inspired by some nice ideas like reducing custom attributes, one instance per method , etc. The Internet is full of blog posts and articles about xUnit. In this link you can find a comparison of xUnit and other popular unit testing frameworks.

For the CI I use MSBuild script that compiles, runs test, generates documentations, etc.  For test projects I have the habit of postfixing them with "Tests”  to simplify fetching all test dlls in MSBuild scripts using wild cards (Who dares to claim that Convention over Configuration is new in the .Net world? ). Running all NUnit tests in some project would look like :

<CreateItem Include="tests\*\bin\Debug\*.Tests.dll">
  <Output TaskParameter="Include" ItemName="TestAssembly" />
</CreateItem>
<NUnit Assemblies="@(TestAssembly)" ToolPath="$(NUnit-ToolPath)" />

Adding new test project will require me to add nothing to the build script. I have just to follow the convention.

Unfortunately this is not possible with xUnit task because it accepts only one assembly. Thus I have to specify each assembly separately. The xml result will be also split. @bradwilson nicely wrote me via  Twitter, that I can use a project file to specify multiple assembly.  This, while solving the second problem, doesn’t enable you to use wild cards. You have to add each assembly manually to the project.

Gallio is a tool  or, may better a framework, for running many test frameworks in the same way and provides as unique results format. It integrates also very well with some build servers.

To integrate Gallio in my MSBuild script I downloaded the bits from here and extracted them to the tools folder inside my project root folder.

Running the tests was as simple as

<UsingTask AssemblyFile="$(ToolPath)\gallio\Gallio.MSBuildTasks.dll"
	TaskName="Gallio" />

<Target Name="Test" DependsOnTargets="Build">
	<Message Text="==== Starting Gallio to run tests ===" />

	<CreateItem Include="tests\*\bin\Debug\*Tests.dll">
      <Output TaskParameter="Include" ItemName="TestAssemblies" />
    </CreateItem>
    <Gallio RunnerExtensions="TeamCityExtension,Gallio.TeamCityIntegration"
        Assemblies="@(TestAssemblies)"  />
</Target>

RunnerExtensions is an extension for Gallio to provide TeamCity with status messages. Integrating Gallio with TeamCity will be discussed in a following post.

NUnit   and running multiple assemblies from the console

Even though NUnit MSBuild task supports multiple assemblies, if you want to use NCover with NUnit  you have to do some manual work to enable NCover running NUnit console tool. The @() operator of MSBuild join all paths with a semicolon. On the other hand NUnit console runner expect all assemblies to be separated with a white space. Fortunately you can tell @() how it should concatenate the paths.

<NCover ToolPath="$(NCoverPath)"
        CommandLineExe="$(NUnit-ToolPath)\nunit-console.exe"
        WorkingDirectory="."
        CommandLineArgs="@(TestDll->'%(Identity)', ' ')"
        CoverageFile="$(CoverageFile)"
        LogFile="$(CoveragePath)\Coverage.log"
        AssemblyList="@(CoverageDll)" />

In the fourth line @ concatenate the relative path of each file using a white space. You may need to surround each path with a quote if it contains white spaces.