Mark Needham

Thoughts on Software Development

Archive for the ‘nant’ tag

Nant: Populating templates

without comments

One of the common tasks that we need to do on every project I’ve worked on is ensure that we can create a web.config file for the different environments that we need to deploy our application to.

Nant has quite a neat task called ‘expandproperties‘ which allows us to do this quite easily.

In our build file we would have the following:

build-file.build

  <property name ="configFile" value="${environment}.properties" readonly="true"/>
  <if test="${not file::exists(configFile)}">
    <fail message="Configuration file '${configFile}' could not be found." />
  </if>
  <include buildfile ="${configFile}" />
 
  <target name="GenerateConfigFiles">
    <foreach item="File" property="TemplateFile">
      <in>
        <items>
          <include name="ProjectDirectory/**.template"/>
        </items>
      </in>
      <do>
        <copy file="${TemplateFile}" tofile="${string::replace(TemplateFile,'.template','')}" overwrite="true">
          <filterchain>
            <expandproperties />
          </filterchain>
        </copy>
      </do>
    </foreach>
  </target>

There would be corresponding ‘.template’ and ‘.properties’ files containing the various variables we want to set, like so:

dev.properties

 <project>
	 <property name="SomeHost" value="http://my-host.com:8080"/>
 </project>

web.template.config

<?xml version="1.0"?>
...
<configuration>
   <appSettings>
      <add key="SomeHost="${SomeHost}"/>
   </appSettings>
</configuration>
...

We would call the build file like so:

nant -buildfile:build-file.build GenerateConfigFiles -D:environment=dev

We can then change that ‘environment’ variable depending which one we need to generate the configuration files for.

Written by Mark Needham

January 16th, 2010 at 12:13 am

Posted in Build

Tagged with ,

Nant include task – namespace matters

with 4 comments

We’ve been trying to include some properties into our build file from a properties file today but no matter what we tried the properties were not being set.

We eventually realised that the build file has an XML Namespace set on the project element.

<project name="..." xmlns="http://nant.sf.net/schemas/nant.xsd">

It turns out that if you want to include a properties file in your build file, like so:

<include buildfile="properties.xml" />

…you need to put the namespace on the project attribute of that file as well, otherwise its properties don’t get picked up.

Our properties file therefore needs to look like this, assuming that we have a namespace set on the build file.

<project name="properties" xmlns="http://nant.sf.net/schemas/nant.xsd">
	<property name="foo" value="bar" />
</project>

What’s a bit confusing is that on the NAnt documentation page for the include task it says that project element attributes are ignored! That’s not what we found!

Written by Mark Needham

February 3rd, 2009 at 10:43 am

Posted in Build

Tagged with ,

Configurable Builds: One configuration file per machine

without comments

I’ve covered some of the ways that I’ve seen for making builds configurable in previous posts:

One which I haven’t covered which my colleagues Gil Peeters and Jim Barritt have pointed out is having a build with one configuration file for each machine.

Again the setup is fairly similar to one configuration per user or environment. Using Nant we would have the following near the top of the build file:

<property name="machine.name" value="${environment::get-machine-name()}" />
<include buildfile="${trunk.dir}\config\${machine.name}.properties.xml" />

We could then have one configuration for each developer machine:

machine1.properties.xml

<?xml version="1.0" ?>
<properties>
	<property name="property1" value="onevalue" />
</properties>

machine2.properties.xml

<?xml version="1.0" ?>
<properties>
	<property name="property1" value="anothervalue" />
</properties>

The build file can be run using the following command:

nant -buildfile:build-file.build target-name

The benefit of this approach can be seen (as Gil points out) in pair programming where the settings on any one machine will always be the same regardless of who is logged in. We also still get the advantage of being able to use remote resources on developer machines.

Having machine specific configuration also allows more flexibility for configurations on continuous integration for example. To quote Gil:

Each CI build (multiple builds per build server) get’s it’s own [configuration] based on the build host and build name.

The disadvantage again is we have to add a new configuration file every time we want to run the build on a different machine.

Written by Mark Needham

September 13th, 2008 at 3:54 am

Posted in Build

Tagged with , ,

Configurable Builds: Overriding properties

without comments

Sometimes when configuring our build for flexibility we don’t need to spend the time required to create one build configuration per user or one build configuration per environment.

In these cases we can just override properties when we call Nant from the command line.

One recent example where I made use of this was where we had one configuration file with properties in but wanted to override a couple of them when we ran the continuous integration build.

Since build properties are immutable (i.e. once they are set they can’t be changed) if we set them from the command line the build script makes use of these values.

For example we might have the following in our build file:

<property name="repository.url" value="http://localhost:3000" />

But when we’re running it on cruise control we have the repository on a different machine. We can override it like so:

nant -buildfile:build-file.build target-name -D:repository.url=http://some-remote-url

If we have more than one property we want to override it might be a bit annoying to have to pass them all via the command line. We can define them in a file to overcome this problem:

nant -buildfile:build-file.build target-name @ci.properties.xml

where ci.properties.xml contains the following:

-D:repository.url=http://remote-url:3000
-D:some.other.property=newvalue

If you start seeing a lot of properties in this file then it is probably an indicator that you need to have a more robust solution but this works for providing simple flexibility.

Written by Mark Needham

September 2nd, 2008 at 2:49 pm

Posted in Build

Tagged with , ,

Configurable Builds: One configuration file per user

with 2 comments

Following on from my first post about making builds configurable, the second way of doing this that I have seen is to have one configuration build file per user.

This approach is more useful where there are different configurations needed on each developer machine. For example, if the databases being used for development are on a remote server then each developer machine would be assigned a database with a different name.

The setup is fairly similar to configuring by environment – the main difference is that we don’t have to pass the user in as a parameter. The following would go near the top of the build file:

<property name="user" value="${environment::get-user-name()}" />
<include buildfile="${trunk.dir}\config\${user}.properties.xml" />

We can then have different configurations for two developer machines like so:

developer1.properties.xml

<?xml version="1.0" ?>
<properties>
	<property name="property1" value="onevalue" />
</properties>

developer2.properties.xml

<?xml version="1.0" ?>
<properties>
	<property name="property1" value="anothervalue" />
</properties>

We can then run the build file like this:

nant -buildfile:build-file.build target-name

The disadvantage of this approach is that every time a new developer joins the team they need to create a new configuration file with their settings in. We also need to ensure that the continuous integration build is running using an independent user account. It provides more flexibility and is easier to setup on the plus side.

My colleague Jim Barritt points out a similar technique his team is using here.

Written by Mark Needham

September 2nd, 2008 at 1:53 pm

Posted in Build

Tagged with , ,

Configurable Builds: One configuration file per environment

with 7 comments

One of the most important things when coding build files is to try and make them as configurable as possible.

At the very least on an agile project there will be a need for two different configurations – one for developer machines and one for continuous integration.

On my last two .NET projects we have setup our Nant build to take in a parameter which indicates which build configuration should be used. We then have a configuration file by that name which contains the environment specific data.

The build file would contain the following code before anything else:

<fail unless="${property::exists('environment')}" message="You must provide the environment property to the build script using -D:environment=[dev|ci]" />
<include buildfile="${trunk.dir}\config\${environment}.properties.xml" />

dev.properties.xml would look like this:

<?xml version="1.0" ?>
<properties>
	<property name="property1" value="value1" />
	<property name="property2" value="value2" />
</properties>

We would call the build file for the dev environment like so:

nant -buildfile:build-file.build target-name -D:environment=dev

Configuring the build this way assumes that the dev builds all have the same properties. On the projects where I used this approach this was the case.

The disadvantage of it is that you need to remember to pass in the environment variable each time you call the build. This can be countered by wrapping the full nant call in a batch script if it becomes too much of a hassle.

Written by Mark Needham

September 2nd, 2008 at 1:50 am

Posted in Build

Tagged with , ,

scp Nant Task – ‘scp’ failed to start. The system cannot find the file specified

without comments

I was trying to make use of the Nant Contrib scp task earlier and was getting an error message which at the time seemed a bit strange (now of course having solve the problem it is obvious!)

This was the task I was running:

<scp file="someFile.txt" server="some.secure-server.com" />

This was the error:

1
2
'scp' failed to start. 
     The system cannot find the file specified

I ran it in debug mode to try and see what was going on and got this stack trace:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
NAnt.Core.BuildException: C:\projects\project1\default.build(18,4):
'scp' failed to start. ---> System.ComponentModel.Win32Exception: The system cannot find the file specified
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at NAnt.Core.Tasks.ExternalProgramBase.StartProcess()
   --- End of inner exception stack trace ---
   at NAnt.Core.Tasks.ExternalProgramBase.StartProcess()
   at NAnt.Core.Tasks.ExternalProgramBase.ExecuteTask()
   at NAnt.Contrib.Tasks.ScpTask.ExecuteTask()
   at NAnt.Core.Task.Execute()
   at NAnt.Core.Target.Execute()
   at NAnt.Core.Project.Execute(String targetName, Boolean forceDependencies)
   at NAnt.Core.Project.Execute()
   at NAnt.Core.Project.Run()

Eventually a colleague and I realised that the scp task was assuming that there was an executable called ‘scp’ on the path.

This can be overriden by setting the ‘program’ attribute to whatever you want. In this case since we were running from Windows we downloaded Putty’s pscp executable and put that on the Windows path. The code to call the scp task now looks like this:

<scp file="someFile.txt" server="some.secure-server.com" program="pscp" />

If you don’t want to put it on the path then the following works just as well:

<scp file="someFile.txt" server="some.secure-server.com" program="c:\path\to\pscp.exe" />

Written by Mark Needham

August 30th, 2008 at 4:30 pm

Posted in Build

Tagged with , ,

NCover Nant Team City Integration

with 2 comments

I’ve been spending quite a bit of time setting up NCover and then integrating it into Team City.

I’ve read some posts which cover parts of this process but nothing which covers the end to end process so hopefully my experience can help to fill that void.

Step 1

Download NCover 1.5.8, NCover Explorer 1.4.0.7, NCover Explorer Extras 1.4.0.5 from Kiwidude’s website and the NCover website .

Step 2

Put the following into your Nant build file:

1
2
	<loadtasks assembly="..\lib\NCoverExplorer.Extras\NCoverExplorer.NAntTasks.dll"/>
   	<exec program="regsvr32" workingdir="..\lib\NCover-1.5.8" commandline="/s coverlib.dll"/>

I put this right at the top of the build but I expect it doesn’t matter where it goes as long as it’s called at some stage before NCover and NCover Explorer are called.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
<macrodef name="cover.tests">
	<attributes>
		<attribute name="in.assemblies" />
	</attributes>
	<sequential>	
		<ncover
		    program="..\lib\NCover-1.5.8\NCover.Console.exe"
		    commandLineExe="..\lib\nunit-2.4\nunit-console.exe"
		    commandLineArgs="${build.dir}\UnitTests\UnitTests.dll"
		    coverageFile="${report.dir}\Unit.Test.Coverage.xml"
			assemblyList="${in.assemblies}"
		  />	
 
	  <ncoverexplorer
		program="..\lib\NCoverExplorer\NCoverExplorer.Console.exe"
		projectName="Project"
		reportType="ModuleClassSummary" 
		outputDir="${report.dir}"
		xmlReportName="TestCoverage.xml"
		htmlReportName="TestCoverage.html"
		showExcluded="True"
		satisfactoryCoverage="80" >
		<fileset>
		  <include name="${report.dir}\Unit.Test.Coverage.xml" />
		</fileset>
		<exclusions>
		  <exclusion type="Assembly" pattern="*.Tests" />
		  <exclusion type="Namespace" pattern="*.Tests*" />
		</exclusions>
	  </ncoverexplorer>			  		  
	</sequential>
</macrodef>

This macro can then be called as follows:

1
2
3
<target name="cover.unit.tests"	
	<cover.tests in.assemblies="Project1;Project1" />
</target>

N.B. The projects passed in as the ‘in.assemblies’ argument should be semi colon separated.

Step 3

The next step is to setup the artifacts for your project. From the Team City admin panel navigate to the project configuration settings and select artifacts.

Add the following to the ‘Artifact paths’:

1
TestCoverage.html

It should now show up as a viewable artifact from the project listing page.

Step 4

To get the coverage report to show up on a tab on the build summary page we need to edit the main-config.xml file

The location of this file can be found by browsing to ‘Administration > Server Configuration’ from the Team City admin panel

Add the following line after the other ‘report-tab’ entries in this file:

 

Potential Problems

I encountered some problems in getting this up and running. They were as follows:

NCover: Profiled process terminated. Profiler connection not established

After some Googling I found this post which explains how to solve the problem.

To summarise this problem occurs when trying to run NCover without Administrative privileges. The coverlib.dll shipped with NCover needs to be registered. This can be done two ways:

1) Put the following code into your build file right at the top

1
<exec program="regsvr32" workingdir="\path\to\ncover" commandline="/s coverlib.dll"/>

2) Run the same command from the command line

1
C:\path\to\NCover-1.5.8>regsvr32 CoverLib.dll

NCover – Requested value ‘/r’ was not found

This error occurred when I was using version 1.0.1 of NCover and to cut a long story short, you need to upgrade to get rid of the problem.

More details are on this post.

The information here has been accumulated from my experiences, this post on NCover integration and the official documentation.

Written by Mark Needham

August 25th, 2008 at 9:29 pm

Posted in Build

Tagged with , , ,

Encapsulation in build scripts using nant

with 2 comments

When writing build scripts it’s very easy for it to descend into complete Xml hell when you’re using a tool like nant.

I wondered previously whether it was possible to TDD build files and while this is difficult given the dependency model most build tools follow. That doesn’t mean we can’t apply other good design principles from the coding world however.

Encapsulation is one of the key principles of OOP and it can be applied in build files too. Stephen Chu talks about this in his post on Pragmatic Nant Scripting where he recommends having 3 different levels of targets to help create this encapsulation.

I’ve been trying to follow this advice with our build scripts and today Bernardo made the suggestion of using macros in an English readable way. He calls it OO Scripting – it’s effectively a DSL inside a DSL if you like.

I was having problems with the ncover nant task – the following error message was being thrown every time I called it:

could not find ncover folder in your path in NCoverExplorer.NAntTasks.NCoverUtilities

I managed to find the source code for that class and had a look at it but I couldn’t figure out what was going wrong without debugging through it. The strange thing was that it worked fine from the command line which suggested to me that I was getting something simple wrong.

I created a cover.tests macro to encapsulate the details of how I was executing the coverage.

The plan was to get it working using an exec call to the ncover executable and then phase the ncover nant task back in when I’d figured out what I was doing wrong.

This is what I started out with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<macrodef name="cover.tests">
        <attributes>
                <attribute name="in.assemblies" />
        </attributes>
       <sequential>
                <copy file="\path\to\Coverage.xsl" tofile="${report.dir}\Coverage.xsl" />
 
                <exec program="..\lib\NCover-1.5.8\NCover.Console.exe">
                        <arg value="..\lib\nunit-2.4\nunit-console.exe" />
                        <arg value="${build.dir}\UnitTests\UnitTests.dll" />
                        <arg value="//a" />
                        <arg value="${in.assemblies}" />
                        <arg value="//x" />
                        <arg value="${report.dir}\Unit.Test.Coverage.xml" />
                </exec>
        </sequential>
</macrodef>

//a is the assemblies to include in the report

//x is the name of the report xml file which will be created

The full list is here.

The macro was called like this:

1
2
3
<target name="coverage">
   <cover.tests in.assemblies="Project1;Project2" />
</target>

I substituted the ncover task back in with the same parameters as above and low and behold it worked!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<macrodef name="cover.tests">
        <attributes>
                <attribute name="in.assemblies" />
        </attributes>
        <sequential>
                <copy file="\path\to\Coverage.xsl" tofile="${report.dir}\Coverage.xsl" />
        
                <ncover
                    program="..\lib\NCover-1.5.8\NCover.Console.exe"
                    commandLineExe="..\lib\nunit-2.4\nunit-console.exe"
                    commandLineArgs="${build.dir}\UnitTests\UnitTests.dll"
                    coverageFile="${report.dir}\Unit.Test.Coverage.xml"
                    assemblyList="${in.assemblies}"
                  />        
        </sequential>
</macrodef>

I’m not sure exactly what the problem parameter was but encapsulating this part of the build gave me the option of working that out in a way that impacted very little of the rest of the build file.

*Update*
Fixed the first example to include the opening as pointed out by Vikram in the comments. Thanks again Vikram for pointing that out!

Written by Mark Needham

August 21st, 2008 at 12:40 am

Posted in Build

Tagged with , , , ,

NCover – Requested value ‘/r’ was not found

with one comment

I’ve been trying to integrate NCover into our build and probably making life harder for myself than it needs to be.

The title refers to the error message that I was getting when trying to run the ncover nant task on version 1.0.1 of NCover earlier today.

1
2
3
4
5
6
7
8
[ncover] Starting 'C:\Program Files\NCover\ncover-console.exe 
(//r "\long\path\to\tmp392.tmp.ncoversettings" )' in 'C:\my-project\trunk\src'
[ncover] Unhandled Exception: System.ArgumentException: Requested value '/r' was not found.
[ncover]    at System.Enum.Parse(Type enumType, String value, Boolean ignoreCase)
[ncover]    at NCover.Utilities.Arguments.ParseArgument(String arg, 
CommandLineArgument& key, String& value) in C:\to
ols\eclipse3M6\workspace\ncover\src\NCover\Utilities\Arguments.cs:line 192
...

After some inspired Googling my colleague managed to work out that the problem was that you can’t pass a settings file path which has spaces in to the ncover executable, hence the error message. It’s the same problem in handling spaces that I mentioned in an earlier post on msbuild.

The advice on the forum was to upgrade to one of the more recent versions where the bug has been fixed. Downloads of the free version of NCover (it becomes paid for at version 2.0) are available here.

Written by Mark Needham

August 19th, 2008 at 9:18 pm

Posted in Build

Tagged with , , ,