Mark Needham

Thoughts on Software Development

Archive for the ‘Build’ tag

Configurable Builds: One configuration file per environment

with 7 comments

One of the most important things when coding build files is to try and make them as configurable as possible.

At the very least on an agile project there will be a need for two different configurations – one for developer machines and one for continuous integration.

On my last two .NET projects we have setup our Nant build to take in a parameter which indicates which build configuration should be used. We then have a configuration file by that name which contains the environment specific data.

The build file would contain the following code before anything else:

<fail unless="${property::exists('environment')}" message="You must provide the environment property to the build script using -D:environment=[dev|ci]" />
<include buildfile="${trunk.dir}\config\${environment}.properties.xml" />

dev.properties.xml would look like this:

<?xml version="1.0" ?>
<properties>
	<property name="property1" value="value1" />
	<property name="property2" value="value2" />
</properties>

We would call the build file for the dev environment like so:

nant -buildfile:build-file.build target-name -D:environment=dev

Configuring the build this way assumes that the dev builds all have the same properties. On the projects where I used this approach this was the case.

The disadvantage of it is that you need to remember to pass in the environment variable each time you call the build. This can be countered by wrapping the full nant call in a batch script if it becomes too much of a hassle.

Written by Mark Needham

September 2nd, 2008 at 1:50 am

Posted in Build

Tagged with , ,

scp Nant Task – ‘scp’ failed to start. The system cannot find the file specified

without comments

I was trying to make use of the Nant Contrib scp task earlier and was getting an error message which at the time seemed a bit strange (now of course having solve the problem it is obvious!)

This was the task I was running:

<scp file="someFile.txt" server="some.secure-server.com" />

This was the error:

1
2
'scp' failed to start. 
     The system cannot find the file specified

I ran it in debug mode to try and see what was going on and got this stack trace:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
NAnt.Core.BuildException: C:\projects\project1\default.build(18,4):
'scp' failed to start. ---> System.ComponentModel.Win32Exception: The system cannot find the file specified
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at NAnt.Core.Tasks.ExternalProgramBase.StartProcess()
   --- End of inner exception stack trace ---
   at NAnt.Core.Tasks.ExternalProgramBase.StartProcess()
   at NAnt.Core.Tasks.ExternalProgramBase.ExecuteTask()
   at NAnt.Contrib.Tasks.ScpTask.ExecuteTask()
   at NAnt.Core.Task.Execute()
   at NAnt.Core.Target.Execute()
   at NAnt.Core.Project.Execute(String targetName, Boolean forceDependencies)
   at NAnt.Core.Project.Execute()
   at NAnt.Core.Project.Run()

Eventually a colleague and I realised that the scp task was assuming that there was an executable called ‘scp’ on the path.

This can be overriden by setting the ‘program’ attribute to whatever you want. In this case since we were running from Windows we downloaded Putty’s pscp executable and put that on the Windows path. The code to call the scp task now looks like this:

<scp file="someFile.txt" server="some.secure-server.com" program="pscp" />

If you don’t want to put it on the path then the following works just as well:

<scp file="someFile.txt" server="some.secure-server.com" program="c:\path\to\pscp.exe" />

Written by Mark Needham

August 30th, 2008 at 4:30 pm

Posted in Build

Tagged with , ,

Encapsulation in build scripts using nant

with 2 comments

When writing build scripts it’s very easy for it to descend into complete Xml hell when you’re using a tool like nant.

I wondered previously whether it was possible to TDD build files and while this is difficult given the dependency model most build tools follow. That doesn’t mean we can’t apply other good design principles from the coding world however.

Encapsulation is one of the key principles of OOP and it can be applied in build files too. Stephen Chu talks about this in his post on Pragmatic Nant Scripting where he recommends having 3 different levels of targets to help create this encapsulation.

I’ve been trying to follow this advice with our build scripts and today Bernardo made the suggestion of using macros in an English readable way. He calls it OO Scripting – it’s effectively a DSL inside a DSL if you like.

I was having problems with the ncover nant task – the following error message was being thrown every time I called it:

could not find ncover folder in your path in NCoverExplorer.NAntTasks.NCoverUtilities

I managed to find the source code for that class and had a look at it but I couldn’t figure out what was going wrong without debugging through it. The strange thing was that it worked fine from the command line which suggested to me that I was getting something simple wrong.

I created a cover.tests macro to encapsulate the details of how I was executing the coverage.

The plan was to get it working using an exec call to the ncover executable and then phase the ncover nant task back in when I’d figured out what I was doing wrong.

This is what I started out with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<macrodef name="cover.tests">
        <attributes>
                <attribute name="in.assemblies" />
        </attributes>
       <sequential>
                <copy file="\path\to\Coverage.xsl" tofile="${report.dir}\Coverage.xsl" />
 
                <exec program="..\lib\NCover-1.5.8\NCover.Console.exe">
                        <arg value="..\lib\nunit-2.4\nunit-console.exe" />
                        <arg value="${build.dir}\UnitTests\UnitTests.dll" />
                        <arg value="//a" />
                        <arg value="${in.assemblies}" />
                        <arg value="//x" />
                        <arg value="${report.dir}\Unit.Test.Coverage.xml" />
                </exec>
        </sequential>
</macrodef>

//a is the assemblies to include in the report

//x is the name of the report xml file which will be created

The full list is here.

The macro was called like this:

1
2
3
<target name="coverage">
   <cover.tests in.assemblies="Project1;Project2" />
</target>

I substituted the ncover task back in with the same parameters as above and low and behold it worked!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<macrodef name="cover.tests">
        <attributes>
                <attribute name="in.assemblies" />
        </attributes>
        <sequential>
                <copy file="\path\to\Coverage.xsl" tofile="${report.dir}\Coverage.xsl" />
        
                <ncover
                    program="..\lib\NCover-1.5.8\NCover.Console.exe"
                    commandLineExe="..\lib\nunit-2.4\nunit-console.exe"
                    commandLineArgs="${build.dir}\UnitTests\UnitTests.dll"
                    coverageFile="${report.dir}\Unit.Test.Coverage.xml"
                    assemblyList="${in.assemblies}"
                  />        
        </sequential>
</macrodef>

I’m not sure exactly what the problem parameter was but encapsulating this part of the build gave me the option of working that out in a way that impacted very little of the rest of the build file.

*Update*
Fixed the first example to include the opening as pointed out by Vikram in the comments. Thanks again Vikram for pointing that out!

Written by Mark Needham

August 21st, 2008 at 12:40 am

Posted in Build

Tagged with , , , ,

Building in release mode with no pdbs with msbuild

with 3 comments

I’ve been having trouble trying to work out how to build our projects in msbuild in release mode without creating the customary pdb files that seem to be created by default.

I tried calling msbuild.exe with the ‘Release’ configuration:

'C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.Exe ( Proj.csproj /p:OutputPath=\output\path\ 	/p:Configuration=Release)'

To no avail. It still created the pdb file. Next I tried setting the ‘DebugSymbols’ property to false:

'C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.Exe ( Proj.csproj /p:OutputPath=\output\path\ 	/p:Configuration=Release /p:DebugSymbols=false)'

Still it created the file. Finally I found this post which suggested that you actually needed to make the change in the Proj.csproj file itself.

I changed this part of the file so that DebugType is now ‘none’. It had a value of ‘pdbonly’ when I opened the file.

  none
  true
  bin\Release\
  TRACE
  prompt
  4

The pdb is no longer created.

*Update*
This can also be done by passing /p:DebugType=none as a command line argument as Tim points out in the comments.

Written by Mark Needham

August 20th, 2008 at 6:50 pm

Posted in .NET,Build

Tagged with , , , ,

NCover – Requested value ‘/r’ was not found

with one comment

I’ve been trying to integrate NCover into our build and probably making life harder for myself than it needs to be.

The title refers to the error message that I was getting when trying to run the ncover nant task on version 1.0.1 of NCover earlier today.

1
2
3
4
5
6
7
8
[ncover] Starting 'C:\Program Files\NCover\ncover-console.exe 
(//r "\long\path\to\tmp392.tmp.ncoversettings" )' in 'C:\my-project\trunk\src'
[ncover] Unhandled Exception: System.ArgumentException: Requested value '/r' was not found.
[ncover]    at System.Enum.Parse(Type enumType, String value, Boolean ignoreCase)
[ncover]    at NCover.Utilities.Arguments.ParseArgument(String arg, 
CommandLineArgument& key, String& value) in C:\to
ols\eclipse3M6\workspace\ncover\src\NCover\Utilities\Arguments.cs:line 192
...

After some inspired Googling my colleague managed to work out that the problem was that you can’t pass a settings file path which has spaces in to the ncover executable, hence the error message. It’s the same problem in handling spaces that I mentioned in an earlier post on msbuild.

The advice on the forum was to upgrade to one of the more recent versions where the bug has been fixed. Downloads of the free version of NCover (it becomes paid for at version 2.0) are available here.

Written by Mark Needham

August 19th, 2008 at 9:18 pm

Posted in Build

Tagged with , , ,

msbuild – Use OutputPath instead of OutDir

with 8 comments

We’ve been using msbuild to build our project files on my current project and a colleague and I noticed some strange behaviour when trying to set the directory that the output should be built to.

The problem was whenever we tried to set the output directory (using OutDir) to somewhere where there was a space in the directory name it would just fail catastrophically. We spent ages searching for the command line documentation before finding it here.

According to this though:

“OutputPath: This property is typically specified in the project file and resembles OutDir. OutputPath has been deprecated and OutDir should be used instead whenever possible. ”

We decided to try changing OutDir to OutputPath and it started working again! The code is simple, but for those wondering:

1
2
3
4
<exec program="\path\to\msbuild35">
  <arg value="${projectfile}" />
  <arg value="/p:OutputPath=${build.dir}\${project.name}\" />
</exec>

I can’t decide whether the documentation is just wrong or if it’s now a convention that you can’t have spaces in your build output path. Surely the former?

Written by Mark Needham

August 14th, 2008 at 7:54 pm

Posted in Build

Tagged with ,

TeamCity’s strange default build location

without comments

We’ve been using TeamCity on my current project and it’s proven to be fairly impressive in general.

We’re running quite a few different builds which have dependencies on each other and it’s been pretty much one click on the web admin tool to get that set up.

One thing that had me really confused is the default location it chooses to build from. The problem is that it seems to change arbitrarily, with the folder name it builds in being calculated from a VSC hash (not sure quite how that’s worked out but there we go).

I had (naively) assumed that the default build location would always stay the same and was therefore referencing this location from our build script. Turns out it can change whenever it feels like it!

Our 15:38 build was built to c:/TeamCity/Agent/c734523aedrte but our 15:40 build was built to c:/TeamCity/Agent/d6420ghi2.

Cue much confusion as I repeatedly looked at the first folder trying desperately to work out why nothing was being checked out. Eventually, more by fluke than skill, I figured it out and our build is now being built to c:/WorkingDirectory every time.

Lesson learnt: Take control and set your default directory, don’t let TeamCity do it for you!

Written by Mark Needham

August 8th, 2008 at 7:52 pm