Mark Needham

Thoughts on Software Development

Archive for the ‘scripting’ tag

Shell: Create a comma separated string

without comments

I recently needed to generate a string with comma separated values, based on iterating a range of numbers.

e.g. we should get the following output where n = 3

foo-0,foo-1,foo-2

I only had the shell available to me so I couldn’t shell out into Python or Ruby for example. That means it’s bash scripting time!

If we want to iterate a range of numbers and print them out on the screen we can write the following code:

n=3
for i in $(seq 0 $(($n > 0? $n-1: 0))); do 
  echo "foo-$i"
done
 
foo-0
foo-1
foo-2

Combining them into a string is a bit more tricky, but luckily I found a great blog post by Andreas Haupt which shows what to do. Andreas is solving a more complicated problem than me but these are the bits of code that we need from the post.

n=3
combined=""
 
for i in $(seq 0 $(($n > 0? $n-1: 0))); do 
  token="foo-$i"
  combined="${combined}${combined:+,}$token"
done
echo $combined
 
foo-0,foo-1,foo-2

This won’t work if you set n<0 but that’s ok for me! I’ll let Andreas explain how it works:

  • ${combined:+,} will return either a comma (if combined exists and is set) or nothing at all.
  • In the first invocation of the loop combined is not yet set and nothing is put out.
  • In the next rounds combined is set and a comma will be put out.

We can see how it in action by printing out the value of $combined after each iteration of the loop:

n=3
combined=""
 
for i in $(seq 0 $(($n > 0 ? $n-1: 0))); do 
  token="foo-$i"
  combined="${combined}${combined:+,}$token"
  echo $combined
done
 
foo-0
foo-0,foo-1
foo-0,foo-1,foo-2

Looks good to me!

Written by Mark Needham

June 23rd, 2017 at 12:26 pm

Posted in Shell Scripting

Tagged with , ,

F#: Useful for scripting

with one comment

We had the need to do a bit of scripting recently to change the names of the folders where we store our artifacts to signify which artifacts were created from our build’s production branch and which were generated from the main branch.

The problem we had was that we were ending up overwriting old artifacts from the main branch with the production branch’s artifacts so we wanted to fix this.

We had already manually changed some of the folder names to work with the changes that had already been made to our deployment script to read from the proposed new folder names.

We therefore had a folder structure that looked like this:

  • Artifacts
    • 12
    • 20
    • 45
    • 1000
    • 1001
    • Trunk-1050
    • Prod-23

All the folders with numbers over 1000 were from the trunk build since our production build is only up to around 50. The trouble in the renaming was around the lower numbers where some could be production or trunk.

I think this could have been calculated by checking the creation date of the folders but I decided it was quicker at the time to just scan through them and manually note which were of each type.

At the time we didn’t have F#¬†installed so I wrote the script in Ruby but I decided to rewrite it later on in an F# script file and then use F# interactive to execute the script.

This is the script I’ve ended up with:

(rename.fsx)

open System
open System.IO
open System.Text.RegularExpressions
 
let get_position_of_last_folder (dir:string) = dir.LastIndexOf('\\') + 1
 
let get_last_folder_name (dir:string) = dir.Substring(get_position_of_last_folder dir)
let get_rest_of_dir_name (dir:string) = dir.Substring(0, (get_position_of_last_folder dir)-1)
 
let create_new_dir_name dir = 
    let prodVersions = seq { yield! [47..48]; yield! [37..45]; yield! [28..34]; } 
    let folderName = get_last_folder_name dir
    let isProdDir = prodVersions |> Seq.exists (fun item -> item = Int32.Parse(folderName))
 
    let build_dir_name branch = (get_rest_of_dir_name dir) + "\\" + branch + "-" + folderName
 
    if (isProdDir) then build_dir_name "Prod" else build_dir_name "Trunk"
 
let rename_directories =  Array.filter (fun dir -> Regex.IsMatch(get_last_folder_name dir , "^[0-9]") ) >> 
                          Array.iter (fun dir -> Directory.Move(dir, create_new_dir_name dir))
 
rename_directories <| Directory.GetDirectories("C:\\artifacts")

I don’t really like the ‘get_position_of_last_folder’ function but it helped to remove the duplication in the following two functions. Maybe there’s a better way to remove this duplication that I’m not aware of.

We can then execute this by using the following command (note I have added ‘C:\Program Files\FSharp-1.9.6.2\bin’ to the path):

fsi --exec --nologo rename.fsx

I wrote this script file in Visual Studio so that I could get all the Intellisense help that I need but it’s not part of any project – it stands alone!

I learnt about the possibility to do scripting in F# from a blog post by Chris Smith where he talks about some of the ways that he’s been able to use F# for scripting in his work.

I thought running an F# script would be significantly slower than running an equivalent Ruby one since the F# code needs to be compiled first but I didn’t notice that the F# script ran any slower than the Ruby one just from observation.

Written by Mark Needham

June 9th, 2009 at 11:29 pm

Posted in F#

Tagged with ,