Mark Needham

Thoughts on Software Development

Archive for the ‘vagrant’ tag

Vagrant: Multi (virtual) machine with Puppet roles

with 5 comments

I’ve been playing around with setting up a neo4j cluster using Vagrant and HAProxy and one thing I wanted to do was define two different roles for the HAProxy and neo4j machines.

When I was working at uSwitch Nathan had solved a similar problem, but with AWS VMs, by defining the role in an environment variable in the VM’s spin up script.

In retrospect I think I might have been able to do that by using the shell provisioner and calling that before the puppet provisioner but Nathan, Gareth Rushgrove and Gregor Russbuelt suggested that using facter might be better.

When I initially looked at the ‘Custom Facts’ section of the docs I thought it was only possible to set facts for the Vagrant file as a whole but you can actually do it on a per VM basis which is neat.

I added a method called ‘provision_as_role’ to the ‘Vagrant::Config::V2::Root’ class:

module Vagrant
  module Config
    module V2
      class Root
        def provision_as_role(role)
          vm.provision :puppet do |puppet|
            puppet.manifests_path = "puppet/manifests"
            puppet.module_path = "puppet/modules"
            puppet.manifest_file  = "base.pp"
            puppet.facter = { "role" => role.to_s }
          end 
        end
      end
    end
  end
end

and then passed in a role depending on the VM in my Vagrantfile:

require File.join(File.dirname(__FILE__), 'lib', 'root.rb')
 
Vagrant.configure("2") do |config|  
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"
 
  config.vm.define :neo01 do |neo| 
    neo.vm.hostname = "neo01"
    neo.vm.network :private_network, ip: "192.168.33.101"    
    neo.provision_as_role :neo
  end
 
  config.vm.define :lb01 do |lb|
    lb.vm.hostname = "lb01"
    lb.vm.network :private_network, ip: "192.168.33.104"    
    lb.provision_as_role :lb
  end
end

We can now access the variable ‘$role’ in our puppet code which I used like so:

puppet/base.pp

class all_the_things {
  exec { 'apt-get update': command => '/usr/bin/apt-get update'; }
  package { 'curl': ensure => '7.22.0-3ubuntu4', }
  class { 'apt': }
}
 
node default {
  class { 'all_the_things': }
  class { $role:
    require => Class['all_the_things']
  }
}

The ‘neo’ and ‘lb’ classes look like this:

class neo {
  class { 'java': version => '7u25-0~webupd8~1', }
  class { 'neo4j': require     => Class['java'], }
}
class lb {
  class { 'haproxy':  }
}

The full code is on github but it’s behaving a bit weirdly in some scenarios so I’m still trying to get it properly working.

Written by Mark Needham

June 30th, 2013 at 1:13 pm

Posted in Software Development

Tagged with

Vagrant 1.2.2: `[]’: can’t convert Symbol into Integer (TypeError)/The following settings don’t exist

without comments

As I mentioned in my previous post I’ve been playing around with Vagrant for the past couple of days and I was trying to adapt a Vagrantfile that Nathan created a few months ago to do what I wanted.

I’m using Vagrant 1.2.2 and I started out with the following Vagrantfile:

Vagrant.configure("2") do |config|   
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"
 
  config.vm.define :neo01 do |neo|    
    neo.vm.network :hostonly, "192.168.33.101"
    neo.vm.forward_port 8080, 4569
  end
end

Unfortunately a ‘vagrant up’ doesn’t quite work as expected:

$ vagrant up
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/plugins/kernel_v2/config/vm.rb:146:in `[]': can't convert Symbol into Integer (TypeError)
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/plugins/kernel_v2/config/vm.rb:146:in `network'
	from /Users/markneedham/projects/support/haproxy/Vagrantfile:19:in `block (2 levels) in <top (required)>'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/config/v2/loader.rb:37:in `call'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/config/v2/loader.rb:37:in `load'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/config/loader.rb:104:in `block (2 levels) in load'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/config/loader.rb:98:in `each'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/config/loader.rb:98:in `block in load'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/config/loader.rb:95:in `each'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/config/loader.rb:95:in `load'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/environment.rb:329:in `machine'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/plugin/v2/command.rb:131:in `block in with_target_vms'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/plugin/v2/command.rb:164:in `call'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/plugin/v2/command.rb:164:in `block in with_target_vms'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/plugin/v2/command.rb:163:in `map'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/plugin/v2/command.rb:163:in `with_target_vms'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/plugins/commands/up/command.rb:42:in `block in execute'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/environment.rb:206:in `block (2 levels) in batch'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/environment.rb:204:in `tap'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/environment.rb:204:in `block in batch'
	from <internal:prelude>:10:in `synchronize'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/environment.rb:203:in `batch'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/plugins/commands/up/command.rb:41:in `execute'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/cli.rb:46:in `execute'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/lib/vagrant/environment.rb:467:in `cli'
	from /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.2/bin/vagrant:84:in `<top (required)>'
	from /Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in `load'
	from /Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in `<main>'

The problem line is ‘neo.vm.network :hostonly, “192.168.33.101”‘ which I learnt is because the way you setup networking has changed slightly between the versions.

We now need something like this:

Vagrant.configure("2") do |config|   
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"
 
  config.vm.define :neo01 do |neo|    
    neo.vm.network :private_network, ip: "192.168.33.101"
    neo.vm.forward_port 8080, 4569
  end
end

If we run ‘vagrant up’ again we’ll see that error has been removed and we have a new one!

$ vagrant up
Bringing machine 'neo01' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:
 
vm:
* The following settings don't exist: forward_port

It turns out the way that we do port forwarding has also changed. Annoyingly if you google ‘vagrant port forwarding’ it still points you to the old documentation rather than the newer one.

We need to change our code to the following:

Vagrant.configure("2") do |config|   
  config.vm.box = "precise64"
  config.vm.box_url = "http://files.vagrantup.com/precise64.box"
 
  config.vm.define :neo01 do |neo|    
    neo.vm.network :private_network, ip: "192.168.33.101"
    neo.vm.network :forwarded_port, host: 4569, guest: 8080
  end
end

If we run ‘vagrant up’ now it’s much happier:

$ vagrant up
Bringing machine 'neo01' up with 'virtualbox' provider...
[neo01] Setting the name of the VM...
[neo01] Clearing any previously set forwarded ports...
...

Written by Mark Needham

June 29th, 2013 at 8:44 am

Posted in Software Development

Tagged with

Vagrant/Virtual Box: There was an error executing the following command with VBoxManage – Progress object failure: NS_ERROR_CALL_FAILED

with one comment

I’ve been playing around with Vagrant a bit again lately and having installed it on a new machine was running into the following exception when I tried to run ‘vagrant up’ on a new virtual machine:

ERROR vagrant: /Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/providers/virtualbox/driver/base.rb:292:in `block in execute'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/util/retryable.rb:17:in `retryable'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/providers/virtualbox/driver/base.rb:282:in `execute'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/providers/virtualbox/driver/version_4_2.rb:165:in `import'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/providers/virtualbox/action/import.rb:15:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/builtin/handle_box_url.rb:38:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/providers/virtualbox/action/check_accessible.rb:18:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/runner.rb:61:in `block in run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/util/busy.rb:19:in `busy'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/runner.rb:61:in `run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/builtin/call.rb:51:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/providers/virtualbox/action/check_virtualbox.rb:17:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/warden.rb:34:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/builder.rb:109:in `call'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/runner.rb:61:in `block in run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/util/busy.rb:19:in `busy'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/action/runner.rb:61:in `run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/machine.rb:129:in `action'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/commands/up/command.rb:37:in `block in execute'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/plugin/v2/command.rb:182:in `block in with_target_vms'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/plugin/v2/command.rb:180:in `each'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/plugin/v2/command.rb:180:in `with_target_vms'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/plugins/commands/up/command.rb:32:in `execute'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/cli.rb:46:in `execute'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/lib/vagrant/environment.rb:406:in `cli'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.1.2/bin/vagrant:60:in `<top (required)>'
/Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in `load'
/Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in `<main>'
 INFO interface: error: There was an error executing the following command with VBoxManage:
 
["import", "/Users/markneedham/.vagrant.d/boxes/precise32/virtualbox/box.ovf"]

I tried running the command directly against Virtual Box to see if that would tell me anything else but it didn’t shed much light:

$ VBoxManage import "/Users/markneedham/.vagrant.d/boxes/precise32/virtualbox/box.ovf"
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Interpreting /Users/markneedham/.vagrant.d/boxes/precise32/virtualbox/box.ovf...
OK.
Disks:  vmdisk1	85899345920	-1	http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized	box-disk1.vmdk	-1	-1
Virtual system 0:
 0: Suggested OS type: "Ubuntu_64"
    (change with "--vsys 0 --ostype <type>"; use "list ostypes" to list all possible values)
 1: Suggested VM name "precise64"
    (change with "--vsys 0 --vmname <name>")
 2: Number of CPUs: 2
    (change with "--vsys 0 --cpus <n>")
 3: Guest memory: 384 MB
    (change with "--vsys 0 --memory <MB>")
 4: Network adapter: orig NAT, config 3, extra slot=0;type=NAT
 5: CD-ROM
    (disable with "--vsys 0 --unit 5 --ignore")
 6: IDE controller, type PIIX4
    (disable with "--vsys 0 --unit 6 --ignore")
 7: IDE controller, type PIIX4
    (disable with "--vsys 0 --unit 7 --ignore")
 8: SATA controller, type AHCI
    (disable with "--vsys 0 --unit 8 --ignore")
 9: Hard disk image: source image=box-disk1.vmdk, target path=/Users/markneedham/VirtualBox VMs/precise64/box-disk1.vmdk, controller=8;channel=0
    (change target path with "--vsys 0 --unit 9 --disk path";
    disable with "--vsys 0 --unit 9 --ignore")
0%...
Progress object failure: NS_ERROR_CALL_FAILED

That gave me enough information to Google my way to an issue on github where it was suggested that this was a problem with VirtualBox 4.2.14 and that downgrading with 4.2.10 would more profitable.

As I’d downloaded the latest VirtualBox the version I had was indeed 4.2.14:

$ VBoxManage --version
4.2.14r86644

I grabbed an older copy of Virtual Box and that seemed to do the trick although I later ran into a problem which I foolishly haven’t kept the stack trace of and found that upgrading my vagrant version solved that.

So to summarise, my current versions of vagrant and Virtual Box which work splendidly:

$ vagrant -v
Vagrant version 1.2.2
 
$ VBoxManage -version
4.2.10r84104

YMMV and all that!

Written by Mark Needham

June 29th, 2013 at 7:38 am

Posted in Software Development

Tagged with ,

Puppet: Installing Oracle Java – oracle-license-v1-1 license could not be presented

with 5 comments

In order to run the neo4j server on my Ubuntu 12.04 Vagrant VM I needed to install the Oracle/Sun JDK which proved to be more difficult than I’d expected.

I initially tried to install it via the OAB-Java script but was running into some dependency problems and eventually came across a post which specified a PPA that had an installer I could use.

I wrote a little puppet Java module to wrap the commands in:

class java($version) {
  package { "python-software-properties": }
 
  exec { "add-apt-repository-oracle":
    command => "/usr/bin/add-apt-repository -y ppa:webupd8team/java",
    notify => Exec["apt_update"]
  }
 
  package { 'oracle-java7-installer':
    ensure => "${version}",
    require => [Exec['add-apt-repository-oracle']],
  }
}

I then included this in my default node definition:

node default {
  class { 'java': version => '7u21-0~webupd8~0', }
}

(as Dave Yeung points out in the comments, you may need to tweak the version. Running aptitude versions oracle-java7-installer should indicate the latest version.)

Unfortunately when I ran that I ended up with the following error:

err: /Stage[main]/Java/Package[oracle-java7-installer]/ensure: change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install oracle-java7-installer' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  java-common
Suggested packages:
...
Unpacking oracle-java7-installer (from .../oracle-java7-installer_7u21-0~webupd8~0_all.deb) ...
 
oracle-license-v1-1 license could not be presented
try 'dpkg-reconfigure debconf' to select a frontend other than noninteractive
 
dpkg: error processing /var/cache/apt/archives/oracle-java7-installer_7u21-0~webupd8~0_all.deb (--unpack):
 subprocess new pre-installation script returned error exit status 2
Processing triggers for man-db ...
Errors were encountered while processing:
 /var/cache/apt/archives/oracle-java7-installer_7u21-0~webupd8~0_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

I came across this post on Ask Ubuntu which explained a neat trick for getting around it by making it look like we’ve agreed to the licence. This is done by passing options to debconf-set-selections.

For a real server I guess you’d want some step where a person accepts the licence but since this is just for my hacking it seems to make sense.

My new Java manifest looks like this:

class java($version) {
  package { "python-software-properties": }
 
  exec { "add-apt-repository-oracle":
    command => "/usr/bin/add-apt-repository -y ppa:webupd8team/java",
    notify => Exec["apt_update"]
  }
 
  exec {
    'set-licence-selected':
      command => '/bin/echo debconf shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections';
 
    'set-licence-seen':
      command => '/bin/echo debconf shared/accepted-oracle-license-v1-1 seen true | /usr/bin/debconf-set-selections';
  }
 
  package { 'oracle-java7-installer':
    ensure => "${version}",
    require => [Exec['add-apt-repository-oracle'], Exec['set-licence-selected'], Exec['set-licence-seen']],
  }
}

Written by Mark Needham

April 18th, 2013 at 11:36 pm

Posted in DevOps

Tagged with ,