Mark Needham

Thoughts on Software Development

Puppet: Package Versions – To pin or not to pin

with 6 comments

Over the last year or so I’ve spent quite a bit of time working with puppet and one of the things that we had to decide when installing packages was whether or not to specify a particular version.

On the first project I worked on we didn’t bother and just let the package manager chose the most recent version.

Therefore if we were installing nginx the puppet code would read like this:

package { 'nginx':
  ensure  => 'present',
}

We can see which version that would install by checking the version table for the package:

$ apt-cache policy nginx
nginx:
  Installed: (none)
  Candidate: 1:1.2.6-1~43~precise1
  Version table:
     1:1.2.6-1~43~precise1 0
        500 http://ppa.launchpad.net/brightbox/ruby-ng/ubuntu/ precise/main amd64 Packages
     1.4.0-1~precise 0
        500 http://nginx.org/packages/ubuntu/ precise/nginx amd64 Packages
     1.1.19-1ubuntu0.1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise-updates/universe amd64 Packages
     1.1.19-1 0
        500 http://us.archive.ubuntu.com/ubuntu/ precise/universe amd64 Packages

In this case if we don’t specify a version the Brightbox ‘1:1.2.6-1~43~precise1′ version will be installed.

Running dpkg with the ‘compare-versions’ flag shows us that this version is considered higher than the nginx.org one:

$ dpkg --compare-versions '1:1.2.6-1~43~precise1' gt '1.4.0-1~precise' ; echo $?
0

From what I understand you can pin versions higher up the list by associating a higher number with them but given that all these versions are set to ‘500’ I’m not sure how it decides on the order!

The problem with not specifying a version is that when a new version becomes available the next time puppet runs it will automatically upgrade the version for us.

Most of the time this isn’t a problem but there were a couple of occasions when a version got bumped and something elsewhere stopped working and it took us quite a while to work out what had changed.

The alternative approach is to pin the package installation to a specific version. So if we want the recent 1.4.0 version installed we’d have the following code:

package { 'nginx':
  ensure  => '1.4.0-1~precise',
}

The nice thing about this approach is that we always know which version is going to be installed.

The problem we now introduce is that when an updated version is added to the repository the old one is typically removed which means a puppet run on a new machine will fail because it can’t find the version.

After working with puppet for a few months it becomes quite easy to see when this is the reason for the failure but it creates the perception that ‘puppet is always failing’ for newer people which isn’t so good.

I think on balance I prefer to have the versions explicitly defined because I find it easier to work out what’s going on that way but I’m sure there’s an equally strong argument for just picking the latest version.

Be Sociable, Share!

Written by Mark Needham

April 27th, 2013 at 1:40 pm

Posted in DevOps

Tagged with

  • DC

    Deterministic behaviour is good. Non is eeeeevil. Maven is eeeevil.

  • http://twitter.com/SamJSharpe Sam Sharpe

    If all priorities are 500, then it picks the highest based on epoch, then version, then revision.

    Most of your listed packages don’t have an epoch, but “1:1.2.6-1~43~precise1″ has an epoch of “1” and a version of “1.2.6” and a revision of “1~43~precise1″ so it will be selected in this case as none of the others have an epoch.

    More about Debian versioning here: http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Version

    RPMs also have an Epoch tag for the same reason: http://www.rpm.org/max-rpm-snapshot/s1-rpm-depend-manual-dependencies.html#S3-RPM-DEPEND-VERSION-NOT-ENOUGH

  • Jordan Sissel

    “””The problem we now introduce is that when an updated version is added to the repository the old one is typically removed which means a puppet run on a new machine will fail because it can’t find the version.”””

    Regarding the above –

    The problem is that reprepro only generates repo data (for apt-get) with a single version of any package. You can easily work around this by using apt-ftparchive instead which is just as fast and has less silly constraints. This allows you to push all versions of all internal packages to a single repo and pin versions in puppet itself with package { “…”: ensure => “1.2.3.4”; } and not having to worry about things :)

  • http://www.markhneedham.com/blog Mark Needham

    @google-75fdd336ae5f818276f1cf79e0468b2d:disqus ah neat didn’t know that. So I suppose we could also use that to get/store copies of external packages to achieve the same thing.

  • http://twitter.com/AutomatnMonkey Automation Monkey

    Maven’s determinism problems are essentially over and have been since 2.0.9. Admittedly, that’s pretty late in the game but Java dependency management is/was a difficult nut to crack.

  • Pingback: When Puppet and Chef Aren’t Quite Enough | Metafor SoftwareMetafor Software