At work, our old code deployment strategy was basically a wrapper script doing an svn checkout and some symlinking. With our move to Puppet for config management, we also moved to using Apt packaging for our code deployment, tying them together with a line similar to :
class foo-export {
package { 'foo-export': ensure => latest }
}
So that whenever we deploy a new version of a package to our apt-repo, it can then be installed with a:
puppet agent --test
(and with an initial dry-run using --noop)
( I should mention I manage our Puppet runs via our own distributed scripts, rather than having the nodes set up to check in every 30mins - when I'm doing so much work on our Puppet setup and config, I'd rather not having machines check in automatically in case the config is in a broken state )
Inevitably I would run the above Puppet command and it would not find any new packages, because ‘d'uh!', of course I still need to run an apt-get update.
I've been using Puppet stages for a while now, in order to group package installations in a broader sense rather than manually spelling out every dependency with a require => stanza, so it was a simple addition to add in a pre stage, and have the nodes run apt-get update before any runs.
In order to use stages, you need to first define them in your site.pp. By default every defined class runs under Stage[main], so you just need to add the new stages and define the running order. (full Puppet stage documentation is here)
At the top of my site.pp file, I added a pre and post stage, then define the execution order via:
stage { [pre, post]: }
Stage[pre] -> Stage[main] -> Stage[post]
Then I created a class called apt-hupdate (sorry, i use stupid naming conventions!) in
modules/apt-hupdate/manifests/init.pp
which contained:
class apt-hupdate {
exec { "aptHupdate":
command => "/usr/bin/apt-get update",
}
}
And finally, include that in your site.pp with:
class { apt-hupdate: stage => pre }
Now every time you do a Puppet run, apt-get update will be the first task run.
I have a speedtest perl script i wrote - nothing complicated, takes a file and uploads it to a remote FTP or SFTP server, while calculating how long, then gives you a a measure of the MB/per second bandwidth between two sites.
I want it available on a selection of machines so it can run from wherever, so I thought i'd package it up as a .DEB file and stick it in our local repo. Nothing complicated in that, and there are a number of online tutorials about building your own debs. The main drawback with most I found was that they assume you are actually building from source rather than just distributing a script, although I also found a relevant Ubuntu thread which is pretty simple and to the point.
However, even using these tutorials it still took me a few hours to figure out. There are just a couple of non-obvious points, so i figure writing out my own steps is worth recording -
So first, grab your required packages:
apt-get install dh-make dpkg-dev debhelper devscripts fakeroot lintian
You will need to build from a directory with the name of your script in the form packagename-version, so for mine i created /tmp/speedtest-1.0, then copied in my script ‘speedtest‘ and it's data file 25MBFLAC.file ( which i could have created with dd on the box rather than copy over, but downloading the file is actually quicker in this situation ).
The first step is to run:
dh_make -s --indep --createorig -e thor@valhalla.com
(dash-s means create a single binary .deb - i.e. no source version; indep means architecture-independent; and createorig is to indicate you are the original maintainer)
this creates a top-level ‘debian‘ directory containing all the necessary config files.
The main one you need to edit is debian/control - you prob only need fill in “section”, “homepage” and “Description”
Mine looks like:
Source: speedtest
Section: web
Priority: extra
Maintainer: Thorsten Sideboard <thor@valhalla.com>
Build-Depends: debhelper (>= 7.0.50~)
Standards-Version: 3.8.4
Homepage: http://github.com/sideboard/speedtest.git
Package: speedtest
Architecture: all
Depends: ${misc:Depends}
Description: Test Upload Speeds
One of the things which baffled me for a while, which was answered in the askubuntu link above, was how to specify where something is installed — it goes in a file ‘debian/install‘ which isn't created for you. The format of the file is ‘filename location/to/be/installed” (without the initial slash)
so in my case, i ran:
echo "speedtest usr/local/Scriptz/" > debian/install
echo "25MBFLAC.file usr/local/Scriptz/" >> debian/install
At this point, you should then be able to run:
debuild -us -uc
and you should have a deb file built. but..
First i ran into :
dpkg-source: error: can't build with source format '3.0 (quilt)': no orig.tar file found
As the above-mentioned askubuntu post says, you can
echo "1.0" > debian/source/format
then re-running the debuild -us -uc i ran into
dpkg-source: error: cannot represent change to speedtemp-1.0/25MBFLAC.file: binary file contents changed
This error is due to leftover build-cruft from my last run - if you check the directory one step up from where you are, you'll see debuild has already built some files for you, typically a tar.gz, a .dsc and a .build file. Delete all them, then re-run debuild -us -uc — now it should build properly!
ah!
dh_usrlocal: debian/speedtemp/usr/local/Scriptz/speedtest is not a directory
This one also caught me out for a while - turns out this is caused by my specifying “/usr/local/Scriptz” as my install location -
Most third-party software installs itself in the /usr/local directory hierarchy. On Debian this is reserved for private use by the system administrator, so packages must not use directories such as /usr/local/bin but should instead use system directories such as /usr/bin, obeying the Filesystem Hierarchy Standard (FHS).
(from here)
So, yeah, i changed my debian/install file to be “speedtest usr/bin”
and finally! running debuild -us -uc completes properly, outputting a /tmp/speedtest_1.0-1_all.deb which can then be installed via
dpkg -i /tmp/speedtest_1.0-1_all.deb
One last note — there are four useful scripts to also know about — preinst, postinst, prerm, postrm — these should be in the debian/ directory - pretty self-explanatory - pre- and post- install and remove scripts - if these exist, they will be run exactly as they are named, so for example, i wanted my 25MBFLAC.file still to be installed under /usr/local/Scriptz, so i listed it to be installed in the debian/install file as “25MBFLAC.file tmp” and then in my postinst file, i added:
#!/bin/sh
mv /tmp/25MBFLAC.file /usr/local/Scriptz/