Reliable rake task execution

My News Sniffer project needs to regularly do some back-end stuff like checking a bunch of rss feeds and downloading web pages. I do this with some rake tasks, which I call using the cron daemon.  Recently I’ve been having problems where some tasks take a bit longer than usual to complete and end up running in parallel. This slows things down, which means more tasks end up running in parallel and then my little virtual machine eventually falls on it’s face under memory pressure.

I could implement some locking in my application, but it’s always good to avoid as much new code as possible so, in the good old *NIX fashion, I cobbled together a short bash script taking advantage of existing tools. What this does is executes the given rake task in the given rails root using the Debian/Ubuntu tool start-stop-daemon (provided by the dpkg package, which is therefore always installed). start-stop-daemon uses a pid file to keep track of the rake program for the given task, so it will never run a second concurrent instance of rake for this task. Cron just keeps trying to run it every 5 minutes or whatever, but it only runs once concurrently.
Continue reading “Reliable rake task execution”

Xen on Ubuntu Edgy broken – Debian to the rescue

Xen LogoI found a great HOWTO on getting a Xen virtual machine up and running on Ubuntu Edgy. Unfortunately, Xen is pretty broken on Edgy. I managed to get one virtual machine running easily but on starting a second one, Dom0 panics and the whole box freezes. It’s been reported on launchpad but as it’s in universe I guess it’s just not a priority.

As a workaround, I installed the Xen kernels from Debian testing (Etch). These pulled in the Debian Xen userspace packages too, but they didn’t clobber anything. So now I have a couple of mostly Edgy virtual machines which I plan to use as a little test lab.

Whilst troubleshooting, I also found another good Debian specific Xen HOWTO with lots of good general advice too.

Etch is looking good. I work with Sarge a lot on servers and firewalls but I’ve not played with Etch much. I think I’ll get it installed in a Xen machine and have a play. I feel bad for not helping with its testing though.