Category: Tech

Sun’s ZFS on Linux via FUSE

Ricardo Correia has been porting Sun’s recently GPLed ZFS to Linux using FUSE. I’ve been playing with it and I’m quite impressed. The FUSE port is alpha quality, so isn’t to be trusted with important data yet – but it’s fun to play with.

ZFS merges the concept of a volume manager and a filesystem. It’s a bit like LVM, with zpools being volume groups and zfs being formatted logical volumes. Zfs “partitions” can change size at any time in any way. It’s also hierarchical, so zfs partitions can have child partitions inheriting their attributes. It also does away with fstab – all mount points are specified as zfs attributes and are automatically mounted when a zpool is brought online.

(more…)

8/2/2007 Add Comment

Active Resource not in Rails 1.2!

Whilst planning some changes to my News Sniffer project, I thought I’d have a play with Active Resource.

Currently, all the forum and news article downloading and scraping happens on a different machine to the web server. It has a VPN connection to the database and memcache servers, but I’d like to integrate the Ferret text indexing system for better searching capabilities. To centralise Ferret, I have a three options:

  1. regularly reindex new content from the database on the web server;
  2. DRb a Ferret Object;
  3. or use ActiveResource to access the models via the web service.

DRb-ing a Ferret Object would be quite elegant, but using ActiveResource would also replace the need for a database and memcache connection (and I could do much better fragment caching actually).

Anyway, I searched high and low for some docs – lots of blog entries about how great it is, but no real api docs. When I searched through the Rails code and found nothing either, I got suspicious. Finally I found a couple of blog entries stating that ActiveResource was dropped from Rails 1.2. It seems to be planned for Rails 2.0. Not sure how I missed this. I guess my search-foo is lacking.

I’ll be investigating other options. I’d much prefer not to build a SOAP or XMLRPC interface. Ugh.

5/2/2007 1 Comment

Maintenance pages status codes and Lighttpd

I’ve noticed a few very broken maintenance page Lighttpd config examples around, including the one on the mod_magnet documentation page. They all manage to display the maintenance page ok, but they return a HTTP 200 status code to the client, rather than the more appropriate HTTP 503 code.

As with all 500 status codes, the 503 code is an error code but it signifies a temporary error. The client should try again later (in fact you can specify how much later using the Retry-After header).

A 200 code tells the client everything is normal and OK. So the user gets your nice maintenance page telling them of a temporary outage, whereas their browser gets told that everything is fine. Now this might not be a problem for a user, but if the client is a search engine or a caching proxy then it will assume the maintenance page is the new valid content for the request.

If the Google crawler hits your site when you have the maintenance page up, it will update its search index with your “we’re down for now” message, rather than your cash prizes blog content. Your page rank will drop, your fat Adsense cheque will diminish and you’ll have to go back to your regular nine to five job in the city with people you don’t like in clothes you hate wearing.

So, as you can see, it’s important to return the correct status code. Here’s how to do it with Lighty and mod_magnet:

(more…)

10/1/2007 2 Comments

Xen on Ubuntu Edgy broken – Debian to the rescue

Xen LogoI found a great HOWTO on getting a Xen virtual machine up and running on Ubuntu Edgy. Unfortunately, Xen is pretty broken on Edgy. I managed to get one virtual machine running easily but on starting a second one, Dom0 panics and the whole box freezes. It’s been reported on launchpad but as it’s in universe I guess it’s just not a priority.

As a workaround, I installed the Xen kernels from Debian testing (Etch). These pulled in the Debian Xen userspace packages too, but they didn’t clobber anything. So now I have a couple of mostly Edgy virtual machines which I plan to use as a little test lab.

Whilst troubleshooting, I also found another good Debian specific Xen HOWTO with lots of good general advice too.

Etch is looking good. I work with Sarge a lot on servers and firewalls but I’ve not played with Etch much. I think I’ll get it installed in a Xen machine and have a play. I feel bad for not helping with its testing though.

7/1/2007 Add Comment

Encrypted partitions with Ubuntu/Debian

I figured out how to set up an encrypted partition on Ubuntu the other day. There are a bunch of ways of doing it but I found this to be the simplest. It should work on Debian too, since all the relevant packages are Debian ones anyway. In my example I’m encrypting an LVM partition (logical volume), but it should work with any device, including removable USB keys (see end notes). UPDATE: This is broken in Edgy but I figured out a simple fix, see below.

(more…)

6/12/2006 17 Comments

Opt-out of centralised NHS records

The government are centralising our medical information onto something called the “NHS Spine”. So our entire NHS medical histories will be moved to this system opening it up to general access for millions more employees of:

  • various government agencies including the police and social workers
  • private investigators, media organisations and other commercial entities.

Well, you apparently have the legal right to opt out of this “data rape”:

In June 2005, FIPR developed an opt-out letter to send to the Secretary of State. People who sent this off have been fobbed off. We now recommend that you opt out via your GP. Ask your GP to enter into your record the code 93C3 (“refused consent for upload to national shared electronic record”). You can also ask for your address and phone number to be kept off the NHS internal directory, and for your hospital records also to not be uploaded to central systems: see here for details. We encourage you to opt out even if you have nothing to hide; if only people who do have something embarrassing in their records opt out, then doing so will carry a stigma.

4/11/2006 Add Comment

IPSEC VPN problems upgrading to Ubuntu Edgy

I upgraded my home gateway firewall to Edgy today in the hope of fixing some SATA problems I’ve been experiencing. The new Edgy kernel might help – we’ll see.

Anyway, it went pretty well. Two runs (?) of apt-get dist-upgrade -u, a reboot and there I was.

Unfortunately I had two problems with my Openswan IPSEC VPNs. I’m not so sure if these count as bugs. I’ll be investigating further and reporting if so. Anyway, techie details follow…
(more…)

2/11/2006 1 Comment

News Sniffer: Revisionista

The latest News Sniffer project went live today: Revisionista. It tracks changes in corporate news articles and marks the differences. So you can choose a BBC news article and see how it’s changed since it was first created. Most changes are on breaking new articles which get updated as more information becomes available, but some changes are rather telling of policy.

Currently only the BBC is monitored, but it’s pretty easy for me to add support for any site with an RSS feed.

Some examples:
(more…)

10/9/2006 6 Comments

BBC Have Your Say

A while back I wrote a system to detect when the BBC censor comments on their Have Your Say website. It’s been running for a few months now and recently I wrote a fancy front end for it to allow easy browsing of the database.

It’s called ‘Watch Your Mouth’ and it’s part of my News Sniffer project, which will include other corporate media monitoring tools (coming soon).

It was broken for the duration of last week so no censored comments got detected, but it’s been working since and there is a large archive of censored comments too from the last few months.

I spotted some BBC ip addresses browsing the site already too.

17/8/2006 9 Comments

Bruce Schneier whistles white noise

The new ELER strip is about Bruce Schneier and links to a Chuck Norris style Bruce Schneier Facts Database system I wrote. It got linked from Boingboing, Metafilter, Reddit and even Bruce Schneier’s own blog!

As you might imagine, it’s getting trabigillions of hits right now. Lucky I wrote it in Ruby on Rails with its built in page caching which (after first render) serves as fast as the webserver can serve. And since I’m using Lighttpd too, that is very fast.

17/8/2006 Add Comment

Referrer Securer

Did you know that Firefox (and Epiphany) don’t send referrers when following a link from an SSL encrypted site? The target site cannot tell whether you clicked a link or typed the url in directly.

I don’t know about other browsers, but this seems like a sane behaviour.

16/8/2006 1 Comment

Can I Compost This?

Louisa and I am announcing our latest project: Compost This. It’s a directory of items with information about it’s compostability.

For example Tea and Flour can be put on the compost heap, but Bindweed and Walnuts cannot. And it’s not always a good idea to put Orange peel on there either.

For geeks: I wrote Compost This using Ruby on Rails, which is one of the best web frameworks I’ve used, and I’m really starting to love the Ruby language too. I’ll release the code soon as an example.

14/8/2006 2 Comments