This was the second chicken I’ve killed myself (for food or otherwise) – the first one took a bit of mental preparation but this one was a bit easier.

He has a pretty good free range life, was killed quickly and we’ll waste very little of him (we’ve already had a soup made from cooking his carcass in the slowcooker).

Louisa has written up the experience in more detail on her blog.

Here are some photos – you might consider them a little grisly.

Wired: Some users are chafing at Google’s insistence that they provide real names. Explain the policy against pseudonyms.

Horowitz: Google believes in three modes of usage—anonymous, pseudonymous, and identified, and we have a spectrum of products that use all three. For anonymity, you can go into incognito mode in Chrome and the information associated with using the browser is not retained. Gmail and Blogger are pseudonymous—you can go be captainblackjack@gmail.com. But with products like Google Checkout, you’re doing a financial transaction and you have to use your real name.For now, Google+ falls into that last category. There are great debates going on about this—I saw one comment yesterday that claimed that pseudonyms protect the experience of women in the system. I felt compelled to respond, because I’ve gotten feedback from women who say that the accountability of real names makes them feel much more comfortable in Google+.

Notice that Horowitz did not answer the question, and what he did say was just ridiculous nonsense. Steven Levy at Wired didn’t seem to notice, or care.

Horowitz tries to make us think that we need our real name when making a financial transaction.  Thousands of years of currency proves that is not the case.

Horowitz then goes on to blurrily equate making a financial transaction with sharing videos of cats on Google+.

And then the cherry on the top: Google+ protects women.

This was the closest there was to a serious question in the whole interview and Horowitz just laughed out of his arse at it.

I emailed Steven Levy and asked him why, given the opportunity of interviewing Horowitz, he didn’t ask anything close to a serious question. It’s been over a week and I’ve had no response.

TL;DR: Don’t trust Steven Levy to report honestly about Google.

Hi Steven,

I hope you're well.

I read your article "Inside Google Plus"[1] the other day and I have a
couple of questions I hope you'll have time to answer,

I was surprised to notice that you didn't ask any questions about
privacy - Google+ unequivocally raises numerous serious privacy
concerns.  Why did you decide not to put any of these questions to
Horowitz?

Regarding your question about pseudonyms, Horowitz didn't really answer
your question. He kind of claimed that financial transactions cannot be
anonymous and that Google+ is somehow like a financial transaction.  To
be frank, his answer seemed just hand waving - why didn't you challenge
him on this harder?

Best regards,

John Leach

[1] http://www.wired.com/magazine/2011/09/ff_google_horowitz/

Rather than write support for this into our apps, we used Postfix to redirect the mail to our devs.

In our case, our staging deployments use a local installation of Postfix and the systems are generally not used by anything else, which makes this dead easy.

Firstly, write a rewrite map file, with the following one line of content. Call it /etc/postfix/recipient_canonical_map:

/./	devteam@example.com

Then configure Postfix like this (in /etc/postfix/main.cf):

recipient_canonical_classes = envelope_recipient
recipient_canonical_maps = regexp:/etc/postfix/recipient_canonical_map

Now all mail going through this relay will be redirected to devteam@example.com. It rewrites only the envelope, so the important headers are not changed.

Due to Puppet changes being idempotent, this is usually solvable by running puppet a few times (ew). Or you can do this properly by diligently setting all the dependencies for all of your packages on your apt-get update command, and having that depend on your source configs, but that’s pretty fiddly.

New versions of Puppet now have a feature called run stages that can be used to solve this problem. You put your source configs and apt update command in a run stage and tell puppet to run everything in that run stage before the other stages (there is an implicit run stage called main where everything goes by default).

This seems quite neat at first, but only the new parameterized classes can be set to be in certain run stages – so you end up putting things into a class just to put it into a run stage. It’s really not much better than being diligent with your dependencies for this problem – worse in many ways. (There is a long discussion about the implementation of run stages in the Puppet issue tracker that might help you understand the use case for them).

But there is a new relationship syntax to make setting dependencies much easier which I’m using to mass-depend all packages on my apt-get update command:

class apt {
  exec { "apt-update":
    command => "/usr/bin/apt-get update"
  }

  # Ensure apt is setup before running apt-get update
  Apt::Key <| |> -> Exec["apt-update"]
  Apt::Source <| |> -> Exec["apt-update"]

  # Ensure apt-get update has been run before installing any packages
  Exec["apt-update"] -> Package <| |>
}

Remember, Puppet is declarative – the dependencies set there get applied to all Apt::Key, Apt::Source and Package resources even if they’ve not yet been defined. (This example also assumes the only packages you’re defining are apt ones btw!)

In summary, run stages are hard and fiddly. You probably don’t need them. Learn how to use the new relationship syntax.

It’s neat but I wasn’t quite happy with the search options – I’ve always thought logs should be indexed with a real full text indexer. So, I knocked up a couple of scripts to do just that, as a proof of concept.

It uses rsyslog to receive the messages and write them to a named pipe.  A small ruby script called rsyslog-solr reads from the other end of the pipe and writes batches of the incoming messages to the full text indexer. I chose solr as the full text indexer as it has some very good options for scaling up, which will be necessary when indexing lots of logs.

Solr indexes, compresses and stores the messages sent to it, so we can retrieve the full text without having to store the original log. I wrote a custom schema definition optimized for this.

Then another script, rsyslog-solr-search, is used to query Solr and display the matching messages.

Querying is fun, for example I’ve searched all ssh authentication failures across all hosts and then searched on the originating IPs to see what other probes they made.

You don’t have to do advanced searches though, you can just display all logs from the last hour, or day or whatever.

One important note, any user that can generate logs that are sent to the system can cause a denial of service attack by sending specially malformed messages. This can be fixed by moving the formatting of the log entries from rsyslog into the ruby script, but I’ve not done it yet.

I’ve pushed the code to github under the MIT license. Feel free to improve it.

I’ve realised this is an illusion. Nobody is sitting in wait for my next blog entry. Nobody else has noticed I’ve not blogged in ages. Only I know I’ve not blogged in ages. And I certainly shouldn’t care about what I think.

So I’m breaking my accidental self-imposed blog embargo with this mundane entry which says nothing of import.

If you were hoping for something of more consequence then I make no apology, though your hopes disprove my above realisation, which is irksome to say the least.

Cory Doctorow [will] cost you $25,000 (£15,800) to get him to speak at your conference…

But what does Doctorow speak about? Well, ironically, he’s a proponent of giving away content for free as a business model – and for years he’s been telling the music industry to adapt to it. Am I the only one to see the irony in this?

I don’t see the irony. This is exactly what Doctorow recommends. Give your content away and charge to perform it. Give your music away and charge for your gigs. I bet the content of his slides is creative commons, and I bet the recordings of this talks are creative commons even. But if watching a video of him isn’t enough for you and you want him in person, then you pay for it.

It seems that Helienne Lindvall does not understand even the basic ideas of free culture.

UPDATE: Helienne Lindvall seems to have been misinformed anyway, as per this tweet from Doctorow himself:

@helienne, I’m afraid you were badly misinformed. I don’t have a “booker”, I don’t charge anything like the sum quoted, most talks are free

UPDATE: Doctorow has since written an interesting article rebutting Lindvall.

I wrote it in Ruby using the Rails 3 framework, with the dns records being served by the PowerDNS MySQL back end (though I’ll likely be switching it to use a custom back end using my powerdns_pipe library for more flexibility).

We’re building a big new cloud system over at Brightbox and we’ve been thinking how to provide convenient dns records for our customers.  We already have some basic integration but the resulting records are quite a mouthful. ipq.co is just a bit of an experiment to explore other ways of solving the problem.  There has already been some discussion over on Hacker News about possible applications (and implications) of the service – I’m interesting in how people will use it.

I’ve got some plans for other features which I’ll be adding over the next few weeks, and then I’ll be selling it to Google for low 7 figures, so watch this space.

UPDATE: Some ipq.co records were used to point at some phising sites and Google blacklisted the entire site.  I’ve requested a delisting with Google but that might take some time.  Any thoughts on how to avoid this in future?  I’m thinking check the IP with some well established  banlists on create (and possibly check them all regularly after that too).

So a write to a normal LV is just a write, but a write to a snapshotted LV (or an LV snapshot) involves reading the original data, writing it elsewhere and then writing some metadata about it all.

This quite obviously impacts performance, and due to device mapper having a very basic implementation, it is particularly bad.  My tests show synchronous sequential writes to a snapshotted LV are around 90% slower than writes to a normal LV.

Once copied and written, writes to the same chunk are only 15% slower.  Reads are super fast, only a 5% speed impact.

Still, not many usage patterns involve huge full speed sequential writes to a filesystem, so LVM is still useful in most circumstances.

I did some tests to see how writes to one snapshotted LV impacted the performance of writes to a completely separate normal LV. Does a snapshotted LV ruin the performance of all your other LVs? Yes, especially if you’re using the cfq disk scheduler. Switching to the deadline scheduler made things considerably better for the normal LV (but slowed writes to the snapshotted LV a little further).

I did these tests on a 12 disk hardware RAID10 system. The test is a synthetic benchmark so I urge you to do your own tests, but it’s safe to say that device mapper does not implement clever snapshotting like btrfs or zfs – don’t expect great performance from it.

Improving LVM Snapshot performance

There are a few ways to improve performance of LVM snapshots.  The most obvious one is the chunk size, which can be tweaked when creating the snapshot.  This controls the size of the data that will be copied and written on write operations.  The best setting will depend on lots of stuff, such as your RAID stripe size and your usage patterns.

There is an as-yet uncommitted patch that improves snapshot write performance a bit by being a bit clever about the disk queuing, but it’s still slow.

Also, device mapper supports non-persistent snapshots (i.e: lost after reboot), which should avoid having to write the change metadata to disk (which will save a lot of seeks and writes) but LVM doesn’t seem to support creating these yet.

Putting the snapshot device on a separate disk would help too – I’m not sure it’s possible with LVM, but device mapper does support it.

So I wrote a have_xml matcher using the rspec dsl which uses the libxml library to do the testing.  It’s so simple it’s not really worthy of a gem, so here it is (licensed under public domain).  The text check is optional and, to be honest, doesn’t belong here really.  It should be a separate matcher.

require 'libxml'

Spec::Matchers.define :have_xml do |xpath, text|
  match do |body|
    parser = LibXML::XML::Parser.string body
    doc = parser.parse
    nodes = doc.find(xpath)
    nodes.empty?.should be_false
    if text
      nodes.each do |node|
        node.content.should == text
      end
    end
    true
  end

  failure_message_for_should do |body|
    "expected to find xml tag #{xpath} in:\n#{body}"
  end

  failure_message_for_should_not do |response|
    "expected not to find xml tag #{xpath} in:\n#{body}"
  end

  description do
    "have xml tag #{xpath}"
  end
end

So, add that somewhere (usually spec/spec_helper.rb) and use it like this:

it "should include the xen_machine_id" do
  @xml.should have_xml('/domain/name', 'bb-example-001')
end

it "should include the network devices" do
  @xml.should have_xml "/domain/devices/interface[1]/ip[@address='1.2.3.4']"
  @xml.should have_xml "/domain/devices/interface[1]/mac[@address='aa:00:01:02:03:04']"
  @xml.should have_xml "/domain/devices/interface[1]/script[@path='/etc/xen/scripts/vif-bridge']"
  @xml.should have_xml "/domain/devices/interface[1]/source[@bridge='inetbr']"
end

Anyway, whilst holding my webcam to different parts of my body (if you ever use my webcam, wash your hands) I discovered that my eye, on its side, with the right lighting, and right shadows, and bad focus, through a webcam… looks kinda, possibly, a bit like girl bits.

It’s probably fair to say that, for a large proportion of the random strangers on Chat Roulette, the “Next” button is usually clicked in the hope of seeing a girl flashing some part of her body.

Combine these two seemingly unconnected facts together, and you get some of the reactions you see in my Eye Vagina video!  The music is “My Vagina” by NOFX. I edited out roughly 300 people jerking off.  The vid has had more than half a million hits on you tube. I’m expecting my share of their fat advertising profits any day now.

I recorded it using recordmydesktop and edited it using Pitivi (which actually had some very annoying audo sync problems I had to jump through hoops to avoid, which was a shame).

Funding your content through advertising is hugely inefficient. Of the people who visit your site, usually only a tiny proportion click on (or notice) an advert, and only a tiny proportion of those then spends any money.  So a tiny, tiny proportion of your visitors give any money to your advertisers. So money filters down this system in tiny margins.  Then, at the bottom of the system, a tiny amount of the profits from the income covers the cost of advertising.  Then this money moves back up the system to you, usually via your advertising agent who takes a nice cut (I’ve heard Google pass as little as one twelfth onto the publisher in some cases).

And this doesn’t consider the costs of the advertiser choosing and designing the ad or the tonnes of bandwidth and gatrillions of CPU cycles used to serve the actual adverts.

It also does not consider externalities, such as pollution. Advertising is mind pollution. Advertising is designed to affect the behaviour of people for the benefit of the advertiser.  Why would anyone willingly expose themselves to something designed to steal their attention?

You might argue that advertising creates value – some viewers choose to buy when otherwise they wouldn’t have. But what of the huge proportion of people who just had their attention stolen? No value was created there.

Because not everyone is suckered in by it, advertising squanders billions of hours of attention every day to produce nothing.

Begging

Walking down a street in town I might get approached by a beggar who needs money to eat.  In the UK we have the Vagrancy Act of 1824 which prevents these people from harassing me. They can be punished by detention, hard labour and whipping apparently.

Advertising is just a company harassing me for money to eat.  They’re just better funded.  I believe we should be able to detain and whip their advertising departments.

Unobtrusive payment

But seriously, advertising is a broken method of paying for stuff.  If we could unobtrusively pay for content on the Internet, I’m sure enough people would do so to more than cover costs of production.

We need good, unobtrusive payment methods and a change in culture to pay for good content would follow.  We can work on changing the culture now though: support sites that are ad-free (or have ad-free subscriptions).  I pay for a couple of ad-free subscriptions myself – be the change you want to see in the world.

I look forward to the day when a site with advertising is a clear signal that nobody would pay for it otherwise. My ad-blocker could then just block the whole site to save my wasting my attention.

Full Disclosure

My company does a (very small) amount of advertising, though I’m not involved directly in it. I don’t know how well it performs. We also sponsor conferences and events, which is of course advertising. I also help run a couple of web sites that make money from advertising.

From a viewer’s perspective, I hate advertising. From a publisher’s perspective, I can make a few quid from it and kinda just hope the pollution isn’t so bad (we do not allow annoying flashing adverts, and block ads from particularly evil corporations whenever we can but frankly, I’m mostly getting by on cognitive dissonance).  I’ve not thought about it until now, but from an advertisers perspective, it’s of course nice to get new customers (though I’m not sure of the “quality” of the custom we get via advertising – I’m now interested in investigating this further).

I’m in no way dependent on any of my income from advertising, so it’s hard to speak from these perspectives.

Update: Poor people

I’m basically saying that because adverts are not well targetted, the majority of advert views are wasted. They’re mind pollution.

But in order for adverts to get more accurate, the ad companies need to collect personal information about us: what we do online, what we like etc. So we’re supposed to hand over our privacy, just so we can ethically view “free” stuff on the Internet?

Let’s suppose advertising becomes perfectly targeted. Every advert you see is something you really can’t do without and something you can afford. Wouldn’t this mean you buy everything you get shown? Wouldn’t this mean you’d run out of money?

Is it unethical for poor people to view ad-supported online content if they can’t afford anything being advertised? However well targeted the ads are, they have no money to spend so it’s completely fruitless. Perhaps ad supported websites should ban public library Internet addresses – poor people are reading for free!

Discussions Elsewhere

• Rob Sayre: Why adblockers work
• Brian Carpet: Advertising is devastating to my well being
• Comments on Hacker News

Being a self-confessed full text indexing nerd and a Ruby-lover, I wrote Xapian Fu: a library to provide access to Xapian that is more in line with “The Ruby Way”.

I started writing Xapian Fu exactly a year ago today but left it for a couple of months, then restarted work on it on the train on the way back from the 2009 Scotland on Rails conference.  Development was test driven, so it’s got an extensive test suite (using rspec).  Documentation is in rdoc and is quite detailed.  As of the latest version, it supports Ruby 1.9 too.

Xapian Fu basically gives you a Hash interface to Xapian – so you get a persistent Hash with full text indexing built in (and ACID transactions!).

Example

For example, create a database called example.db, put three documents into it and search them and print the results:

  require 'xapian-fu'
  include XapianFu
  db = XapianDb.new(:dir => 'example.db', :create => true,
                    :store => [:title, :year])
  db << { :title => 'Brokeback Mountain', :year => 2005 }
  db << { :title => 'Cold Mountain', :year => 2004 }
  db << { :title => 'Yes Man', :year => 2008 }
  db.flush
  db.search("mountain").each do |match|
    puts match.values[:title]
  end

There are of course a whole bunch more examples in the documentation.

Schema-less

The hard work of full text indexing and storage is of course done by the Xapian library, but I have added a couple of useful features.  One in particular is the ability to use symbols (or strings) as field names. Xapian has no real concept of fields, but you can store arbitrary data that it calls values in a numbered slot alongside each document.  Instead of making you deal with field numbers, Xapian Fu uses a hash function to convert your field names into numbers.  This means Xapian Fu is schema-less - you can add or omit fields whenever you like.  It’s useful to define fields when opening databases so that Xapian Fu can recognise them in searches or to give Xapian Fu some clues on the type of data you’ll be using, but it’s not necessary.

Efficient storage of fields for ordering

If you tell Xapian Fu what type of data you’ll be storing in your fields, it can store them more efficiently.  For example, if you don’t specify the type, integers will be converted to strings as is, so 354,441,945,266,899 becomes “354441945266899″ – that’s  fifteen bytes!  When you tell Xapian Fu that your field is going to be an Integer,  it will store them in double precision floating point format which is 8 bytes and can represent up to about 16 decimal digits.  Also, it’s stored in big-endian format, so Xapian can still use the field for ordering results. XapianFu will store Time objects like this too, so again, it’s size efficient and can be used for ordering results.

Since Xapian Fu now knows what type the field is, it can convert it back when you access it too, so you get an Integer or a Time object (rather than a String, which is how Xapian represents it internally). It currently supports Integer, Fixnum, Bignum (up to a certain size), Float, Time and Date.  You can add your own types easily by decorating your instances with special methods.

Stemming and stopping

Xapian has stemming support for loads of languages (via the Snowball library), but no stop word lists.  Xapian Fu uses the appropriate stemmer when you specify a language for your document or database and comes with stop word lists for 13 languages (also automatically used).  This means Xapian doesn’t have to index these common stop words, so you get faster indexing and search times, a smaller database and more relevant search results.

Xapian Fu also knows that your searches won’t work right unless you stem them too! It automatically stems queries using the database language (though this does fall down a bit if you have different documents with different languages in your database at the moment, but it can be disabled (and isn’t too difficult to add support)).

Will Paginate support

will_paginate is a pagination library for ActiveRecord (and other DB abstraction layers).  It has helpers for drawing page list interfaces.  Xapian Fu supports will_paginate by using the same method names in result sets (such as :current_page and :total_pages).  You can pass a Xapian Fu result set to the will_paginate helpers and you’ll get lovely page list interfaces (you need to handle accepting the parameters in your action and setting up the next search of course!)

Active Record support

Xapian Fu does not yet have an Active Record plugin (I’ll add one soon) but as Xapian Fu uses the :id field as the Xapian primary key by default, it’s trivial to use it in your Rails app right now. See the “ActiveRecord Integration” section of the README for code examples.  In this case, you probably don’t need to actually store any data in the Xapian database, just the index information (and the :id field of course, but that’s stored by default) – so you get a smaller database (though you still need to store fields that you want to group by (collapse) or order results with).

Multi-master replicated full text indexing service

Xapian Fu doesn’t do this. I’m designing something that might though :)

Getting Xapian Fu

It’s available in gem form from Rubyforge/cutter.  The code is on github here.  You’ll need the Xapian Ruby bindings installed – on Debian/Ubuntu this is the libxapian-ruby1.8 package.  The gem named xapian claims to provide Xapian and the Ruby bindings but it failed form me on 64bit.  The gem named xapian-full claims to provide the Ruby bindings without Xapian (you’ll obviously need to build and install Xapian yourself) but I’ve not used that either.  RPMs, source files and other downloads are listed on the Xapian downloads page.

There is also this weird kinda splash page I made, in some kind of attempt to host something about Xapian Fu on my own domain. Not really sure what real purpose it serves.

Ruby’s case statement calls the === method on the argument to each of the when statements

So, this example:

case my_number
  when 6883
    :prime
end

Will execute 6883 === my_number

This is all fine and dandy, because the === method on a Fixnum instance does what you’d expect in this scenario.

However, the === method on the Fixnum class does something different. It’s an alias of is_a?

That is cute, because it allows you to do this:

case my_number
  when Fixnum
    "Easy to memorize"
  when Bignum
    "Hard to memorize"
  end

But it won’t work as you might expect in this scenario:

my_type = Fixnum
case my_type
  when Fixnum
    "Fixed number"
end

This won’t work because Fixnum === Fixnum returns false because the Fixnum class is not an instance of Fixnum.

My workaround for this is to convert it to a string first. Not sure if that’s the best solution, but it works for me(tm).

my_type = Fixnum
case my_type.to_s
  when "Fixnum"
    "Fixed number"
end

go = Proc.new { sleep 24.hours }
self.wants :sedatation
begin ; nil ; end
case go ; where "no" ; nil ; end
self.wants :sedatation
self.get '/airport'
self.put '/airport/plane'
before self.insane? do
  3.times { hurry! }
end
return if self.can_control? :fingers
return if self.can_control? :brain
5.times { "no" }

I recorded me singing it, which is kinda stupid tbh.

I used mencoder to convert this to something Youtube found tasty. Like this:

mencoder -ss 15 -endpos 1:18 -vf pp=al:f,scale=480:360 -oac mp3lame -ovc lavc -lavcopts vcodec=libx264:mbd=1:vbitrate=2000 MOV01362.MPG -o MOV01362.x264


Also, pimp for another Geek/Ukelele project: Ukepedia, all 3 million Wikipedia articles one song at a time

More connections will use more RAM, but how much?  We don’t want to overcommit, as the connection tracker uses unswappable memory and things will blow up. If we set aside 512MB for connection tracking, how many concurrent connections can we track?

There is some Netfilter documentation on wallfire.org, but it’s quite old. How can we be sure it’s still correct without completely understanding the Netfilter code? Does it account for real life constraints such as page size, or is it just derived from looking at the code? A running Linux kernel gives us all the info we need through it’s slabinfo proc file.

We can peek at how the kernel is using RAM using the proc file /proc/slabinfo and clear this up.  The nf_conntrack entry from here tells us, on one particular firewall, that there are 26,702 active entries (or objects), that each object is 304 bytes in size and 13 of them fit in each slab (and that each slab is 1 kernel page).  So we know that conntrack entries take up 304 bytes each.  But if we’re going to be accurate, then we have to account for the overhead of the kernel page size.

The Linux kernel uses a slab memory allocator, so rather than allocating 304 bytes every time a conntrack entry is needed, they are allocated in “slabs” of one or more kernel pages which reduces memory fragmentation and improve performance.  When they’re done with, they’re not immediately freed – instead the memory is reused the next time another object of the same type is needed.

In the kernel we’re using, the page size is 4096 bytes. As slabinfo told us, 13 nf_conntrack objects fit in each slab and each slab takes up 1 page. 13 objects of 304 bytes is 3952 bytes in total, which leaves 144 bytes of waste per slab.  So every 13 objects we waste 144 bytes. So a nf_conntrack object consumes about 316 bytes on this box, giving us almost 1.7 million entries for our 512MB.

You can get your kernel’s page size with the command: getconf PAGESIZE. The slabtop program, installed on most modern GNU/Linuxes, shows the info from /proc/slabinfo it in a pretty table and lets you sort the values.

For those of your with short attention spans, I finally get started with the talk at about 2mins 30, and start singing the first article, Otitis Media, at about 7mins.