Netfilter Conntrack Memory Usage

On a busy Linux Netfilter-based firewall, you usually need to up the maximum number of allowed tracked connections (or new connections will be denied and you’ll see log messages from the kernel link this: nf_conntrack: table full, dropping packet.

More connections will use more RAM, but how much?  We don’t want to overcommit, as the connection tracker uses unswappable memory and things will blow up. If we set aside 512MB for connection tracking, how many concurrent connections can we track?

There is some Netfilter documentation on wallfire.org, but it’s quite old. How can we be sure it’s still correct without completely understanding the Netfilter code? Does it account for real life constraints such as page size, or is it just derived from looking at the code? A running Linux kernel gives us all the info we need through it’s slabinfo proc file.
(more…)

My Ukepedia Talk at Barcamp Leeds 2009

Tim Dobson very kindly recorded and uploaded my talk on the Ukepedia at Barcamp Leeds last Saturday.

For those of your with short attention spans, I finally get started with the talk at about 2mins 30, and start singing the first article, Otitis Media, at about 7mins.

Live this Saturday at the Packhorse in Leeds, The Gillroyd Parade

My band, The Gillroyd Parade, are hosting an evening of acoustic music at the Packhorse Pub this Saturday (7pm to 11pm, 16th May). Supported by Ukelele Bitch Slap. Do come along, it’d be just dandy to see you.  Full poster here.

The Gillroyd Parade

April Fool: A man in Jalawla walked into a bar…

Medialens spotted that the BBC attributed a bomb attack on Monday in Iraq to “al-Qaeda”, with apparently little evidence.  They wrote to the BBC’s “man in Baghdad”, Hugh Sykes, and asked him “what is the evidence that al-Qaeda, rather than some other insurgent group, were behind the attacks”?.

Hugh’s answer genuinely made me think this was an early April Fool’s joke. In fact I’m still not sure Medialens aren’t making me look like an idiot:

No proof, but circumstantial evidence and reasonable presumption of AQI [al-Qaeda in Iraq] involvement – very much their modus operandum. Suicide attacks are their signature method, and this was a dramatic detonation suggesting a lot of explosive – again, very AQI.

And…who else would do this?

So, process of elimination, history of AQI attacks in Diyala etc.

And the logic of it Sunni Arab vs Iraqi Kurds. As a man in Jalawla told Reuters:

“Al-Qaida is targeting the Kurds because it believes that
we are involved in the political process and collaborating
with the Americans.”

This blows my mind. “very AQI” and “a man in Jalawla told Reuters”. “Who else would do this?”

As Medialens point out, the BBC claim they are “committed to evidence-based journalism”. Except they pick and choose when their committment applies, such as when they refused to report the use of banned weapons by US forces in their November 2004 assault on Falljuah.

My NWRUG Ferret Talk

I did a short talk on Ferret, the Ruby “Information Retreival Library”, at the North West Ruby Users Group last Thursday.  We had a bit of a theme too, with Will Jessop speaking about Sphinx and Asa Calow speaking about Solr.

I got to have a bit of a nosey around the Manchester BBC building too – though I was worried I’d open the wrong door and end up on TV. Didn’t fancy having to apologise to Jeremy Paxman.

Brightbox also sponsored some pizza, and gave away t-shirts and stickers like candy (there was no candy though).

My slides are available here, and contain a little example file system indexer. I made my slides with webby and S6 if you’re interested.

Women in Technology

Dom kicked up a women in technology debate again recently.  I’ve seen a few responses, from one chap who thinks women have achieved equality already to a woman who doesn’t think girl’s brains are generally good for “programming” – and someone else who thinks there isn’t a problem as long as you’re thick skinned enough to put up with a sexually hostile workplace.

The main gripe appears to be with “women only” conferences, such as the Women on the Web conference, organised by a group called Forward Ladies, or the Geek Girl dinners.

I think a fair summary of his, and some other commenters, opinion is that these “women-only” events don’t help the effort to get more women involved in technology. Comparing it to positive discrimination in many ways.

(more…)

Leeds Market Big Wigs

More Leeds Market photos here on my Flickr profile.

Techietubbies live video podcast

I’m joining Dom and Rahoul tonight on a live video broadcast of their Techietubbies podcast thing.

From the site:

“Techietubbies is a weekly podcast covering a multitude of subjects, from a round up of the week’s tech news, live callers, competitions, questions and answers… and beer :)”

Though I’m driving, so no tech news for me. I think it’s recorded if you can’t see the live thing.  It’ll be broadcast live here via ustream.tv

My native language

Severed head I’m currently reading Nudge, by Richard H. Thaler and Cass R. Sunstein. It says many psychologists and neuroscientists agree that we humans have two general types of thinking, intuitive and rational. Also known as automatic and reflective.  When dodging a ball thrown at you, getting nervous when your aeroplane hits turbulence or smiling when you see a cute cat the automatic system is working.  When doing some mathematics, or writing a blog post, you (mostly) use reflective.  Speaking native, or “first” languages uses the automatic.  Speaking a second language usually uses reflective.

I realised that having tinkered with computers heavily almost my entire life, a lot of my “computer skills” have shifted into the intuitive, automatic systems.  I obviously (hopefully) use the rational systems a great deal, but underlying it is definitly intuition – the gut feeling of where to go next to solve the problem.  I regularly come up seemingly random avenues of investigation that lead to gold and I couldn’t say with any certainty why I thought of it.  I’m assuming this is the same for most computer geeks (and chess geeks, cooking geeks, music geeks etc. :).  It’s become a native language for us.

I don’t think the average rational system can easily deal with very complex problems.  It’s great for some more-linear concentrated work or planning, but for big stuff with lots of parts – hard work.  I think I usually research and “pre-process” a bunch of material around a problem using my rational system, then my automatic system gets to work mulling over the bigger picture.  Then when I’m making rational decisions about it, I’m heavily informed by the intuition. Or sometimes just when I’m showering.

Anyway, not sure where I was going with this other than a “aren’t I great” blog post. The summary would be, don’t rely on your rational systems so much. Give the intuitive some good mulling time. And shower regularly.

Gravedigg: What will die next?

Gravedigg is like Digg, but rather than voting for pictures of cute cats or top ten lists of stuff, you vote on what you think will die or fail next.  Companies, celebrities, technologies… whatever.  So maybe you think the Perl programming language is on it’s way out very soon, or that Iceland is on its last legs or that Steve Jobs is boned.

Louisa and I put this together in just a few days, me coding and Louisa designing. Was a fun little project to do.

SAS and the R Programming Language

This New York Times article about the R programming language is pretty good, though there is a hilarious quote in it from proprietary software company that apparently make a similar product. Anne H. Milley, director of technology product marketing at SAS says:

“We have customers who build engines for aircraft. I am happy they are not using freeware when I get on a jet.”

That’s pretty funny. She’s basically saying

“It’s better to build important things with tools you can’t examine for yourself.”

SAS claim to have over 40,000 customer sites worldwide.  The news article claim 250,000 people use R regularly.  The difference here isn’t in the numbers of users, it’s that, with R, every user is a potential developer.  SAS can’t possibly compete with that.

Virtualized Storage Talk at WYLUG

I’m doing a talk tonight about virtualizing your storage with LVM on Linux at the West Yorkshire Linux User Group. Sorry about the short notice here (it was announced earlier in the week elsewhere though).

My mate Paul Brook is talking about RAID on Linux too.

Come along for the talk, or the beer, or the socialising – or all three.