Caius Theory

Now with even more cowbell…

Install capybara-webkit gem on Ubuntu

Dear future Caius searching for this issue,

The apt package you need to install to use the capybara-webkit rubygem on ubuntu (tested on 10.04 and 11.10) is libqt4-dev. That is, to gem install capybara-webkit, you need to run aptitude install libqt4-dev.

Yours helpfully,
Past Caius

End Of An Era

I remember watching Steve Jobs' commencement speech for the first time and being fairly touched by the three stories he told in it. The major one that resonated with me at the time, and has since made more sense to me, is the first story he tells about joining the dots later on when you're looking backwards. To quote from it:

Of course, it was impossible to connect the dots looking forward when I was in college. But it was very very clear looking backwards ten years later. Again, you can't connect the dots looking forward, you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future.

It made me think back to a time when I was a teenager and someone far older and wiser than me told me one way to think about the future and how what we do today affects where we end up. He described it as holding a piece of string in your hand, with the other end dangling free. The end of the string not attached to you is the future, and where you're holding it is the present. As you do things in life your hand moves around, flicking the string around and amplifying the movements of the present in the future.

I think what he was trying to impart by telling me that story was to be careful not to do anything too extravagant in the present (like get expelled from school for instance), so as not to affect the future too much. Which I never really took on board at the time, but now I look back at the dots of my (some would say) rather short life so far, and connect them to see how some things influenced other things and how everything works out in the end. But that sometimes you need to push yourself or do something unexpected to get there.

Looking Backwards to go Forward

When I look back over the last few years of my life, I find the dots quite amazing to connect. First there was BarCamp Sheffield 2007 where I met Dom, and there was Twitter and BarCamp Manchester & Leeds where I met geeks like Rahoul, John and Jeremy both online and offline. And then in June 2008 I moved to Leeds, sharing a house with Dom and quite quickly ended up being hired into what was basically my dream first job at Brightbox.

And that's where I've been working for the last 1198 days, having far too much fun, solving weird, wonderful and sometimes downright frustrating problems, all with fantastically awesome and hilarious colleagues. And working on a massive variety of things, from tiny utilities to the newly launched Cloud Platform.

And today I flick my wrist ever so slightly more than normal by leaving Brightbox, without really knowing where the future end of the string will eventually end up, but knowing that I'll look back in a few years and see dots connected that I can't imagine today. I truly don't think I could have had a better job to start off as a professional geek, and especially want to thank John and Jeremy for hiring me and helping me start my career in the best way possible.

As for my next challenge, I'm solving interesting problems in the online advertising world with Cristiano, Dom, Melinda and Rahoul. And we're bloody amazing together. :-)

Defining Ruby Superclasses On The Fly

Any rubyist that's defined a class should understand the following class definition:

class Foo < Object
end

It creates a new Constant (Foo) that is a subclass of Object. Pretty straightforward. But what you might not know is that ruby executes each line it reads in as it reads them. So we could do the following to show that:

class Foo < Object
  puts "we just defined object!"
end

And get the following output when we run that file:

# >> we just defined object!

So.. we know ruby is executing things on the fly whilst defining classes for us. How can we use this for profit and shenanigans?! (Err, use this for vanquishing evil and creating readable code I mean. Honest.)

How about if we've got a couple of opinionated people who like to think they've got the biggest ego in the interpreter? The last one to be loaded likes to have any new people ushered into the interpreter to be a subclass of themselves. Lets abuse a global for storing it in, and use a method to choose that on the fly when creating a new class.

def current_awkward_bugger
  $awkward_bugger
end

class Simon
end
$awkward_bugger = Simon

class Fred < current_awkward_bugger()
end
Fred.superclass # => Simon

class Harold
end
$awkward_bugger = Harold

class John < current_awkward_bugger()
end
John.superclass # => Harold

Fred.superclass # => Simon

Okay, so that looks a bit different to normally defined classes. We create Simon, assign him to the awkward bugger global then create Fred, who subclasses the return value of the current_awkward_bugger method which happens to be Simon currently. Then Harold muscles his way into the interpreter and decides he's going to be the current awkward bugger, so poor John gets to subclass Harold even though he's defined in the same way as Fred. (And as you can see on the last line, Fred's superclass is unchanged even though we changed the awkward_bugger global.)

On a somewhat related note there's a lovely method called const_missing that gets invoked when you call a Constant in ruby that isn't defined. (Much like method_missing if you're familiar with that.) Means you can do even more shenanigans with non-existent superclasses for classes you're defining.

class Simon
end
class Harold
end

class Object
  def self.const_missing(konstant)
    [Simon, Harold].shuffle.first
  end
end

class Fred < ArrogantBastard
end
Fred.superclass # => Simon

class Albert < ArrogantBastard
end
Albert.superclass # => Harold

So here we're choosing from Simon and Harold on the fly each time a missing constant is referenced (in this case the aptly named ArrogantBastard constant.) If you run this code yourself you'll see the superclasses change on each run according to what your computer picks that time.

Experimental Procrastination

A while ago I read a blog post that Elizabeth N wrote, on the value of writing self-serving code. Ever since I've been moderately aware of when I've written self-serving code, usually either at hackdays, or just little projects where I'm either experimenting with something or just bashing out a new idea.

In fact, I even wrote about one of my recent "self-serving" projects I bashed out in an evening, TweetSavr. It has no tests, was written moderately quickly and not refactored immensely (here's hoping the git history backs me up on that! I certainly didn't knowingly majorly refactor it at any point at least.) But it "scratched an itch" and solved the problem I had, and it works for the limited use case I need it to, so it's a success as far as I'm concerned.

More recently a friend remarked to me in a private conversation that everyone needs to procrastinate occasionally, to save them going "stir crazy". Whilst I agree with what she said, everyone needs to switch gears and do something that perhaps you shouldn't, or that won't directly contribute to completing your current task, I couldn't help but draw a link between procrastinating and writing self-serving code.

Now I'm a programmer, it's what I did for a hobby through school, it's what I leapt into a career doing when I was offered the chance and even when I've had a particularly exhausting week, it's something I'll eventually turn back to. But I realised that often when I procrastinate, I do so by writing self-serving code. My creative output or process if you will is to create things digitally on the computer, be that a web app, hacky script to let me do something I'm not supposed to be able to with someone else's application, just dicking around or exploring whatever tidbit of interesting info/behaviour in a language or library someone's just shared on IRC.

Aside: I've often joked (semi-seriously) that if/when I have enough cash in the bank to not have to actually have a "day job", I'll just spend all day building the random ideas that get tossed out on IRC instead. Quite often the smaller things I code up anyway already of an evening and they end up in my gists.

Now it's unhealthy and counter-productive to want to program 24/7, at least in the long term. (Doing the odd 24 hour hackday event here and there can mean winning fun prizes to play with however.) And sometimes all that you need to do to solve a problem you've been butting your head against for the last couple of hours is to get off the damn computer. (I usually find taking a shower makes my subconscious reveal the answer it's been quietly computing and hey presto, I know how to solve the problem properly!) At other times it just requires changing gears and flexing a different part of your brain muscle. Like say, writing self-serving code. And procrastinating by doing so.

I'm not entirely sure what the point of this thought process is, or if there can really be a point to it, but it really intrigued me drawing a link between procrastinating and writing self-serving code. I can imagine other creatively minded people might do much the same thing, an artist just sketching for the sake of sketching, or a writer taking a couple of hours off from her next novel to write a short story for her blog. (That last one is actually something a friend's done, go read her short stories, they're quite good. Start from the bottom though.)

And of course sometimes you just need to vegetate and read facebook (or twitter), but constructive procrastination does serve a real purpose I think, and can be quite useful as well.

Install GCC-4.2.1 (Apple build 5666.3) with Xcode 4.2

As of Xcode 4.2 Apple have stopped bundling GCC with it, shipping only the (mostly) compatible llvm-gcc binary instead. The suggested fix is to install GCC using the osx-gcc-installer project. However, I wanted to build and install it from source, which apple provides at http://opensource.apple.com/.

You should already have installed Xcode 4.2 from the app store, then basically the following steps are to grab the tarball from the 4.1 developer tools source, unpack & compile it, then install it into the right places.

Update 2016-07-03: I'd suggest just using homebrew to install this these days:

brew install homebrew/dupes/apple-gcc42

Instructions

# Grab and unpack the tarball
mkdir ~/tmp && cd ~/tmp
curl -O http://opensource.apple.com/tarballs/gcc/gcc-5666.3.tar.gz
tar zxf gcc-5666.3.tar.gz
cd gcc-5666.3

# Setup some stuff it requires
mkdir -p build/obj build/dst build/sym
# And then build it. You should go make a cup of tea or five whilst this runs.
gnumake install RC_OS=macos RC_ARCHS='i386 x86_64' TARGETS='i386 x86_64' \
  SRCROOT=`pwd` OBJROOT=`pwd`/build/obj DSTROOT=`pwd`/build/dst \
  SYMROOT=`pwd`/build/sym

# And finally install it
sudo ditto build/dst /

And now you should have gcc-4.2 in your $PATH, available to build all the things that llvm-gcc fails to compile.

TweetSavr

I've had a dream for a while. A simple webapp that takes the last tweet in a conversation and outputs that conversation in chronological order on a page you can link to forevermore. Occasionally I'll google to see if anything new's turned up, but they all seem to do far more, require the start and end tweets or are covered in ads.

So one friday evening I just built it. It's called TweetSavr. It's very simple—to the point the error page is just a standard 500 server error page currently. It fetches, caches and displays a conversation, given just the last tweet in said conversation.

KISS extends to the interface as well, I'm quite a fan of URL hacking to use webapps, so TweetSavr works on that basis as well. The homepage sort of has some help telling you how to use it, but you basically take the (old-twitter) URL of the last tweet and paste it after tweetsavr.com in the address bar. Eg, http://tweetsavr.com/http://twitter.com/ElizabethN/status/19766711653765120. It'll then redirect you through to the actual page for that conversation. You can also put just the status id on the end of the URL, http://tweetsavr.com/19766711653765120 and hey presto, it loads.

The caching layer is moderately rudimentary, after fetching a tweet that isn't in the cache it writes out a hash of data for that tweet into a yaml file. And when looking up a tweet it checks to see if that file exists, reading it in from disk if it is. Bonus side-effect is it builds up a corpus of tweets as yaml files on disk.

It lives on the internet at http://tweetsavr.com/ and the source is on github at http://github.com/caius/tweetsavr

Side note: isn't it wonderful what we can create given just a few hours, a server somewhere in the cloud, and an idea? Never ceases to amaze me what can be built in just a short amount of time, even the dead simple things.

#to_param and keyword slugs

Imagine you've got a blogging app and it's currently generating URL paths like posts/10 for individual posts. You decide the path should contain the post title (in some form) to make your URLs friendlier when someone reads them. I know I certainly prefer to read http://caiustheory.com/abusing-ruby-19-and-json-for-fun vs http://caiustheory.com/?id=70. (That's a fun blog post if you're into (ab)using ruby occasionally!)

Now you know all about how to change the URL path that rails generates—just define to_param in your app. Something simple that generates a slug consisting of hyphens and lowercase alphanumerical characters. For example:

# 70-abusing-ruby-1-9-json-for-fun
def to_param
  "#{id}-#{title.gsub(/\W/, "-").squeeze("-")}".downcase
end

NB: You might want to go the route of storing the slug against the post record in the database and thus generating it before saving the record. In which case the rest of this post is sort of moot and you just need to search on that column. If not, then read on!

Now we're generating a nice human-readable URL we need to change the way we find the post in the controller's show action. Up until now it's been a simple @post = Post.find(params[:id]) to grab the record out the database. Problem now is params[:id] is "70-abusing-ruby-1-9-json-for-fun", rather than just "70". A quick check in the String#to_i docs reveals it "Returns the result of interpreting leading characters in str as an integer base base (between 2 and 36)." Basically it extracts the first number it comes across and returns it.

Knowing that we can just lean on it to extract the id before using find to look for the post: @post = Post.find(params[:id].to_i). Fantastic! We've got nice human readable paths on our blog posts and they can be found in the database. All finished… or are we?

There's still a rather embarassing bug in our code where we're not explicitly checking the slug in the URL against the slug of the Post we've extracted from the database. If we visited /posts/70-ruby-19-sucks-and-python-rules-4eva it would load the blog post and render it without batting an eyelid. This has caused rather a few embarrassing situations for some high profile media outlets who don't (or didn't) check their URLs and just output the content. Luckily there's a simple way for us to check this.

All we want to do is render the content if the id param matches the slug of the post exactly, and return a 404 page if it doesn't. We already know the id param (params[:id]) and have pulled the Post object out of the database and stored it in an instance variable (@post). The @post knows how to generate it's own slug, using #to_param.

So we end up with something like the following in our posts controller, which does all the above and correctly returns a 404 if someone enters an invalid slug (even if it starts with a valid post id):

def show
  @post = Post.find(params[:id].to_i)
  render_404 && return unless params[:id] == @post.to_param
end

def render_404
  render :file => Rails.root + "public/404.html", :status => :not_found
end

And going to an invalid path like /posts/70-ruby-19-sucks-and-python-rules-4eva just renders the default rails 404 page with a 404 HTTP status. (If you want the id to appear at the end of the path, alter to_param accordingly and do something like params[:id].match(/\d+$/) to extract the Post's id to search on.)

Hey presto, we've implemented human readable slugs that are tamper-proof (without storing them in the database.)

(And bonus points if in fact you spotted I used my blog as an example, but that it isn't a rails app. (Nor contains the blog post ID in the pretty URL.) It's actually powered by Habari at the time of posting!

Abusing Ruby 1.9 & JSON for fun

Ever since I found out about the new hash syntax you can use in ruby 1.9, and how similar that syntax is to JSON, I've been waiting for someone to realise you can just abuse eval() for parsing (some) JSON now.

For example, lets say we have the following ruby hash, which could be generated by a RESTful api:

thing = {
    :person => {
        :name => "caius"
    }
}

If we pull in the JSON gem and dump that out as a string, we get the following:

jsonstr = thing.to_json
# => '{"person":{"name":"caius"}}'

That's… not quite what we wanted. It's not going to turn back into valid ruby as it is. Luckily javascript will parse objects without requiring the attributes to be wrapped in quotes, eg: {some: "attribute"}. We could build a JSON emitter that does it properly, or we could just run it through a regular expression instead. (Lets also add a space after the colon to aid readability.)

jsonstr.gsub!(/"([^"]+)": /, '\1: ')
# => '{person: {name: "caius"}}'

Okay, so now we've turned a ruby hash into a JSON hash, that can still be parsed by the browser. Here's a screenshot to prove that:

Valid JSON 'thing'

As you can see, it parses that into a valid JS object, complete with person and then (nested) name attributes. If we wanted to, thing["person"]["name"] or thing.person.name would access the nested value "caius" just fine.

Now then, we've proved that is successfully parsed into javascript objects by the browser, generated from a ruby hash. No great shakes there, that's fairly simple and has worked for ages. Now for my next trick, I'm going to turn that string of JSON back into a ruby hash, all without going anywhere near the JSON gem.

Some of you might have guessed what I'm about to do and have started hoping you've guessed wrongly — just for the record I don't condone doing this except for fun and games. The JSON gem is there for a reason ;) With that little disclaimer out the way, here we go!

thing2 = eval(jsonstr)
# => {:person=>{:name=>"caius"}}
thing2 == thing
# => true

Oh snap! We just turned javascript objects back into valid ruby objects, in one simple method call. And we'd be able to access the "caius" value by calling thing2[:person][:name], or creating OpenStructs from the hashes and calling thing2.person.name. Which is uncannily like the JS!

Updated 2011-02-07: Jens Ayton pointed out unquoted keys aren't strictly valid JSON, which is correct. Amended to say they're parsed as javascript objects instead, with no mention of it being valid JSON.

App Store Hidden Preferences

See the Update at the end before you get excited :(

Having just installed 10.6.6 to use the Mac App Store, I was slightly annoyed that it fills my dock with apps as I install them. I'm a bit strange, in that I use a hidden preference to make the dock uneditable (it stops me accidentally dragging an app off.) But that means I can't drag off the Mac App Store installed apps either.

Had a quick look through /Applications/App Store.app/Contents/MacOS/App Store with strings (love that tool) and noted a few strings that looked interesting. (There's a full list in this gist.) There wasn't anything that explicitly stated it stopped it putting anything in the dock, but I did notice an option that stopped it showing install progress in the dock.

Yank up a terminal window, bash out the following…

defaults write com.apple.appstore FRDebugShowInstallProgress -bool NO

…head back to the MAS and install another (free) app, and hey presto, it's leaving my dock alone! Hopefully that's all I needed to continue using my Dock as I like. (Hidden, and left alone.)

Update 2011-01-06:

Seems my joy was short-lived. I'd re-downloaded an app I'd already purchased and it just showed download progress in the MAS app, not in the dock. Installing new applications still shows up in the dock (annoyingly.)

I've been having a poke through how it all hangs together, and if it's possible to actually block downloads from the Dock or not. It doesn't look like there's a hidden preference to hide new apps from downloading in the dock, you can just disable the progress bars in the dock with prefs. The MAS.app seems to be codenamed "Firenze", with the "hidden" prefs being prefixed with "FRDebug".

As I understand it, the App\ Store.app invokes a binary inside /System/Library/PrivateFrameworks/CommerceKit.framework called "storeagent" to do the actual downloading/talking to the dock. From looking at the class-dump of storeagent it communicates with the dock to place a new type of DockTile. Interesting sounding methods to (potentially?) swizzle are -[DownloadQueue sendDownloadListToDock] and -[DownloadQueue tellDockToAddDownload:].

I've given up for now, but I reckon it should be possible to create a bundle that swizzles the right methods in storeagent to stop it placing the downloads on the Dock.

+[NSObject load] in MacRuby

If you've not heard of it, MacRuby is an implementation of Ruby 1.9 directly on top of Mac OS X core technologies such as the Objective-C runtime and garbage collector, the LLVM compiler infrastructure and the Foundation and ICU frameworks. Basically means you write in Ruby using Objective-C frameworks, and vice versa. It's pretty damn cool to be honest!

What is +[NSObject load]?

From the documentation:

Invoked whenever a class or category is added to the Objective-C runtime; implement this method to perform class-specific behavior upon loading.

This means when your class is loaded, and implements the load method, you get a load message sent to your class. Which means you can start doing stuff as soon as your class is loaded by the runtime.

The main place I've seen it used (and used it myself) is in SIMBL plugins. A SIMBL plugin is an NSBundle that contains code which is loaded (injected) into a running application shortly after said application is launched. It lets you extend (or "fix") cocoa applications with additional features. So you have this bundle of code, that gets loaded into a running application some point after it starts, and you want to run some code as the bundle is loaded - usually to kick off doing whatever you want to do in the plugin. This is where load becomes useful.

Here's a quick implementation that just logs to the console:

    @implementation MainController
    
    + (void) load
    {
        NSlog(@"MainController#load called");
    }

Now where does MacRuby come into this?

Well I came across a need to do the same in ruby, have some code triggered when the class is loaded into the runtime. Tried implementing Class.load but to no avail. Then remembered MacRuby is just ruby! And I can call any code from within my ruby class definition.

For continuity I still call it Class.load, but then call it as soon as I've defined it in the class. Eg:

class MainController
  def self.load
    NSLog "MainController#load called"
  end

  self.load
end

Of course, I'm not sure when the Objective-C method is called, it's probably after the entire class has been defined rather than as soon as load has been loaded into the runtime. So you might want to move the self.load call to just before the closing end.