Caius Theory

Now with even more cowbell…

Remove OS X Disk Password

I recently reinstalled a laptop and in doing so setup full disk encryption in a slightly strange fashion. The basic flow I followed was:

  1. Boot into Recovery mode (hold ⌘-R at boot)
  2. Erase the internal HD as HFS+ (Journaled, Encrypted) and set a disk password
  3. Install OS X onto the internal disk
  4. During setup, use Migration Assistant to copy clone containing previous install data from backup disk

This worked great in the end, once I'd recompiling various utilities I had installed. (Downside of moving from one CPU arch to another - can't just copy all compiled binaries over.)

However, I failed at step 2 above and entered "password" as my disk password since it was only intended to be temporary. Usually OS X's full disk encryption (FileVault 2) allows the machine users to unlock the disk, and not a standalone password. Due to the slightly odd way I setup the machine, I had the option of either using the disk password or my user account's password.

Having hunted around trying to find how to change or remove this disk password and leave only my users password, I finally stumbled across the magic incantations in an apple discussion thread asking How to disable "Disk Password" on boot?.

The magic incantations are as follows:

  1. List all the passwords that can currently unlock the drive

    Make sure there is a second password listed or removing the disk password will lock you out of the disk.

     $ sudo fdesetup list -extended
     ESCROW  UUID                                                    TYPE USER
             28376DDE-B6E1-48BE-A06F-4212067581D6    Disk Passphrase User
             4DBF8CEF-40F7-4F00-902F-A47AA643C656                 OS User caius
    
  2. Note the UUID of the "Disk Passphrase" entry, and remove that from the list

     sudo fdesetup remove -uuid 28376DDE-B6E1-48BE-A06F-4212067581D6
    
  3. List the passwords again to make sure the Disk Passphrase entry was removed

     $ sudo fdesetup list -extended
     ESCROW  UUID                                                    TYPE USER
             4DBF8CEF-40F7-4F00-902F-A47AA643C656                 OS User caius
    

Hey presto, only your user is left being able to unlock the disk.

#to_param and keyword slugs

Imagine you've got a blogging app and it's currently generating URL paths like posts/10 for individual posts. You decide the path should contain the post title (in some form) to make your URLs friendlier when someone reads them. I know I certainly prefer to read http://caiustheory.com/abusing-ruby-19-and-json-for-fun vs http://caiustheory.com/?id=70. (That's a fun blog post if you're into (ab)using ruby occasionally!)

Now you know all about how to change the URL path that rails generates—just define to_param in your app. Something simple that generates a slug consisting of hyphens and lowercase alphanumerical characters. For example:

# 70-abusing-ruby-1-9-json-for-fun
def to_param
  "#{id}-#{title.gsub(/\W/, "-").squeeze("-")}".downcase
end

NB: You might want to go the route of storing the slug against the post record in the database and thus generating it before saving the record. In which case the rest of this post is sort of moot and you just need to search on that column. If not, then read on!

Now we're generating a nice human-readable URL we need to change the way we find the post in the controller's show action. Up until now it's been a simple @post = Post.find(params[:id]) to grab the record out the database. Problem now is params[:id] is "70-abusing-ruby-1-9-json-for-fun", rather than just "70". A quick check in the String#to_i docs reveals it "Returns the result of interpreting leading characters in str as an integer base base (between 2 and 36)." Basically it extracts the first number it comes across and returns it.

Knowing that we can just lean on it to extract the id before using find to look for the post: @post = Post.find(params[:id].to_i). Fantastic! We've got nice human readable paths on our blog posts and they can be found in the database. All finished… or are we?

There's still a rather embarassing bug in our code where we're not explicitly checking the slug in the URL against the slug of the Post we've extracted from the database. If we visited /posts/70-ruby-19-sucks-and-python-rules-4eva it would load the blog post and render it without batting an eyelid. This has caused rather a few embarrassing situations for some high profile media outlets who don't (or didn't) check their URLs and just output the content. Luckily there's a simple way for us to check this.

All we want to do is render the content if the id param matches the slug of the post exactly, and return a 404 page if it doesn't. We already know the id param (params[:id]) and have pulled the Post object out of the database and stored it in an instance variable (@post). The @post knows how to generate it's own slug, using #to_param.

So we end up with something like the following in our posts controller, which does all the above and correctly returns a 404 if someone enters an invalid slug (even if it starts with a valid post id):

def show
  @post = Post.find(params[:id].to_i)
  render_404 && return unless params[:id] == @post.to_param
end

def render_404
  render :file => Rails.root + "public/404.html", :status => :not_found
end

And going to an invalid path like /posts/70-ruby-19-sucks-and-python-rules-4eva just renders the default rails 404 page with a 404 HTTP status. (If you want the id to appear at the end of the path, alter to_param accordingly and do something like params[:id].match(/\d+$/) to extract the Post's id to search on.)

Hey presto, we've implemented human readable slugs that are tamper-proof (without storing them in the database.)

(And bonus points if in fact you spotted I used my blog as an example, but that it isn't a rails app. (Nor contains the blog post ID in the pretty URL.) It's actually powered by Habari at the time of posting!

Education Network Restrictions

This is a re-run of an old post I took offline in an old server move and hadn't re-published.


Having been on two college systems and various university networks, I'm just amazed at the levels of freedom you have on some, and how locked down others are.

Take the first university network I ever used for example. It was pretty much totally open, to the point that I could game quite freely, and the administrator only picked me up because I was logged in as admin and not a normal user. (I didn't have an account for that machine.)

Going from that to my school network was a very big shock as it was moderately filtered through third party filtering software. This meant you couldn't go on the usual NSFW stuff, but still had access to other sites that could be seen as bad, such as proxy sites, or IRC java Clients for example.

Having moved from my old (slightly crass) college to my new one, its interesting how filtered this one is. You can't seem to go on a site with proxy or irc in the URL, except clean sites like the BBC or Wikipedia. The Proxy searching only came about through looking for web based IRC solutions.

Personally I think the universities have got it right. With all the students they have, they just limit the things they definitely have to, and allow everything else. (Blacklisting technique.) Both colleges seem to do the opposite - block everything until its verified and unblocked. (Whitelisting technique.)

The way I see it, the problem with the white listing technique is that people will always find a way around whatever restrictions are in place. For instance, I'm locked out of all of my web based email sites, so I can't email anyone. Its not the not being able to send that bothers me, its the not being able to save text that I've written in college to a website to then retrieve it from home that annoys me.

So how did I work around this restriction? Well I remembered that Google had bought Writely at some point recently, so one quick sign in later and I've got my own little area where I can save, organise and edit text based files. All I have to do when I get home is login, copy / paste into my email client and hit send.

One word that isn't blocked yet is blog, so I can still post this, and edit my posts. However, I'm still writing it in Writely and checking my markdown syntax is correct with Dingus. The writely interface is just that much nicer than notepad.

Validating Data with Regular Expressions in Ruby

I happened to be sent a link to the OWASP paper on Rails Security recently and started reading it. Partway in there's a section on Regular Expressions, which opens with the following line:

A common pitfall in Ruby's regular expressions is to match the string's beginning and end by ^ and $, instead of \A and \z.

Now I've never used \A and \z in my regular expressions to validate data, I've only ever used ^ and $ assuming they matched the start and end of the string. This becomes an issue with validating data in rails, because %0A (\n URL encoded) is decoded by rails before passing the string to your model to validate.

Testing our expectations

Lets say we want to validate the string as a username for our app. A username is 5 characters long and consists only of lowercase letters.

regex = /^[a-z]{5}$/

First we make sure it matches the data we want it to:

"caius".validate(regex) # => true

Excellent, that validated. Now we'll try a shorter string, which we expect to fail.

"cai".validate(regex) # => false

Once more, it behaves how we expected it to. The shorter string was rejected as we wanted it to be. Now, what happens if we test a string with a newline character in it? We'll make sure the data before the \n is valid, and then add some more data after the newline.

"caius\nfoo".validate(regex) # => true

Uh oh! That validated and would've been saved as a username?!

Lets have a look at exactly what's happening there, the $ matches the \n character, so the regex is only matching the first 5 characters of the string, and just ignores anything after the \n. As it turns out, this is exactly what we've asked the regex to match, but we didn't want this behaviour.

Now you might be thinking, "So what? someone can have a username with a newline in it." For starters this will probably display weirdly anywhere you use their username, but more importantly it opens your application to an injection attack. Suppose they took advantage of this by setting their username to include some javascript on the page which stole your login cookie and sent it to them. You view their account in the admin section and oh no! They can login as your admin account and do what they want.

Simple example of this is just having it output an alert dialog. (This is actually the code I'll use to test an application as its not malicious, but blindingly obvious if the javascript is executed or not.)

"caius\n<script>alert('hello')</script>".validate(regex) # => true

Ok, so that was the result we were expecting this time, although it's still not the outcome we wanted. Anytime their username is viewed (providing you aren't escaping the data to HTML entities) you'll see the following:

javascript alert dialog

The Solution

Having realised from our testing above that ^$ matches the beginning/end of a line in ruby not the beginning and end of a string, I hear you cry, "How do we make sure we're matching the entire string?!"

The answer is pretty simple. Just swap out ^$ for \A\z. Lets go ahead and try this with the same data as we have above, but with the modified regular expression.

new_regex = /\A[a-z]{5}\z/
"caius".validate(new_regex) # => true

That's a good start, the valid string still matches.

"cai".validate(new_regex) # => false

Looks like it's going well, invalid string is invalid.

"caius\nfoo".validate(new_regex) # => false

Oh Excellent! It's validating this one correctly now.

And just for consistency, lets test it with a more likely attack string.

"caius\n<script>alert('hello')</script>".validate(new_regex) # => false

Fantastic! We've fixed the security hole in our validation of the user's username.


If you want to actually run the code above you'll need the following at the start of the ruby script to patch the validate method into String.

class String
  def validate regex
    !self[regex].nil?
  end
end

Update: I had \Z in the new_regex rather than the \z it should've been. Thanks Ciarán.