Caius Theory

Now with even more cowbell…

SmartOS Recovery mount /usbkey

Recently I managed to hose a box in a perfectly self-inflicted storm of idiocy. Imagine a SmartOS server with the following issues:

Needless to say, this caused a tiny issue in the server doing what it's supposed to. Luckily I had access to a KVM remote console for the box and the following worked.

I brought the machine up, choosing the second option for recovery at the grub menu. Waited for a login prompt, then logged in with root/root.

Realised quite quickly that /usbkey must be persisted on the zones zfs pool otherwise the configuration would be lost after shutdown, so imported the correct pool, created a directory to mount into and then mounted the zfs share.

zpool import zones
mkdir /usbkey
mount -F zfs zones/usbkey /usbkey

Changing hostname in SmartOS Zone

Given a non-global zone in SmartOS that we want to change the hostname of, we need to make sure to edit the following files to change it:

A quick way to do that is with sed (renaming "fred" to "beth" here):

sed -e 's/fred/beth/g' -i /etc/hosts /etc/nodename

Then shutdown & start the zone (from my testing a restart doesn't apply it).

Changing root password in global zone

SmartOS mounts /etc/shadow from /usbkey/shadow so we can change the root password for the global zone after install. Here's how:

  1. Fire up a console or ssh session as root in the global zone
  2. Check the existing permissions on the file

     $ ls -l /usbkey/shadow
     -r--------   1 root     root         560 Oct 19 16:45 /usbkey/shadow
  3. Make the file writable

     $ chmod 600 /usbkey/shadow
  4. Fire up vi to edit the file

     $ vi /usbkey/shadow
  5. Edit the line containing root to change the crypted password. See shadow(4) if you need help with the format of /etc/shadow & use /usr/lib/cryptpass to generate a hash for the password you desire. (Remember to clean the bash history!)

  6. Save the file and exit vi

  7. Make the file readonly again

     $ chmod 400 /usbkey/shadow
  8. Double check permissions are correct on the file again

     $ ls -l /usbkey/shadow
     -r--------   1 root     root         560 Oct 19 16:49 /usbkey/shadow

Job done. Verify by logging in as root (invoking /usr/bin/login from an ssh session makes this easy to verify.)

Compiling SmartOS for AMD processors

There's a few community-provided patches for SmartOS that enable KVM on AMD processors amongst other things, and given the HP Microserver has an AMD processor, that's quite useful for turning it into a better lab server. The main list of so called "eait" builds was hiccuping when I tried to download the latest, and all I could find was a 20140812T062241Z image here.

The source code for the eait builds is maintained at, and you can see the patches applied on top of the normal SmartOS master by going to

So here's how to use SmartOS to compile a more up to date AMD-friendly Smartos!

  1. Grab the latest multiarch SmartOS image (which has to be used, or the compile will fail.) The latest at the time of writing was 4aec529c-55f9-11e3-868e-a37707fcbe86, so that's what I'll use.

     imgadm import 4aec529c-55f9-11e3-868e-a37707fcbe86
  2. Spin up a zone for us to build in (the Building SmartOS on SmartOS page has extra info about this):

     echo '{
       "alias": "platform-builder",
       "brand": "joyent",
       "dataset_uuid": "4aec529c-55f9-11e3-868e-a37707fcbe86",
       "max_physical_memory": 32768,
       "quota": 0,
       "tmpfs": 8192,
       "fs_allowed": "ufs,pcfs,tmpfs",
       "maintain_resolvers": true,
       "resolvers": [
       "nics": [
           "nic_tag": "admin",
           "ip": "dhcp",
           "primary": true
       "internal_metadata": {
         "root_pw": "password",
         "admin_pw": "password"
     }' | vmadm create
  3. Login to the created zone:

     zlogin <uuid from `vmadm create` output>
  4. Update the image to the latest packages, etc:

     pkgin -y update && pkgin -y full-upgrade
  5. Install a few images we'll need to compile & package SmartOS:

     pkgin install scmgit cdrtools pbzip2
  6. Grab the source code of the fork containing the patches we want, from arekinath/smartos-live

     git clone
     cd smartos-live
  7. Optional: Edit src/Makefile.defs and change PARALLEL = -j$(MAX_JOBS) to PARALLEL = -j8 to do less at once. (Microserver only has a dual core CPU!)

  8. Copy the configure definition into the right place and start configuration:

     cp {sample.,}configure.smartos

    (You'll probably get asked to accept the java license during configuration, so keep half an eye on it)

  9. Once configure has completed (which doesn't take too long, 15 minutes or so), start building:

     gmake world && gmake live
  10. Once the build is successfully finished, time to package an iso & usb image:

    export LC_ALL=C

Hey presto, you've a freshly built AMD-friendly SmartOS build to flash to a USB key / put on your netboot server and boot your Microserver from!