07.29

Backups or The Dreaded `rm -rf /*`

At my startup, we have a moderately sane deployment environment. Check-ins are automatically pushed to a staging server (And i’m notified via notifo with this script). When we want to push code to our live server, we manually move things.

For a while that meant we had to run a

svn export

command by hand once a day or so. (we are under heavy development still)

Eventually we needed a couple more things done when we moved code to the live server (among them run google’s closure compiler to reduce our js footprint). So I wrote a script to handle these things. Apparently while testing, I got distracted and never finished it, because when i ran it, the code was this:

# ... snipped ...
#
# code to assure you're running as root
#

if [ -z $2 ]
    then
        loc= '/path/to/code'
    else
        $loc = $2
fi

# nuke old files
rm -rf $loc/*

# ... snipped ... 

Now, to the astute, you will notice there is a bug here. Can you find it? It’s in the assignment of

$loc = $2

It SHOULD be similar to the loc assignment above it.

loc=$2

Well. This throws an error, but happily keeps moving on down the script. Next question: what happens here when $loc is an empty string ”? That’s right. I just ran rm -rf /*as sudo on our main server.

I realized quickly what had happened. Errors whizz by about permissions, and finally I Ctrl+c’d it out of existence. Everything still in memory was intact. Our application was still running, I was still ssh’d in. So I took stock of my life, and tried to calm down. I had no idea what to do at this point; thoughts ranged from spending the next 12 hours slowly rebuilding everything (I had the latest copy of our code and db on my local computer, so our code was safe) to never returning my CEO’s emails again and abandoning our company. I initially did some fruitless googling for ‘restore rm -rf *’ even though I knew – short of serious hard drive-level manipulation – I was NOT getting anything back.

Then I remembered that we are using Jungle Disk to do daily and weekly backup. I figured out that Media Temple (our awesome host) allows us to reinstall our OS if something like… this happens. So first I checked that we had a recent backup. We did. So i went into our dasboard, and clicked on the button to ‘Revert to Default’ (MT’s oh-shit button). And waited. After that was done, the first thing I reinstalled was Jungle Disk and set it up to recognize itself. Then I simply ran Jungle Disk’s restore program, and voilà! I installed a few more programs that we needed, and we were 100% back up and running in about 3 hours.

We back up our /etc, /var, and /home folders daily and our entire file system weekly, so not too much data was lost. As it turns out, this doesn’t include our Postfix maildir, so we lost about a week’s worth of email (personally, I use gmail and it downloads all the email, so I didn’t lose anything). We have since fixed this to save email daily.

This story is another among many to remind you to back your shit up. Fortunately we ended up ok; thanks in no small part to Jungle Disk.

If you have any questions about our staging=>live process (which is now Idiot Proof©), or our backup strategy, drop me a line at hello at andrebluehs dot net.


06.19

Subversion and Notifo and Post-Commit

A service called Notifo just launched recently. They facilitate ridiculously simple push notifications. So I wanted to be notified when any of my developers checked in code, and what the message they sent was. This is useful for me because we all work remotely, and I can’t just walk over to their office to check in on them. So I wrote a couple-line post-commit hook to tell me these things. Then i figured other people would want it, so i made it prettier and put it up on github.

Pick up the script at github Notifo Post Commit Hook

You’ll need to change the NAME and APISECRET variables, and then it will work straight out of the box. You can test it without having to actually check anything in by doing cd’ing into your repo’s ‘hooks’ folder, and then run the command ‘./post-commit /path/to/your/repo <a revision number>’

Some highlights from the code:

Notifo requires that the data you send it be url-encoded (makes sense), but there is no command line utility for that, specifically. So I had to use a tiny little perl script I found on this stack overflow question that had the same url encoding from a bash script problem. I also put in handling an empty message (yes, some people suck).

Don’t forget to make sure you have curl installed. php5-curl (or equivalent apache plugin) will not do, you need to actually have the command-line tool for this to work. The other programs used i’m pretty sure come standard. Svnlook comes with a standard subversion command-line installation (which I assume you have if you’ve got a repo on there…).

Something i’d like to improve is to get the author and message in one call. However, i can’t figure out how to separate them with awk, or any other command line utility.

Notes on Notifo

I am super happy with this service. I’ve wanted to add push notifications to several of my projects for a while, but the barrier to entry with sending notifications is pretty steep. The only other comparable option is Prowl, but it’s api page is sparse, with no examples, and it’s only for iPhone (And costs $2.99). Also, they do not allow the flexibility of Notifo. With Notifo, you can send yourself a notification, you do not have to have a registered application or service. This is what allows my script to work, as well as the Chrome to Notifo link sharing extension written by Notifo co-founder Paul Stamatiou.

Email me at hello at andrebluehs dot net with any questions.


06.04

Coffin Nails and Native Mobile Apps

One of my startups, Usable Health, is making an iPad-specific version of our web app to be installed in restaraunts. (Side note: watch our CEO talk at Ignite Atl) Since we were going to set-and-forget these applications, we needed a way to prevent people from going to other sites (as this is a web app just running in safari). Then I came across this article by Alex Kessinger. And we realized:

We wanted a native iPad app. So we made a web app.

Alex’s article gets in to a bit too much detail for us. We really only need one line:

<meta name=”apple-mobile-web-app-capable” content=”yes” />

This one line of code transformed our web app (built for ipad size and features) into a native-looking app with no Safari toolbars on the top or bottom. Add it to the home screen, and you are off and running.

There are other features of that post that we could use, such as setting a home screen icon, but we don’t care about this, because our app should never be closed anyway while at an installation.

Here are screens of before:

after:

The total time from start to finish to get this working? 26 minutes. This includes me sharing the link, and our developer reading the article.

This is why i believe that for most applications, native apps are becoming obsolete. There are HUGE incentives for more adoption of this method:

  • no need for apple developer licence (cheap for small companies)
  • don’t have to deal with the app store

There are, however, cons:

  • hardware acceleration is spotty. games are mostly out.
  • hard to charge to download app. however, can charge for access.

For the people who say that “Well, it doesn’t work on other browsers”. That’s only half true. While this method of removing top and bottom bars only works on iPhone/iPad, the app still functions 100% normally in other mobile browser. Building a mobile app gets you 90% of the way, and adding that ONE LINE gets you the rest of the way for apple browsers.

Here is where you can try it on your ipad (or iphone/ipod touch): http://ablu.us/files/uh_ipad