Niall’s Weblog

For quite a long time now, OpenBSD has, among numerous exploit mitigation techniques, had a very strict mmap()-based malloc() implementation. Recently re-written by Otto Moerbeek, it is even harsher now. I find that this feature makes OpenBSD one of the best platforms to develop C programs on. If you have a double-free, use-after-free, off-by-one, or other typical mistake in your program, chances are OpenBSD’s omalloc will trip up on it eventually. Especially on a strict-alignment, long-pointer architecture like sparc64, running it under OpenBSD is a great way to gain confidence that your program is solid.

Anyway, recently Otto has made OpenBSD’s malloc even stricter. I upgraded my home machine over the weekend to the latest snapshot with these commits, and I’m currently running a bunch of Unworkable BitTorrent download processes on it to make sure my code still holds up. So far so good!

If you have some C code you care about, I’d recommend taking the time to run it under OpenBSD for a while - you might find you catch some bugs which even Valgrind missed. Enjoy!

I think food is one of the most basic things in life. Most people eat around three times a day, every day, for their entire life. How food tastes, and what it does to your body, would seem to me to be absolutely critical. Few people here in the San Francisco tech industry seem to cook for themselves, or pay much attention to the kind of stuff they put into their bodies, which I find quite bizarre. The culture of consuming vast quantities of heavily refined sugars (via sodas, donuts, pastries, icecream and all that junk) along with highly processed carbohydrates (pizza, bread, potato chips, etc) along with plenty of fatty, fried foods, seems completely insane. And then people wonder why they are overweight, and suffer from all manner of health problems - diabetes, heart disease, and of course deadly cancers.

While I don’t advocate going crazy and completely cutting out entire food groups willy nilly - probably such an effort will be unsuccessful in the long run, and almost certainly decrease your overall happiness and emotional well being - I do think its important for people to reclaim their connection to food and at least choose their own ingredients, cooking methods, and generally go through life being aware of the details of food. If ever details are important, its the details pertaining to food.

Anyway, on to this week’s recipe. Andronico’s had organic New York Steak on special, so I bought a nice cut and cooked it.

Allspice Steak with Broccoli and Shitake Mushrooms

• One USDA organic New York Steak
• 1/2 tsp. Jamaican Allspice
• One crown of broccoli
• 1/3 lb fresh Shitake mushrooms
• 1 tsp salt

Preparation:

A friend of mine, Chris, who is a fantastic cook (and chefs professionally) once told me the secret to good tasting steak is salt. While my steak isn’t as good as his, he’s certainly right about the salt. Rub both sides of the steak with plenty of salt, and then rub in the allspice. I cooked mine on a George Foreman, but you can easily do its under your own non-contact grill or even use a pan. I like my steaks to be done medium, so I cooked it for around 6-7 minutes. In the meantime, I finely chopped the shitake mushrooms and coarsely broke up the brocolli crown (I like brocolli coarse). Shitake mushrooms are well-known for not only their anti-cancer properties, but a whole myriad of health benefits such as anti-viral effects, thrombosis reduction, and so on. Brocolli, like all cruciferous vegetables, have tons of health-promoting qualities. Both of these foods taste great, too. So, dump your veggies in a pan with some water and sautee until they’re done to your liking. I like mine on the less-cooked side. Serve and enjoy!

DealExtreme are selling these “LED Power gyro wrist ball things for just under USD $8. They seem to be cheap clones of NSD Powerballs. The NSD Powerballs are much more expensive, though.

Apparently, these products can provide some RSI and wrist-strength benefits. One of my friends got one of the cheap DealExtreme versions, and says its a novelty at best. I think I’d probably be much better off spending $20 on the Captain’s of Crush no. 2 gripper (which I wrote about here), now that I can close the no. 1. Still, I’ll give it a go some time.

I’m a big fan of Baudelaire’s poetry. I am particularly fond of his interest in conveying beauty through typically repulsive imagery - e.g. Une Charogne which features vivid description of a decomposing human body.

While I enjoy reading the original French, I’ve recently discovered a fantastic online resource - http://fleursdumal.org/. This site not only includes the original French, but also numerous English translations. Not being a French native speaker, I find it illuminating to compare the various styles of translations - poetic and literal - with my own understanding of the French material.

Certainly one for the bookmarks, to be enjoyed on a rainy November night in San Francisco.

Andronico’s near my place have had a great deal on pork loin chops for the past few days, you can pick up a couple of pork chops for around $3. Pork chops are super easy to cook and are very tasty. When I cook red meat, I make a special effort to mix in some powerful and healthy vegetables - for example cooking with shallots or having sauteed broccoli and shitake mushrooms on the side. Onions are extremely good for cancer prevention, and shallots (which are essentially a kind of onion) are the most potently beneficial sort. Here’s what I did:

Roasted pork loin chops with shallots

• 2 pork loin chops
• Salt and pepper to taste
• Lots of shallots, peeled and sliced
• 1 tsp extra virgin olive oil

Preparation:

Preheat oven to 350 degrees F (175 degrees C). Coat baking dish with oil. Place meat in baking dish. Rub in salt and pepper to taste. Cover chops with shallot slices. Pour water in, making sure chops are not covered. Cook for 45-50 minutes, ensuring chops are cooked through and slightly brown. Serve with healthy veggies. Enjoy!

I run OpenBSD on all my machines. I think its a great operating system with excellent range of features and all the components fit together nicely. One of my favourite things about OpenBSD is the highly aggressive release schedule. While a stable release is cut every 6 months, Theo is producing complete, full builds of the system for most architectures from CVS HEAD on a nearly daily basis. The entire ports tree is baked into binary packages very frequently too - although since this is much more time-consuming it is more like a full package build appears on mirrors every week or two. Such releases are called ’snapshots’.

In any case, I don’t run OpenBSD ‘release’ or ’stable’ builds on any of my machines - I run snapshots everywhere. So I am frequently downloading new snapshots. While its not exactly difficult to mirror a directory via FTP by hand, I wrote a small Python program to do it for me. The Python program has a few nice options. It defaults to using ftp.openbsd.org as the mirror, but this can be trivially overriden by the -m flag. I typically use -m rt.fm, rt.fm I have found to be an excellent mirror for the USA. The program also automatically detects the architecture of the machine you are running on - but you can override this via the -a flag. It also doesn’t download the very large ISO images which are built along with the snapshots. Finally, once it has completed downloading everything, it will (if there is an MD5 file present) verify the MD5 checksums of each downloaded file.

This isn’t a complicated program, but I find it useful, and I thought I’d share. Here it is in its 80-odd line entirety (or download it here).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

#!/usr/bin/env python
# $Id: autosnap.py,v 1.5 2008/11/19 04:20:25 niallo Exp $
 
import fnmatch
import ftplib
import getopt
import hashlib
import os
import sys
 
MIRROR="ftp.openbsd.org"
PATH="/pub/OpenBSD/snapshots/"
DROP_DIR="."
 
ARCH=os.uname()[4]
# list of files not to download - globs supported
FILE_EXCEPT = ['*.iso*']
 
def usage():
    print >> sys.stderr, "autosnap.py [-a arch] [-d drop dir] [-m mirror] [-p path]"
    sys.exit(2)
 
def main():
    ftp = ftplib.FTP(MIRROR)
    ftp.login()
    ftp.cwd("%s/%s" %(PATH, ARCH))
    files = ftp.nlst()
    remove = []
    for p in FILE_EXCEPT:
        remove.extend(fnmatch.filter(files, p))
    for r in remove:
        files.remove(r)
    for f in files:
        print "fetching file %s" %(f)
        ftp.retrbinary("RETR %s" %(f), open("%s/%s" %(DROP_DIR, f), 'wb').write, 4096)
    ftp.quit()
    if 'MD5' in files:
        print "Verifying MD5sums"
        f = open("%s/MD5" %(DROP_DIR), "r")
        md5sums = {}
        for line in f:
            filename = line[line.index('(')+1:line.index(')')]
            if filename in files:
                hash = line.split('=')[1].strip()
                md5sums[filename] = str(hash)
        f.close()
        files.remove('MD5')
        good = 0
        for filename in md5sums.keys():
            f = open("%s/%s" %(DROP_DIR, filename), "r")
            d = f.read()
            f.close()
            m = hashlib.md5()
            m.update(d)
            digest = m.hexdigest()
            if digest == md5sums[filename]:
                print "%s OK" %(filename)
                good += 1
            else:
                print "%s FAIL" %(filename)
        print "%d/%d files verified OK" %(good, len(md5sums))
        if good == len(md5sums):
            sys.exit(0)
        else:
            sys.exit(1)
if __name__ == "__main__":
    try:
        opts, args = getopt.getopt(sys.argv[1:], "a:d:m:p:")
    except getopt.GetoptError:
        usage()
        sys.exit(2)
    for o, a in opts:
        if o == "-a":
            ARCH = a
        if o == "-d":
            DROP_DIR = a
        if o == "-m":
            MIRROR = a
        if o == "-p":
            PATH = a
    main()

This article is a followup to my previous post, Facebook apps in Python and Pylons part 1. I’m going to talk a little more about what is interesting about Facebook apps and how they work in practice. At the end, I provide a little code sample and a convenience decorator to save you some hassle.

Why write a Facebook app?

Even if you are pretty familiar with using Facebook, you would be easily forgiven if you didn’t fully understand what the capabilities of a Facebook application are, and how the flow works. Facebook applications essentially offer you:

• The ability to put your own content on a user’s profile.
• The ability to update a user’s news feed.

While there are a bunch more things you can do with your Facebook application - as described on the official “Anatomy of a Facebook application” page on developers.facebook.com, those two things are likely the most interesting to you.

How do I add content to the user’s profile and to their news feed?

This is the next question! Obviously, the basic answer is “by writing a Facebook app, stupid!”. Of course, you’re looking for a little bit more than just that. The first step is for the user to add your application. I’m about to drop a whole load of Facebook API jargon - specialised terms are highlighted in bold - the user can do this by visiting your canvas URL. You generate the canvas page from your callback URL, and include some FBML to give it an add to profile button. Once the person adds your application, Facebook will redirect them to your post add URL. From the post add hook, you can use the Facebook API to call setFBML to add content to their profile page, and publishUserAction to add stuff to their feed.

Its pretty trivial - but there is an additional caveat. Before you can do anything useful in a Facebook app, you must have a valid Facebook session. Basically, you want most of your entry points to only be loaded if a user has a logged in session, and if they don’t, you want them to be redirected to the login page. This ends up being a fair bit of boiler-plate code. I have written this method decorator to normalise the boiler-plate code into a single place, such that your Pylons controller methods will be handed a valid PyFacebook API object (from the PyFacebook library - see part 1). Here is an extremely basic code skeleton for a Facebook app in Pylons using my decocorator and PyFacebook:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

def require_login(f):
    ''' This decorator first checks to see
       if the user is authenticated.
       If not, it redirects them
       in the appropriate fashion to the
       log in page.  If they are authenticated,
       it sets up the PyFacebook Facebook
       object and passes it down to our wrapped method. '''
    def redirect(fb, url):
        if fb.in_canvas:
            log.info("doing fbml redirect 302")
            return '<fb:redirect url="%s" />' %(url, )
        else:
            log.info("sending a 302")
            response.status_int = 302
            response.headers['location'] = url
            return 'Moved temporarily'
    api_key = config['pyfacebook.apikey']
    secret_key = config['pyfacebook.secret']
    appid = config['pyfacebook.appid']
    auth_token = request.params.get('auth_token', None)
    fb = Facebook(api_key, secret_key, app_name='myapp',
             callback_path='/myapp/callback',
             auth_token=auth_token)
    if not fb.check_session(request) or not auth_token:
        log.info("got an unauthenticated session request")
        return lambda a: redirect(fb, fb.get_login_url())
    return lambda a: f(a, fb=fb)
 
class FacebookController(BaseController):
 
    def index(self):
        return 'Hello World'
 
    @require_login
    def post_add(self, fb=None):
        fb.auth.getSession()
        log.info("got a valid session from user %s", fb.uid)
        fb.profile.setFBML('<fb:wide></fb:wide>')
 
    @require_login
    def callback(self, fb=None):
        c.uid = fb.uid
        return render('/canvas.fbml')

Hopefully that is enough to get you started. I’ll be writing more about this subject so stay tuned. If you have any specific questions, feel free to post a comment!

I’ve been riding a fixed gear bicycle daily to work (10 mile commute round-trip) for over a year now. I love riding fixed gear. I wrote several articles about my experiences building my own fixie on this blog. Something I learned how to do pretty recently was to do a track stand. If you haven’t heard the term before, a picture speaks a thousand words - here’s someone doing a no-hands track stand:

Anyway I learned to do a basic track stand - that is standing, with one or two hands on the handlebars, a couple of months ago. My friend Nathaniel Cafolla, who in addition to being a talented marine scientist and all-around great guy has worked a fair bit as a courier in Dublin and has amazing knowledge of all things bicycle, taught me the basic technique over the summer. I must say its a lot of fun, very satisfying to do. Although I’m pretty good at it now, I’ve embarked on the next step - track stand while seated. I can do this successfully about 70% of the time. My goal is to eventually be able to do a no-hands track stand. I’m working towards that by doing seated track stands with just one hand on the handlebars. I’m pretty sure that within a month or two I’ll be able to do it with no hands.

So whats the point. Well, its nice not to have to take your feet off your pedals while stopped at a light. It also looks cool, and is fun. Its one of those little skills thats great to work on a little every day. I think it also improves balance to some degree, and forces you to relax into “the zone” - like most balancing feats, its easier if you clear your mind and don’t over think what you’re doing. I find that track standing on steep hills can require quite a of bit leg strength, too.

Just as riding a bicycle regularly is a nice thing to add to your routine, and riding a fixed gear is a nice variation, practicing a track stand is worth mixing in too. A little test of strength, co-ordination and balance is just the thing to spice up your commute while you’re waiting for a light to change! Enjoy.

Recently I have been working on a pretty simple Facebook application. I’ve found that the tough thing about writing a Facebook app is not the app per se, but figuring out what a Facebook app actually is, and how it is supposed to work! Anyway, I’m hoping to shed some light on the subject as I figure it out. This first post is mostly some background and description of the various Python implementations of the Facebook API.

Unfortunately, Facebook’s documentation is pretty bad, with a poorly maintained wiki full of out-of-date and plain misleading information. Also, it is all heavily biased toward PHP. On the one hand, this makes a certain amount of sense, since PHP is a very popular Web scripting language, and Facebook itself is implemented in PHP. However, Facebook apps are fundamentally a HTTP-level construct - that is, Facebook makes HTTP requests of various sorts to your application. HTTP is obviously language independent - any language capable of speaking HTTP can be use to implement a Facebook app. I feel that Facebook should focus more on the system at the HTTP level rather than having a load of PHP code and other examples. Anyway, I digress - what you really care about is writing Facebook apps in Python!

There really two distinct things going on in a Facebook app. One is answering HTTP requests from their servers in a specific way. As I wrote above, any language capable of speaking HTTP can handle this. The second is sending requests back to Facebook, in the form that they expect, with data correctly marshaled and so on. While you could implement this interface yourself, there are existing libraries which do all this marshaling and de-marshaling for you. The two I found for Python were minifb and PyFacebook.

I chose to go with PyFacebook, mostly because it seems actively maintained and has a little more documentation than minifb. The biggest gripe I have with PyFacebook is that it is very Django-centric. The second biggest gripe I have is that it is not very well documented. There is a tutorial on the Facebook wiki which is sadly lacking polish, but its still better than nothing.

The PyFacebook source and documentation talk about some Pylons middleware stuff. I tried to use this but found it to be more hassle than its worth - you can just use their provided Facebook class yourself and avoid some dicking about with middleware.py and so on. Actually all you need if you go this route is their __init__.py file. I dropped this in a subdir named ‘facebook’ in my Pylons lib directory. Then you can just do this to use it in your controller:

1

from intothebin.lib.facebook import Facebook

Then from your callback URL handler, you can do stuff like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

def callback(self):
    def redirect(fb, url):
        if fb.in_canvas:
            return '<fb:redirect url="%s" />' %(url, )
        else:
            response.status_int = 302
            response.headers['location'] = url
            return 'Moved temporarily'
    api_key = config['pyfacebook.apikey']
    secret_key = config['pyfacebook.secret']
    appid = config['pyfacebook.appid']
    auth_token = request.params.get('auth_token', None)
    fb = Facebook(api_key, secret_key, app_name='myapp',
            callback_path='/myapp/callback',
            auth_token=auth_token)
    if not fb.check_session(request) or not auth_token:
        log.info("got an unauthenticated session request")
        return redirect(fb, fb.get_login_url())
    fb.auth.getSession()

I’ll talk more about what all this stuff means, and what you actually need to do, in the next post. Stay tuned for more!

In the past, I have written here about typing, sore hands and window managers. I’ve had some experience with RSI or carpal tunnel-like symptoms. Basically, sore hands, seemingly brought on by a combination of martial arts training (particularly heavy bag punching) and non-ergonomic daily computer work.

I’ve tried a whole load of different things to try to address this - lots of rest, changes to how I use the computer, exercise, ibuprofen - some of which work to greater and lesser degrees. Overall, my hands have been pretty good for most of this year. However, one recent discovery has made a huge difference. Much greater than any of the other things. As part of the CrossFit training which I’ve really gotten into lately, I’ve started doing some Power lifting and Olympic weightlifting. I’ve also been doing handstand push-ups. In particular, these exercises really need strong hands and wrists and fairly randomly I went into a sports store about a month ago and bought an Everlast “extra-strength” hand gripper. While fairly puny relatively speaking, this thing has made a big difference to my hand and wrist strength. Its paid of in the gym, when lifting plates with fingers or anything, and doing hand-stand push-ups, and holding the barbells - my hands no longer feel like a limiting factor. Furthermore, I don’t get hand and wrist pain from typing any more.

I started reading about the hand-strength scene and in particular Iron Mind’s Captains of Crush hand grippers. They were only $20 on Amazon, and I had a voucher, so I went ahead and bought the no. 1. I really like this thing! Its way harder to close than the wimpy Everlast “extra-strength” ones. It feels very well made and I love that it has a rough finish, like a barbell, for conditioning the skin on your hands.

Anyway, these Captains of Crush things have quite a following - supposedly only 5 people in the world are able to close the no. 4 gripper. I have a new-found appreciation for hand strength - it really seems like something which is greatly neglected.

If you have issues with pain in your hands and/or wrists, I strongly recommend getting a hand gripper and trying to strengthen them. Captains of Crush brand grippers have impressed me a lot so far.