2012 Crossfit Open post-mortem, data-crunch & thoughts

The two things that attracted me to Crossfit when I started last July were that  it was extremely hard, and secondly, that it's very data-driven. People do workouts till they can barely hold a pen, and then they proceed to do so to note down time, reps, weight and whatever else can be measured or clocked.

The other thing I didn't really know about when heading in, is how open and welcoming and unpretentious the atmosphere is. Everyone really goes out of their way to welcome newcomers and to encourage anyone pushing themselves, no matter where they might rank compared to others. Nowhere is this as evident as in the Crossfit Open, the opening stage of the yearly Crossfit games.

The open is pretty unique in that any and all around the world can compete. You needn't belong to an affiliate; you can just do the workouts in your garage per instructions, upload a video to youtube, and get ranked along with everyone else. This means there can always be a cinderella story where some guy/gal comes out of nowhere and finishes on the podium.

That cinderella story will NOT be me. I did the open this year and ranked near the bottom on all events (the open consists of five events over five weeks), but that's ok. I headed into it with several goals: (1) Complete all five workouts at prescribed weight/movement, so I could at least post a score, (2) understand how I stacked up, so I could benchmark myself again next year, and (3) understand how I ranked vs others, so I know what areas are my weakest points. It's with this last goal in mind that I'm writing this post.

Data Analysis

The results of the open are posted on a leaderboard located here. Some rough observations:

• 61,000 people entered the open and submitted scores for round one. With each round a few dropped, and by round five, 33,000 people submitted scores. By this metric, I'm in the top half just by submitting five scores. Woohoo!. I'm not counting that though :-/
• In the NW region where I'm located, it scaled down as well, with 2001 scores submitted for round 1, and just shy of 1400 scores for round 5.

Since there's no export function for the table data (more on this later), what I did was used the NW region as a basis, and then took a sampling of that data at intervals of fifty (1st, 51st, 101st...) and plugged them into a table. This let me plot some curves for the data which are kind of interesting. lets have a look.

Workout 1: 

AMRAP burpees in 7 minutes. Anyone can do at least one, so we'd expect to see an S-curve of results, with a ramp in the middle and an upswing at the top end of 'elites' and a downswing at the low end for the weaker folk (like me). Here's what we see:

I'm WAY down the list in the bottom fifth percentile. Burpees are weak for me and I knew this going in. I even did 12.1 twice but only gained an additional 2 reps. Definitely one of my areas for improvement.

There's an interesting elbow in the curve at the top as well. Seems getting past 120 gets REALLY hard. Maybe there's a time limited where you need to focus on efficiency? a straight drop from touching at the top? Hmm...

Workout 2:

Snatches, in sets of 30 starting at 75#, and incrementing with each set of 30 (75, 135, 165, 210); max reps in 10 min. Here we'd expect to see a 'staircase' as a group of people can do one weight but can't get to the next one. We did get that, but with more of the S-curve mixed in, as people timed out (I guess) on their max weight somewhere mid-set.

Here I fared a little better, in the fifteenth percentile. Of course that's only because the tool put my score at the top of that big cluster of people that couldn't get beyond the first set. That's ok though because I did the first set in only a couple of minutes and was very close to being able to snatch the 135. Again, something to work on, but I felt a tiny bit better about this one.

Workout 3:

18 minutes AMRAP of 15 24" box jumps, 12 115# push press, 9 toes-to-bar. Here again I expected an S-curve, and sure enough got one. more of a 'pure' S-curve as it was more about efficiency and endurance through the lengthy time.

I was in the 10th percentile (ok, I guess). I've since nailed the rhythm on the T2B, so if I were to re-do this, I'd have more time to rest probably could push a little further than I did.

Workout 4:

This was an interesting one. 150 wallballs, followed by 90 double-unders, followed by 30 muscle ups, as a 12 minute AMRAP. Here again we'd expect to see a 'staircase' with a falloff in the low-end for those that can't finish the wallballs then a flat section for those that can't do DUs, and then another for those that can't do MUs. It turned out a little different:

Almost everyone finished the 150 wallballs (I didn't but it's a long story why. I was trying to improvise while at a gym in Costa Rica in heat well over 100 degrees, etc. In short I gave up early. Pretty confidant that if I redid this one, I'd at least get into the DUs). Unlike I'd expect though, there's only a handful (~25) people that capped at 150. I guess if people made the 150 they were inspired to at least try a handful of DUs.

At 240, however, there's a significant size plateau of like 220 people that got through the DUs but couldn't do 1 MU. And from there its a slow rise of folks that could do 1, 2, maybe 5 MUs, with only the top 150 or so that were getting a dozen MUs or more. Only 2 individuals actually got back to wallballs.

Workout 5:

A repeat of last year's final workout, this was a 7 minute AMRAP of 100 lb thrusters and chest to bar pullups, in sets of incrementing 3's (3+3, 6+6, 9+9...). This was another one where we'd expect an S-curve and sure enough got one.

The vertical end of the S-curve at the top is interesting. I'd guess that in addition to being in awesome shape, the top competitors are highly efficient at the movement too, trying to cram in a lot in a narrow window of time.

I was only in the bottom 5th percentile but am nonetheless happy with my result. 6 months ago I couldn't get more than one or two pullups, and the chest-to-bar part made these really hard for me. My score of 35 isn't great, but I'm happy with it and I have a bar to shoot for next year (Hmm... think it'll show up again?)

Post-mortem on my performance

Like I said, I headed into it knowing I wouldn't do well at all, but I'm glad I finished and I know what I have to work on for next year. In short, everything, but in particular, burpees & pullups and my aerobic performance in general. There were a few of these that had me sucking wind - hard.

Suggestions for the Games organizers

Not that they'll ever read this, but I've got some ideas on how they could take this to the next level.

1. Think about how the distribution of competitors across the spectrum will look. With workout 2 and workout 4, there were these large staircase effects. May not matter for top of leaderboard, but if you want individuals to be able to rank among themselves everywhere on the curve, better to allow those to break up. For example, if workout 4 had halved the set size (75+45+15), you'd have people doing double the number of rounds, but also more spread out distribution on the curve. Again, not an issue for top of board, but for the rest of us this is something to consider.

2. PLEASE! Publish an API to get read-only access to the leaderboard data. There is SO much that could be done here if you open this data up to hobbyist and professional developers. I just did a quick and dirty look at one region, and with a sparse sampling. If you had full access it would be trivial to do a number of compelling things. Some ideas:

• Compare curves across regions. Do some areas fare better at the strength-focused workouts? Do some areas have more beginners or more elites?
• Look at standard deviation of rank. For an individual, do you have one area where your rank was out of whack with the other workouts? does that give you an indication of what to work on? How about for a whole box? Does your box suck at a particular workout or area? If so does that tell you something about your programming?
• If someone read the data and took snapshots of it, you could do "motion chart" style animations of how the curves proceeded over the ~100 hours from posting to close. Do the elites submit at the last minute? How many people posted 2 or more scores? etc.
• If someone is 1-2 years away from transitioning to another age group, they could see how they'd rank.
• How about special shout-outs for most improved year-on-year? 

Those area just a couple ideas that come to mind. I'm sure if the organizers opened this up, they'd be surprised at how innovative ideas would come out of nowhere.

Anyhow, back to training and on to 2013! :-)