I could see the hold. Blue. Triangular. Near my right shoulder. Feeling pretty stable, I decided to go for it. Taking my right hand off the wall, I watched myself reach out toward the blue blob of pixels. My hand connected just where it should. Transfer. On to the next one.
Eight in the morning at Seattle’s Vertical World, and a friend and I were trying out third person climbing for the first time. The Cardboard strapped to my face was streaming video from a camera standing about ten feet back from the wall, which gave me a nice overview of the entire traverse as well letting me see how my body was positioned.
This was only my first go and already I was shimmying along without too much trouble, albeit slowly. Sure I wasn’t sticking to any particular route, but I’d half expected to walk up to the wall that first time and either flail about helplessly or completely freak. After all, who’s to say that, with the VR headset effectively acting as a blindfold, your brain wouldn’t imagine that you are dangling from the north face of the Eiger or something? Even when climbing in non-modded reality my brain sometimes pulls that trick. And it wasn’t just me either. My friend made the same traverse even quicker on their first try.
That’s not to say everything was going just peachy. I sometimes found it difficult to see the holds. That’s a bit of a problem. No matter how much I tilted my head, from the camera’s fixed position there was no way to see holds directly in front of me because my body obscured them. Traversing left to right for example, this made it difficult to move my left closer to my body so that I could continue moving right.
A far bigger challenge was judging a hold’s three dimensional shape. Was that orange splotch up there concealing a cavity or was it just a polyurethane pancake from some V10? From my head-on view, it was impossible to say without reaching up for a feel. Again, not that this is without precedent. Many a time while looking through my own eyes, I’ve reached up for a hold optimistically expecting a welcoming pocket but only to find a cold smooth surface. But whereas I can often get away with some improvisation in normal climbing, while climbing in the third person, I found that I either had to memorize the entire route ahead of time or always feel up holds before being confident enough to transfer to them.
To combat this problem, we tried positioning the camera at an angle to the wall. This side view certainly made judging the depth of holds easier but it also meant that my body obscured more of the view. And, at very steep angles, holds close to the camera obscured holds farther out as well. The head-on view was much better overall.
Perhaps what surprised me most though about these first climbs were the challenges we did not face. Over the course of my modded reality experiments, I’ve often struggled to understand where my body is positioned in space. Be it cycling or selfie sticking, it always took me time to trust what I was seeing and go with what the camera was showing rather than what my brain was telling me. And given that in climbing understanding your body position is even more important, I was a little concerned going into this experiment. Yet, now that I was on the wall, I found this was barely a problem in practice.
Look: I’m not an expert climber by any means. An hour or two a week at the gym only gets you so far. On a good day I can conquer most V2s and a smatterings of V3s (one time I even fumbled through a V5, only to have my ego crushed when next week revealed that the same route had been corrected down a notch or two). Still, I do have a theory about why modded reality climbing was not as big a deal as anticipated.
I think that climbing already asks you to understand the world and your body in a different way. While climbing, what you see is important sure, but so too is an awareness of how your body is positioned, how your weight is distributed, how far your limbs can stretch and flex and how long they can support you. You prepare for routes by visualizing how your body will move through them, and then execute these moves, often without observing them directly. Sight is just one of your senses.
So maybe that’s why my brain was able to accept what the camera was showing it so much more easily today. Maybe those weekly gym visits have already changed my brain in some small way, making me more aware of my body and less reliant on sight. Seems plausible at least.
Next up, my friend suggested that we try moving the camera along with us as we climbed. The idea was to simulate having the camera attached behind you, say on a backpack-like device. This turned out to be a nice improvement over the fixed camera in almost every respect.
Since the camera no longer needed to capture the entire route, it could be positioned closer to the wall, improving the resolution of the view and making it easier to spot tiny holds. The shifting perspective also helped me better judge the depth of holds and see around my body. The biggest downside of this perspective was that may hacked together hardware setup did not let the person holding the camera actually see what the camera was capturing. A bit of shouting back and forth about adjusting the view was required.
The other odd thing about the tracking camera is that it made our climbing seem exponentially more intense. Energized, we soon began experimenting with a number of other views that made our admittedly rather meager climbing skills look downright impressive.
By placing the camera at the bottom of the wall pointing upwards, we discovered that our 12 foot route was now a veritable El Capitan and we giants (although getting more than a few feet off the ground in this perspective proved challenging). Continuing in this vein, we next using a selfie stick to hoist the camera about ten feet up onto the wall, pointed downwards at the holds. This top-down view, combined with the fisheye distortion, made even just reaching for the next hold and pulling yourself up look heroic indeed. Taken with the music blasting in the gym, it felt like we were living out one of those crazy climbing showreels, except, you know, without the actual skills part.
However we were soon brought back down to earth by an ill fated attempt at horizontal climbing. Rotating the camera 90 degrees, the holds now appeared to go from right to left. What fun it looked! or so we hoped.
As I started across the wall, I found myself instinctively reaching to the left to grab the next hold. With each move, I had to remind myself to reach up instead of left. It just felt very unnatural, even after my hand connected with the correct hold. And that’s when the same confusion and doubt I know so well from my other modded reality experiments began to kick in. This was the only time when I genuinely became disoriented on the wall, and even a little scared despite only being two feet off the ground. As you may imagine, this put the kibosh on further inventive angles (such as upside down climbing).
Horizontal climbing aside though, the morning went great overall. I really didn’t know what to expect when we started. Climbing in the third person sounded like it could be fun but also really challenging and perhaps a little scary too. I’m happy to report that it really surpassed all of my expectations. My friend and I had a great time trying out different bouldering routes and different climbing perspectives all over the gym. Some of them were more practical than others, but all offered something unique and fun.
The climbing itself, although certainly not easy, was also not nearly as challenging as I had anticipated. The step from first person to third person climbing did not feel as dramatic perhaps as the step from normal life to being on the wall.
And while it’s hard to imagine people climbing with cardboard boxes strapped to their faces in the near future, this experiment suggests some interesting ideas. There’s the possibility of using technology to extend or augment your sight, to see around corners or up over ledges that you have not yet reached. It could be a good way to observe your form in realtime too. And, in the gym, augmented reality certainly seems like it could come into play sooner rather than later. Why color code holds if your glasses can dynamically color in routes for you? They could even provide hints or tips right on the wall. Or, taking things a step farther, perhaps your glasses could replace the climbing wall with a nice sheet of rock, the floor with a two thousand foot drop. That certainly may be a good motivator.
There’s also even more intriguing possibilities in remixing your other senses with climbing. Holds that play different sounds based on how you grab them or how your body is positioned for example, maybe even creating a type of sonar that would let visually impaired climbers locate holds more easily. Or maybe the reverse: routes that adjust based on your senses: holds that move farther apart as you heart-rate increases or if you look at them the wrong way, walls that dynamically budge out and shift based on how you move your body or as you literally pull on them. I know that I won’t have time to explore many of these ideas but it is still fun to imagine.
P.S. Big thanks to Vertical World for letting us use their gym