The hill-climbing paradox
tl;dr stop thinking, charge through the fog
Sometimes I find myself in nightmares where my punches aren’t landing and I can’t see what’s attacking me because of a fog. Then I wake up and realize it’s actually much worse - I can’t code at a time when incredible wealth is being created by software engineers.
Why ambitious people are not making the optimal move: Chris Dixon addresses this question in a blog post from 2009 called “Climbing the wrong hill”1. He recalls talking to ambitious people who want to work in tech long term but they don’t leave their jobs until they’ve wasted years working in ways unrelated to their long term goals. To explain their behavior, he reframes their choices as a computer science problem.
Thought experiment: Imagine you’re on hilly terrain, trapped in a fog, and you need to get to the top of the tallest hill in the area but you can only see a few feet in any direction. He proposes three approaches:
Simple option: always take a step in the direction that takes you higher
Better option: add some randomness into your walk when you do so
Best option: repeatedly drop yourself in random parts of the terrain, then run the simple option
The downside to the first approach is you could be climbing higher on a hill that isn’t the highest so the second approach adds some randomness to increase the chances of stumbling onto the highest hill. The third approach wins out because by constantly resetting yourself you will eventually cover so much territory your guess of the highest hill is probably accurate. The twist is we have the benefit of knowing the biggest hill is technology already.
Why do people take the simple approach?: Chris Dixon’s answer for why people ignore the optimal strategy is that we’re hard-wired for near term rewards. Ambitious people are very sensitive against taking a downward step and are lured by the prospect of a steady upward path. Despite modern pretensions of rationality, we evolved under brutal conditions where prioritizing near term rewards increased our chances of survival and that influences our behavior today. It’s not entirely wrong either, there are plenty of people who dived right into doing tech startups or working in the industry but don’t have much to show for it. It’s possible the herding tendency also contributes to near-termism since there’s a lot psychological safety in sticking to careers that are widely understood to be prestigious.
Is it about unfamiliarity or a skill issue?: tl;dr uncertainty explains strategy selection more than near-termism or skill issues
The problem with these explanations is while launching a startup or developing familiarity with software programming was niche and unfamiliar when the blog was written in 2009, in 2024 it’s widely accepted as a prestigious career. In 2010, “The Social Network” was released to popular acclaim and tech valuations increased significantly since then, albeit with some recent bumps in the road the last few years.
So when I look at my own example, I’ve seen increasing valuations of tech companies and everything that comes with that, including rising salaries. How do I reconcile the abstract knowledge that there is a bigger hill to climb with my lack of action?
I actually had this thought a few months ago so I started my own tech-adjacent consulting business. I can report back that it was like hell on earth: my business got debanked by Stripe, I struggled to sell beyond two customers, each new customer had intense negotiations which was exhausting, I had to do a ton of rework because I didn’t negotiate service levels ahead of time, and the worst part was I don’t think it was worth my time.
I spent a month’s worth of free time (8.3% of a year) just to get the equivalent of 2.5% of my annual pre-tax salary. To illustrate with numbers, imagine a median US worker ($50,000 per year) trades their job making a guaranteed ~$4,167 per month for a chance at making $1,250 occasionally instead. Having tried both, I think the salary is absolutely preferable in the short term. There was a path to making a decent salary on my own over time but the uncertainty in the meantime when you’re starting from scratch is very high. Although I got a nice chunk of change upfront and I intellectually understood there were larger payoffs in running your own business the constant disappointments along the way made it difficult to have enthusiasm about continuing.
The solution is explore the map and remove the fog:
Perhaps the way around this quandary is a thought experiment of our own which incorporates uncertainty more explicitly. For example, in Age of Empires II players start with a village (i.e. a cavalryman, sheep, a few villagers) and they have to scramble to gather limited natural resources on the map to upgrade their technology as well as fend off rivals. There is also a game dynamic which mimics the uncertainty of building a software business:
areas your units are operating in are clearly visible
resources in areas you’ve previously explored are dimly visible but enemy troop movements aren’t
unexplored areas are dark
A few games in you realize that starting conditions have a big impact on the endgame, particularly resource exploitation and maximizing the villagers’ time. Resources are located right outside the clearly visible areas so you need to prioritize scouting to find them while keeping villagers maximally occupied and eliminating whatever are limiting constraint to generating new units at the moment. Within these constraints, players quickly end up developing playbooks (called “build orders”) for eliminating uncertainty and ramping up their civilizations in the game.
When it comes to software projects, I need to do the same - but since I don’t have a build order in mind I’m going to learn what I can, try different projects, and jot down what I’m thinking. I have an ingrained habit of scribbling thoughts on sticky notes and it’s been massively helpful to me in the past. If you’re interested in learning with me, subscribe, and let’s charge into the fog together.
https://cdixon.org/2009/09/19/climbing-the-wrong-hill