On Doubt

There is an emotional toll that comes with being constantly unsure and frequently wrong. How you handle that can define whether you are a good influence on your coworkers.

When I started my career, doubt was a great friend. Doubt made me a better programmer, sitting on my shoulder arguing for me to write better tests and better code. Doubt made me a better engineer when it suggested I think (and Google) twice before I committed to a strong opinion. Doubt made me a better co-worker by keeping my words humble when I was 100% confident the bug was in another team’s system and I should have been 99% confident.

Doubt is less of a friend now.

Confidence levels change with seniority.

I often talk about this shift with folks who are interested in staff roles by talking about confidence intervals.

As an early career IC, most of your decisions should have a 95+% confidence interval. This depends on the company, of course, but if only 1 in 20 of your features ships to production with a bug, that sounds great. Obviously, if your company doesn’t have solid peer review, deployment mechanisms or mentoring, you’ll be less confident than this, and that’s not your fault. More confident and I’d be curious about what your company’s product is.

Senior or career-level software engineers I would expect to dip down to 75% confidence for some of their bigger decisions. Maybe you’re 90% confident that this is the most reliable way to implement a feature or 80% confident that this is the right way to organize your codebase or 75% confident that this is the right interface to build to interact with another team’s component (higher, of course, if product direction was guaranteed not to change). However, you’ll likely still pop back up into 95%+ confidence for much of your day: you’re 100% confident that that code is hard to read or will cause a bug.

Often, staff level work means living in the 70-90% confidence realm. Questions that will underlie all your work don’t have clear answers?

• Is this the best framework for us to use to deliver the most profitable/useful product?
• How and when should we extract a component?
• How and when should we pay down tech debt?
• When does this architecture become more complex than the market needs?
• When is an architectural approach so deficient I should put my career on the line to push back?

These are decisions that the company needs you to facilitate and own, but that are often impossible to become fully confident in over a reasonable timeline. It’s too costly to try everything and then make a decision.

So how do you handle doubt?

Generally, my advice is to take a logical approach of acceptance and reframing.

• Acceptance: Accept that any architecture (or other) decision will eventually be wrong. The key thing is having a clear idea of how long that decision must be right for, hitting that and knowing as soon as possible if you’ve misjudged.
• Reframe: How can we get to 95 percent confident we have the right information and the right (and only the right) decision makers? How can we get to 95% confident we’ll know if we made the wrong decision at the 2 month mark?

None of this touches on how the doubt lingers.

There is an emotional toll that comes with being constantly unsure and frequently wrong. How you handle that can define whether you are a good influence on your coworkers.

So let’s ask again differently:

So how do you handle doubt?

Operating in 95+% confidence intervals means programming provides a steady drip of dopamine, as you consistently knock out one feature after another. My current company even turns this into a management philosophy: teams should ship early and often where “early” means something significant every two weeks. Your code is there, in production (albeit behind some feature flags), working correctly.

You provably changed the internet. And you’ll do it again next week.

For many of us, myself included, that joy in clear accomplishment, that dopamine spike, is a major factor in why we entered the field.

In contrast, staff roles have unclear feedback loops on longer timelines. When there are metrics, they’re very subjective and have the feel of testing your own mock – useful if there’s nothing better, but not a great reason to have confidence. I can stare at a job description or career ladder all day long, but success in a staff role rarely means checking off line items. Mentor which staff? Influence the company in which direction? Deliver which product to what markets with how much technical debt?

Self doubt is hard to live with and I’ve seen many failure modes among engineers who spend much of their time making lower confidence decisions. Most (if not all) of us end up drifting towards these every once in a while. Some common ones that you might see in yourself:

• Evaluating yourself based on the number of ideas you generate or introduce into your team(s). It may feel like you’re doing your job by throwing a bunch of options on the table, but your job is to help management tie business objectives to technical solutions: you’re now responsible for framing the decision and the process. You don’t get credit anymore if the right decision was one of the five you suggested.
• Disengaging: it’s really hard to stay engaged when you have very little idea of what success looks like and may not know for years.
• Iterating too long in private: whether it’s because we think more time will help us get it “right” or because we fear being publicly “wrong” or a combination of the two, we can wait too long to start involving others. When you’re trying to deliver something new, the MVP advice applies: if you’re fully confident in what you have to share, you’ve waited too long. (h/t to Bernerd Schaefer for this great point!)
• Spending too much time on finding the right process. The right decision making process can increase confidence in the final decision. But we often invest too much time iterating on the process and too little time iterating on the actual decision. If your process has multiple defined steps to get to an initial decision but zero defined check-in points for evaluating if that decision is still correct, you might be putting too much pressure on one decision point.

Making peace with doubt

If too little doubt means I’m working on the wrong things, then I need to make peace with doubt. I don’t know a ton of ways to do it, but here are a couple:

Define prior assumptions and be very clear on goals. There is no knowable right solution until you’re past the point where you should have made a decision. However, prior assumptions are often knowable. Goals can be circulated and get buy-in. Reassess often (defined reassessment points can be helpful for decisions with lots of different stakeholders).

Give yourself a break. Work on something that provides a bit more concrete success. Try to tie it back to team goals. Can be helpful for keeping you aligned with the challenges of everyday development. (Thanks to my reviewers for this one.)

Talk about it, with thought. I’m a better engineer when I can talk about where I think I might be wrong. And there’s some psychological relief in having company while wesit with doubt and long feedback cycles. For me, however, I try to make sure I talk about doubt as distinct from imposter syndrome. There’s a lot of talk about imposter syndrome and who can/should claim it. I don’t have the right training to make a call there (I read the wikipedia article; can I now be an expert?). However, for me personally, it’s been many, many years since I felt like I was one mistake away from being discovered as incomptent and unworthy of a career in software engineering. That’s also true for many other senior and long-tenured folks in this industry, however much we struggle with the doubt inherent in our roles. Talking about doubt is a way of recognizing that the success of the work I do is often unclear while not devaluing discussions for those who question if there is a place for them in this industry.

---

Thanks to Bernerd Schaefer and Mike McQuaid for taking a first look at this and providing wonderful feedback!