The Best Staff Engineering Work Often Starts Before Anyone Assigns It

Lalit Maganti's "How I Find Problems to Solve as a Staff Engineer" describes a way of working that feels very familiar to me.

Maganti doesn't set aside an hour to sit somewhere and "think strategically." He listens.

"I act like a sponge."

He pays attention to complaints in meetings, chat threads, emails and conversations. A single request might not mean much. The same frustration appearing across several teams probably does.

Eventually the shape of a larger problem starts to emerge.

I tend to work this way too.

Being remote makes the old water-cooler version of this harder. I don't accidentally overhear two people talking while getting coffee.

But the chatter is still there if you listen for it.

It is in standups, support questions, tickets, chat threads, meetings and the small comments people make about something taking longer than it should.

Someone says they have to perform the same tedious step every morning.

Someone else cannot get data they need.

Another team has built a strange workaround.

Someone asks for a feature that sounds oddly similar to something another person asked about two weeks ago.

Those are often more interesting to me than the request itself.

The first question shouldn't always be, "How do I build what they asked for?"

It should be, "Why do they need this?"

Sometimes helping means showing someone that the thing they need already exists.

Sometimes it means fixing a small problem.

Sometimes several apparently unrelated complaints turn out to be symptoms of the same missing capability.

And sometimes the right answer is to do nothing yet and see whether the problem appears again.

That last part of Maganti's post is important. He deliberately lets problems accumulate before deciding they deserve a project.

A loud request is not necessarily an important one.

That is a useful discipline for staff engineers because there will always be more problems than time.

This only works if engineers are allowed to do it

Maganti includes an important caveat.

His experience comes from infrastructure and developer-tool organizations where engineers have substantial bottom-up influence over the roadmap. He acknowledges that engineers working in more top-down organizations may simply have less room to operate this way.

That made me wonder whether engineering organizations are becoming more top-down generally.

I don't have evidence that they are.

It is a hypothesis based mostly on my own experience and what I hear from other engineers.

But I wonder whether some of the culture of the technology industry has shifted from being strongly technology-led toward being increasingly business-, management- and product-led.

There is an important distinction there.

Product management is not the problem.

Understanding customers, revenue and business priorities is part of building useful software. Engineers shouldn't disappear for six months to solve technically fascinating problems nobody needs.

And the end of the zero-interest-rate environment probably made some tightening inevitable.

A company burning runway and struggling to raise another round has a perfectly rational reason to prioritize work that might produce revenue over an engineer's interesting infrastructure project.

That is not necessarily bad management.

What worries me is when that sensible business pressure mutates into something else.

The engineer stops being someone expected to discover problems and starts becoming someone expected to implement solutions already selected somewhere above them.

Product decides what.

Management decides when.

Planning assigns the work.

Engineering estimates it and builds it.

That may produce predictable roadmaps.

It can also throw away one of the most valuable information sources in a software company: the people who spend every day inside the systems.

DORA's research has repeatedly found that high-performing teams have meaningful autonomy. Teams can experiment, modify specifications during development and pursue ideas without repeatedly asking for permission outside the team.

The guidance is unusually direct: don't treat technical staff as order-takers.

That doesn't prove that engineering autonomy has declined across the industry.

I would actually like to see good longitudinal research on that question.

How much influence does the average software engineer have over what gets built today compared with 2010, 2015 or 2020?

How often can engineers initiate projects?

How much roadmap authority sits with engineering versus product management?

How has that changed by company size, funding environment and industry?

I don't know.

Right now I only have anecdata.

Predictability can become work theater

There is also a less charitable version of top-down planning that has nothing to do with profitability.

Sometimes organizations optimize for work that is easy for management to describe upward.

A project has an estimate.

The estimate becomes a schedule.

The schedule becomes a status report.

The status report becomes evidence that everyone is executing according to plan.

That can create a strange incentive where predictability becomes more important than actually eliminating the problem.

Finishing something in half the estimated time should be excellent news.

In an unhealthy organization it can instead become awkward because the estimate was incorporated into staffing plans, quarterly commitments and management expectations.

The organization starts protecting the plan rather than exploiting what it just learned.

That is work theater.

And it is almost the exact opposite of the process Maganti describes.

His process starts with incomplete information.

You listen.

You notice something.

You wait.

You hear it again somewhere else.

You investigate.

You discover that the requested solution isn't actually the problem.

You prototype something.

You might abandon it.

Eventually you may find a common solution nobody had originally asked for.

That is inherently difficult to put on a quarterly roadmap before the discovery has happened.

But it is also where a lot of valuable engineering work comes from.

Listening is part of engineering

I think this is one of the less visible parts of being a staff engineer.

You aren't only supposed to solve difficult technical problems.

You are supposed to notice which problems are worth solving.

That requires paying attention to people.

For someone working remotely, the water cooler is distributed across the entire workday.

You have to listen a little harder.

But people are constantly telling you where the friction is.

If you keep hearing the same thing from different directions, there is probably something underneath it worth pulling on.

The organization just has to give engineers enough room to pull.

Read Lalit Maganti's full post, "How I Find Problems to Solve as a Staff Engineer."