One of the areas I often see senior engineers struggle with, is raising junior engineers to the next level.
Often this is because we don’t give them the space to explore, learn and understand how to approach problems for themselves.

We’re going to look at three different scenarios to illustrate how decisions are usually made, how they could be better, and how senior engineers should be helping others make decisions, rather than making decisions themselves.

How we make decisions

The biased process

Each of us carries a set of biases on making decisions. These are shaped by years of real-world experience, often experiences that lead to us getting burned. This is an incredibly powerful effect and it draws us towards certain lines of thinking – and more importantly it draws us away from other lines, even if they could lead to more positive outcomes.

When presented with a suggested course of action by another person, we start with the horrible end result that could happen and then construct a path which could lead us there, starting from the new suggestion. From here we start to look for ways to avoid the new proposal completely, using the strength of the scary arguments that would lead to disaster. The number of behavioural biases which come into play here is pretty astounding. Choice-supportive bias, Belief bias, Confirmation bias, Negativity bias, take your pick. The list is long.

Imagine this dialogue taking place:

Junior engineer: Instead of backing up the database once a week,
                 what if we did it every day and made multiple
                 copies of the backup?
Senior engineer: Every day? No that's terrible, what if the
                 backups run long and go into peak traffic?
J: Well, I don't know.. they shouldn't?
S: But they might. And how much more space will daily backups
   take? We can't afford that.
J: No, I suppose you're right.
S: The last time we tried this we took the site down for hours.
   We got in so much trouble.
J: Oh wow. I'm sorry, I didn't realise my idea was so bad.
S: You'll figure it out one day.

A perfectly reasonable, and some might even say prudent, idea was presented but the biases of the senior engineer kicked in hard.
They remembered how much a similar idea had cost them in the past and immediately wrote it off.
Not only did they shoot down the original idea, they didn’t know why the junior engineer was suggesting daily backups take place. They used strong arguments to justify their position for which the junior engineer had no context and couldn’t learn from.’

Worst of all, the junior engineer was left feeling reprimanded for their lack of experience, without any opportunity to learn from the situation.

How we should make decisions

We often talk about “learning by doing”, the idea that we learn best when we try to do some work. Often times, failure from trying is an excellent way to learn new things if we can overcome the bias which negative experiences force on us.

The journey

My own journey of learning how to be a good engineer started in the mid 90’s when I got my first PC. It ran Windows 3.0, had 4Mb RAM and a 512Mb hard drive. After the “cool factor” wore off and I realised it was a pretty dumb, expensive box, I started wanting to do more with it but I couldn’t figure out how or what.

I started by reading computing magazines, mostly for three types of articles:

• Performance tuning tips
• Help and advice columns
• Articles on new technologies

The advice columns were great as a way to understand the problems others were having. I would try and see if I was having those problems, and to replicate the solutions.
Quite often I would end up breaking the system in unexpected ways.

First lesson learned: Have install disks handy.

In this way I built up a body of knowledge on recoverability, safe testing, getting comfortable with worst case scenarios, and so much more.
Many operations/production engineers had similar learning experiences, yet we too often shy away from this in our professional work when we should be encouraging junior engineers to work like this more.

 

Half way there

The above conversation is quite common not just in our industry but in many situations in both professional and personal relationships.

After this, we have what I call the “half-way approach”.
Let’s take a look at how that conversation went:

Junior engineer: Instead of backing up the database once a week,
                 what if we did it every day and made multiple
                 copies of the backup?
Senior engineer: Every day? We tried to do this once before, but
                 it didn't go well.
J: What went wrong?
S: The backups ended up running long, and when we got to peak
   traffic in the day, the site went down for hours.
J: Oh, that sucks. Is there anything we could do about it?
S: I've had some time to think about this since the last time.
   If you start the backups earlier and add a cron job to kill
   them if they take too long, that would save us some trouble.
J: Ok, I can do that.
S: And have that job send us an email so we know it happened.
   But before you do any of this, open a ticket to see if we can
   buy more storage space. If we can't do that, this whole thing
   is moot.
J: Great, I'll get on it. Thanks!

This is certainly an improvement over the previous scenario. There is less hostility, and more progress on getting the work done.
On the other hand we still have two major issues:

• The engineers seem to be working together towards a solution, but the senior engineer is planning the work based on their prior experience, and the junior engineer is simply following already carved-out steps.
  The junior engineer is a passive participant, taking notes and then going off to implement a given solution.
  There is still very little opportunity for the junior engineer to learn and grow.
• The senior engineer still hasn’t asked the junior engineer why they want to do the work, and what problem they’re trying to solve.
  There’s an assumption being made that the two engineers are working on the problem for the same reasons, but we don’t know if that is truly the case.

 

A better approach

Breaking our biases is extremely difficult
Even more difficult is remembering that a lot of ego and personal attachment can play into the decisions we make, too.
When other engineers come to us with problems, ideas and approaches, we can feel a strong desire to direct them to solutions we would like to see implemented, based on our ideas and experiences.

Being a senior engineer means realising not everything should be done the way you want it to be.

The corollary to this is:

Others may make tools, systems or decisions which aren’t as good as the ones you would make, but they should be able make them anyway.

Engineers, whether junior of senior, should be provided guidance and suggestions to help them form stronger ideas and improve their ability to reason, rather than being given solutions to implement or being out-right shot down.

Let’s take the conversation between our two engineers, and see what it would look like with this approach:

Junior engineer: Instead of backing up the database once a week,
                 what if we did it every day and made multiple
                 copies of the backup?
Senior engineer: Interesting idea. What's the problem you're
                 trying to solve?
J: I'm worried that we could lose a week of data if the servers
   crash.
S: That's a good point. You might run into a problem though:
   What if the backups end up running at peak time? We do them
   on the weekend because it's quiet.
J: I see. We can try to start them earlier, and maybe add
   monitoring to alert us if it happens?
S: That's a good start. Set it up and let's see what it looks like.

[ some time later ]

J: I was thinking, wouldn't it be great if we could just back up
   what changed every day, rather than everything? It seems like
   a waste of space.
S: That would be great! I have some ideas on how you could do that,
   but why don't you go and try a few things yourself first?
   If you get stuck, let me know.
J: Thanks!

[ some more time later ]

J: Ok, I think I have a possible solution. It's pretty great!
   I learned about this language called Perl and it made it really
   easy to do what we need.
S: Perl. Huh. Well.. there are some issues you might run into later
   with maintainability because only you and I know Perl. But if
   you like the solution, and it works, let's do it.
   Make sure you write really good documentation on how this works,
   and how to fix it if it breaks.
   And don't forget to comment your code!
S: Oh, and do you have a ticket to track all this work? That's also
   a good way to document what you did and why, so we can look back
   on it later if we need context.
J: Not yet, but I'll make one before I do anything else. Thanks!

It’s important to remember that after exploring the space, the engineers together might realise the solution is unworkable, or inefficient, or not feasible for business reasons. It’s even more likely that the junior engineer would come up with a solution which works fine, but isn’t what the senior engineer would have done themselves.

Fear of these is not justification for blocking the work.

The junior engineer picked up new hard skills:

• Learning new language
• Implementing a backup solution

As well as a bunch of critical soft skills:

• Improved decision making
• Engaging other engineers
• Presenting and refining ideas
• Presenting solutions
• Tracking work

And that’s all before the solution went to production!

Helping junior engineers raise strong proposals

Presenting ideas is a learned skill, we develop it over time from interactions with others, discussions and debates, watching talks at conferences and attending meet-ups.

As senior engineers, it’s important for us to help engineers around us develop this skill.

In the third conversation, the senior engineer asks the junior engineer what the problem is they’re trying to solve – this approach is the Socratic method.
Other good questions to ask are:

• What are the top three solutions you investigated before you settled on this one?
• Why did you reject the others?
• Do any of them have merits which we should investigate later?
• What could go wrong with your solution?
• How maintainable is it if you’re not available?
• What data do you have to back up your assumptions?

The goal is to get your engineers thinking about these questions before they come to you with proposals.
Have them present the answers to these along with their initial idea.
Over the long term this will help them develop strong data-driven solutions, evaluate multiple solutions and make their ideas more approachable.

Almost all work is a learning opportunity, but you shouldn’t encourage junior engineers to pick low-risk or low-impact work purely for the learning opportunity. Find time which can be dedicated to pure learning, and allow them to work on difficult, and even risky tasks with guidance.

The impact of language

To wrap up this post I’d like to take a moment on how words we use can have a huge impact to the tone and impact of a conversation.

In the third conversation, the senior engineer didn’t use the words “no” or “but”. Instead they promoted the idea as being plausible and encouraged the junior engineer to explore the space. The tone of the two conversations was very different. The first bordered on hostile and defensive, while the last was open, encouraging and welcoming.

Even though the solution may not have been designed and implemented the way the senior engineer would have done it, it still worked and provided the junior engineer a feeling of ownership. It gave them the ability to learn a lot more about the space, which they wouldn’t have done if their initial idea had been shut down, or if they were told what to implement and how to do it.

Instead of “no”, “but” or “yes, but…” (a subtle negative), learn to say “yes”, or “yes, and…”.
Encourage other engineers to go forth and come up with solutions to problems, and then help them implement those solutions.
They will not be the same as the ones you would come up with.
They may not even solve all the problems you see now, or see coming down the road.
But they are a start, and an important opportunity for others to make progress, grow and be successful.

Thanks

Thank you to Jon Cowie and John Looney for reviewing this post and helping me refine a lot of the language and focus.