Recently, Paul Graham wrote an essay, making the case to change immigration rules to allow more excellent programmers into the US.

While Paul makes a number of excellent points, I think he’s missed the mark. There is a certain mindset in large parts of our industry that in order to succeed as a programmer, or to succeed as a start up, you must be located in Silicon Valley, and you must always work locally with your team. It’s probably not surprise that most of the people who have this view point are also in Silicon Valley.

Since his essay was published twitter has been ablaze with discussion on this matter, with a number counter points being raised:

• The desire to increase immigration stems from the desire to lower wages.
  While Paul didn’t state this in his essay, this feeling is woven into the fabric of the Silicon Valley mindset. People are encouraged, if not out-right expected, to work long hours without extra pay. Some might argue that this is the price of success for a start up, but I don’t believe it is.
  The recent class action lawsuit against Adobe, Apple, Google, Intel and others highlights the desire by larger employers to keep wages suppressed too. According to data from salary.com, the more senior software engineers can easily cost companies $200,000 or more a year. And given the high demand for filling these positions, you can only expect that to go up.
• The social and economic impact of cramming ever growing numbers of highly paid tech employees into the San Francisco bay area has also led to a great inflation in housing costs, contributed to a housing shortage and other issues.
• There is a viable alternative to bringing more people into the US (and specifically into Silicon Valley), which is distributed teams and remote engineering. Paul has held that it is better to have people sitting together in the same office, that you lose something by not being able to have chance meetings, and that start up founders want people in the same place. Many of us believe that this isn’t necessarily true and that like all good things in life, balance is important. Paul later suggested that the success of distributed teams depends on the product being developed, implying that some (or many?) products cannot be created easily by distributed teams, which I also believe is not accurate.

I agree with many of the points Paul raised (most senior programmers ARE outside the US, and it does often help to have people in the same room to hash things out). However I don’t believe that fixing immigration laws should be the first, or only thing we talk about.

I’ve been involved in the New York start up scene since I joined Etsy in 2010. Since that time, I’ve seen more and more companies there embrace having distributed teams. Two companies I know which have risen to the top while doing this have been Etsy and DigitalOcean. Both have exceptional engineering teams working on high profile products used by many, many people around the world. There are certainly others outside New York, including Automattic, GitHub, Chef Inc, Puppet.. the list goes on.

So how did this happen? And why do people continue to insist that distributed teams lower performance, and are a bad idea?

Partly because we’ve done a poor job of showing our industry how to be successful at it, and partly because it’s hard. Having successful distributed teams requires special skills from management, which arent’t easily learned until you have to manage a distributed team. Catch 22.

My hope if that Paul and others will read this post, and see that managing remote engineers isn’t a losing proposition, and that it can be (and is being) done with great success.

Here are some key factors which I feel enable distributed teams for success.

Culture and management

The primary reasons for the success or failure of distributed teams come down to the culture of the organisation, and the strengths of management to enable and empower engineers to succeed regardless of location. Michael Rembetsy, Etsy’s VP of Operations, has given a number of culture based talks which touch on this. My favourite is one from Velocity EU 2012 on Continuously Deploying Culture. He mentions how Etsy used to have “too many” remote engineers, but “too many” was in the context of a poor culture. Before 2010, we trimmed down the number of remote engineers we had so that we could fix the core cultural issues, which then enabled us to hire many more remotes.

I’ll say this again because it’s very important:
The success or failure of a distributed team hinges on your organisational culture and the strengths of management, not on the product you’re creating or the nature of distributed teams themselves.

So what are some things organisations should work on, to promote success amongst distributes teams?

Become a learning organisation

Peter Senge promoted the idea of a learning organisation in The Fifth Discipline, as an organisation that is constantly learning and evolving. This has been a core part of Etsy’s success, and also integral to enabling our distributed engineers to succeed. The natural human reaction to negative events, which traditional corporate cultures foster, is it point blame and move on. With remote employees this can be even more of a problem. Feeling disconnected and alone while people “somewhere else” decide what happened can be unnerving and promote an unjust culture.

In 2010 Etsy had it’s first no-blame postmortem. John Allspaw wrote about this in 2012, detailing what this process involves and why it is important. Developing a Just Culture is crucial to supporting and empowering distributed teams. Individuals have to feel safe in their work, and supported by the organisation they’re in, if they’re going to do their best work. This is true with local employees, and especially true with remotes.

Over communication is the best communication

Early in my career I worked as a systems administration for a large ISP in the US on the night shift. We had little communication overlap with other teams, but we were primarily responsible for maintenances and downtimes. Some time into my employment, towards the end of one of my shifts, a member of another team came up to me and said “Boy, Av, you sure do reply to a lot of emails. Maybe you shouldn’t reply to everything you see?”

At the time, his advice felt strange and wrong, but I didn’t understand why. It wasn’t until my first position as a remote engineer in 2008 that I truly understood the importance of over communication. Since that time, the comments have turned around and people are glad when I and other remote engineers take the time to write down and express things in detail. Again, this is something that we can all do better regardless of our relative locations, but it’s especially true with remote engineers. Over communicating my actions, work load, thoughts and ideas with my peers has helped advance a sense of trust and team which is harder to do when you’re far away.

Communication has to work both ways. My manager and I have multiple scheduled 1:1 meetings each week. Sometimes they’re hard for us both to get to, but we try because they’re important. We take the idea of a “chance meeting”, and develop it during our chats.

Communicate as remote, by default

One of the final steps all engineers need to take to push success, is the idea of communicating as a remote engineer all the time. This is where Paul’s “chance meetings” really come in. At Etsy, our primary method of communication is IRC. Other organisations use Slack, HipChat, IRCCloud, Skype. The tool is less important than the idea, with the idea being to always use it for communications:

• Is the person 3 desks away? Use IRC.
• Is the person a minute walk away? Use IRC.
• Is the person working at home? Great! You’re already used to using IRC.

In addition to this, we do our best to keep communications public, for anybody to see and join in. This significantly increases the probability of “chance meetings” happening virtually.

Deliberately making teams distributed

In almost every case where I’ve seen a distributed team fail, it was down to a lack of distributedness. Having only one or two people in a team of 10 working from a remote location, does not constitute a distributed team. If you want to increase the chances of success, make the majority of the team remote. In doing so you cause a cultural shift where everybody has to learn how to communicate and work in this mode. You won’t achieve this by having a single person be a remote engineer.

At Etsy we have some teams where many or most people are remote. A happy side effect of this, was the tendency of the remote engineers to band together, create their own IRC channel and develop social relationships across team boundaries. This happened quite naturally and was almost surprising.

Travel

I said early on that I agree with some of Paul’s points, and I truly do. He’s absolutely right that being in the same room as others fosters something special. There are two clear cases I can recall in the last few years when being in the same room as my teammates has caused magic to happen:

• After hearing Joshua Hoffman speak at VelocityConf about Collins, a tool they were developing to manage the infrastructure and provision hosts more easily, several of us got together, rented a whiteboard from the hotel, planned and started coding. We each went home, and still working together knocked out a working prototype for our own such system within a week. It was truly a special moment.
• In late 2011 our Search team was having trouble replicating very large Solr indices across many machines, it simply took too long. One evening in our New York office, I suggested (somewhat off-hand) to my colleague Laurie Denness, that we could use BitTorrent instead. Within moments the sparks were flying as gears shifted and magic was ignited. Our search team later wrote up how the final solution was implemented for the world to share.

My point here isn’t that magic can only happen in person, rather that magic doesn’t happen all the time. We cultivate the magic with frequent travel. I travel to our offices once a month, others travel one per quarter. On each trip, some amount of magic does happen. In between those times, we have a lot of other work to do. We still manage to make magic happen when we aren’t in the same room.

If anything, I would contend that magic happens more often after work, over food and drinks, or during casual conversation. Regularly traveling to meet your peers more than achieves that, and leaves the rest of your time free to focus on implementing the magic you have brought forth.

Balance

And this is the real point I want to make here to Paul and others who are advocating various solutions: Everything needs balance. Not everyone is cut out to be part of a distributed team, and not everyone works best in an office. A great many factors come in to this. But at the same time, closing the door on hiring remote engineers, not focusing on developing solid cultures that foster growth, learning and work irrespective of location hurts companies, individuals and the economy at large.

One data point which I find very interesting: More than half of the senior engineers at Etsy, are remote employees. These are your highly-influencial people, who impact the work of others around them, and bring other engineers up with them across the company. And many of them do it while not being in the same office.

In closing…

Paul states that 95% of the best programmers must be outside the US as 95% of the world’s population is also outside the US, and that “exceptional programmers have an aptitude for and interest in programming that is not merely the product of training.”

You’re right, Paul. Their aptitude isn’t merely the product of training. It is the product of learning, culture and growth. These aren’t things we teach to programmers, rather they are environments in which we enable them to flourish.
I don’t believe we do nearly enough to find and bring forward the people who could be the best programmers here. Our education system is ill equipped to develop skills like critical thinking (with some arguing that I cannot even be taught!), that are necessary to growing more “10X” people. Our business schools don’t teach managers to create Just Cultures and the environments we need to cultivate more expectional programmers..

There is a whole lot we can do, but it isn’t necessarily easy. Flooding the market is equally not the answer – there is no test for what an exceptional programmer looks like, so it’s nearly impossible to legislate around. The result would be opening the market to many many programmers, driving down wages, and potentially increasing unemployment, and increasing difficultly in hiring people – 1000 resumes/CVs are harder to go through than 100. As a corollary to Parkinson’s Law: The number of companies hiring exceptional programmers increases with the number of exceptional programmers. We will never have enough.

If you truly believe there is a talent shortage (in the US or otherwise), or a shortage of exceptionally talented people, I direct you to Andrew Clay Shafer’s Velocity NY 2013 talk There Is No Talent Shortage. Andrew talks deeply about culture, Nash Equilibrium’s, and Organisational Learning. Our organisational cultures are what develop those exceptional “10X” programmers, and we can make more, and they can be anywhere in the world. Doing this industry-wide will take a significant cultural shift to get enough people on board. Ultimately, this approach will have far greater pay offs than changing the immigration policies of a country.

The future is here, it is learning organisations and distributed teams.