Open source at Spotify: engineer describes his efforts to increase collaboration

'Engineers find software very easy but speaking to people really hard', says Dave Zolotusky

The rise of open source enterprise software is one of the most important technological shifts of the past decade. Rather than keeping their systems to themselves organisations now realise it makes better economic sense to cooperate on the wide range of things they have in common and compete by adding value or serving a niche.

In recognition of this fact developers in many companies are allowed time to work on open source projects, the idea being that they will both broaden their experience and have access to a much wider pool of talent and expertise. Some companies also join software foundations such as the Apache Software Foundation, Cloud Foundry or the Cloud Native Computing Foundation (CNCF) as sort of demilitarised zones where cooperation can happen.

A recent recruit to the ranks of the CNCF is music streaming service Spotify. While its core codebase is proprietary, Spotify's Github repositories contain a number of libraries and APIs that are open to inspection and cooperation and it wants to do much more in this area, according to Dave Zolotusky a software engineer at Spotify who spoke at the recent Kubernetes and Cloud Native Conference Europe event.

Like many other companies, Spotify is in the process of moving its data centres to the cloud, rearchitecting its infrastructure to move faster, make changes easier and to comply with the GDPR. Much 'cloud native' technology is cutting edge, and the usual fallbacks of Google searches and StackOverflow may not be much help. You need to talk directly to the people trying to make it work.

There were three areas where Zolotusky was convinced his team needed help from users outside of Spotify and its service providers: support, guidance and information about emerging technologies. But the first problem was figuring out who to engage with.

"Who or what is 'the community'? When starting with Kubernetes we really wanted to talk with some customers but we didn't know who to ask. Can we just pick up the phone and start talking to someone who doesn't know who we are?"

Zolotusky's technology platform team decided that the best approach would be to start by going public about what they were doing, publishing blogs and giving talks at events. Having put themselves out there a bit they started to jump on the Slack channels of the projects they were interested in and ask for advice. Zolotusky said he was surprised at how forthcoming people were.

"We may not have found the magical solution to our problem but we did find people who were trying to do the same thing and were maybe 80 per cent there, and there were people from huge companies like CapitalOne or Zolando happily reaching out to us,"
Zolotusky said.

They tried asking some more and found core developers of 'hot' open-source projects were happy to spend time working through problems by email, and one accepted a request to give a talk to Spotify engineers asking nothing in return. These are not isolated incidents, it's a huge change compared with how things used to work, Zolotusky said. Even large investment banks are starting to adopt the open source model in areas such as containerised applications, something inconceivable until recently.

"Ten years ago a big financial institution like Morgan Stanley would never tell anyone what they were doing, still less open source it."

Whatever the technical problem at hand, someone somewhere will be going through the same thing, Zolotusky went on.

That problem that's keeping you awake at night? There are probably hundreds of people with the same issue

"That problem that's keeping you awake at night? There are probably hundreds of people with the same issue, so we just have to reach out and find them and not isolate ourselves. And if you are that one person with a problem that no-one else is working on, well you might just become a thought leader in that area," he said.

Zolotusky's team is attempting to act as an open source beacon to spread the practice throughout Spotify. Engaging in open source communities is not just about contributing upstream code, it could also be about writing a blog post, improving documentation or helping out someone with a problem. As a company, Spotify is keen for its developers to get more involved in open source and many individuals would like to contribute.

But the distributed, global structure of Spotify in which thousands of engineers are loosely grouped into semi-autonomous teams can make this difficult, explained Zolotusky. Because they have a close focus on a small part of the big picture, developers tend to suffer from 'imposter syndrome', undervaluing the work they are doing, and believing that someone in some other team somewhere must be more advanced at it than them. So they fail to put themselves forward.

In internal surveys many people developers blame Spotify's management for the relative lack of engagement with open source, but, said Zolotusky, they should look to themselves.

"As engineers, we find software very easy but speaking to people really hard, but a lot of that giving back and contributing is on us as individuals."