Three Unexpected Lessons from Rolling Out the SPACE Framework at Scale

Through both my course on engineering team metrics as well as the engineering leaders I support through coaching, I’ve worked with over 200 leaders who are using the SPACE framework and DORA metrics to answer questions about their team. Some of these leaders are responsible for big teams – in the 100s or more – and it’s been interesting to compare their experiences rolling out SPACE at scale to a startup who is using SPACE and DORA from almost day one. 

You don’t need to explain the intricacies of the SPACE framework to everyone in your organisation before you start using it.

Trying to educate every single developer about the SPACE framework, DORA metrics, and why they are used is – simply put – a waste of time. DORA is relatively straightforward and generally easily understood after a quick read of a decently high-level blog post. But the SPACE framework is nuanced and very open-ended. It’s not a list of metrics like DORA, but rather a framework that still leaves the choices up to your organisation.

Some leaders hesitate to get started with SPACE before providing enough explanation and training to their teams of individual contributor engineers. This really stalls progress. While I definitely agree that an understanding and curiosity about SPACE can be beneficial for everyone in an engineering organisation, it’s more important for the decision makers, not the engineers who are feeding data into the system. 

What’s most important for engineers is why the metrics were picked, who sees them, and what happens when the targets are met or missed.

Instead of approaching the explanation by starting with the SPACE framework, let it be a foundational note that gives context and credibility. 

Avoid this:

“We’re implementing the SPACE framework. Here’s an explanation of the framework and this is how we’re going to use it…”

And aim for this instead:

“We’re using metrics to make better decisions about our teams. We’re starting with these metrics. They will be seen by X, Y, and Z groups. Expect A and B to happen if we miss the metrics. We chose these metrics based on our company needs, and also by using the SPACE framework, which is a body of in-depth research about using metrics on engineering teams.”

Survey fatigue is real.

A bigger company usually means more processes. Some startups choose to send out employee satisfaction surveys from early on (think Lattice, Culture Amp, etc), but it’s almost a guarantee at larger companies. 

It’s critical to think about the frequency that an engineer receives a survey from your company as a whole, not just how often you are sending out a developer experience survey aligned to the SPACE framework. If your company’s employee engagement survey is going out every six weeks and then you’re adding in another developer survey every 6 weeks, there will be some months where your team is getting two surveys – which might look and feel pretty similar – from your company.

I’ve even seen engagement surveys at the company level going out every fortnight, meaning that some weeks, people get two surveys to fill out.  Don’t be surprised if the engagement is low in this scenario. It’s a lot.

Aside from frequency, another antidote to low engagement is to make it clear how and when your team members should expect to see changes based on their responses to the survey. 

• Make a plan to share and discuss the data, and inform your team about it before the survey goes out. This encourages them to participate, and gives them confidence that their responses will influence change.

• Discuss the results with your team and make a plan. You might consider using industry benchmark data to supplement your team’s responses.

• Send out follow up surveys.

Things get busy, and often things like developer experience get deprioritised because they’re not perceived as being on the critical path toward revenue. But as soon as you stop making it beneficial to your team to participate, they’ll stop sharing their opinions, and your surveys will get thrown into the ignore pile like so many other employee engagement surveys.

Phishing training really works!

Perhaps the most surprising thing was the sheer amount of change management and communication that’s required in order to roll out these kinds of organisation changes at scale. 

• Where are the metrics?

• Why were they picked?

• When do we talk about them?

• Who collects the data?

• Who has access to them?

• When should they make an appearance in decision making activities?

These are all questions that need to be answered. Beyond that, you need to form habits around them. And sometimes introducing a new habit into a larger organisation is like trying to steer a cruise ship with a spoon. 

I was also surprised to hear anecdotes from several managers at different companies that some of their team members didn’t respond to the developer experience surveys because they thought it was a phishing attempt. This even was after organisation-wide emails or Slack messages that outlined the process for the surveys.

The lesson here is to make it personal. Even if you’re sending out organisation-wide messages about these changes, it’s critical for each manager to be equipped to discuss the details with their teams in a smaller group where the impact feels more direct.