How Useful Are Free Salary Benchmarking Reports?
Last week, a salary data report from Hired.com was making the rounds on some Slack communities I’m in, as well as on Twitter. The timing was apt, because the same week, a few of my coaching clients brought up questions about salary benchmarking data, and how they could design a salary scheme as their companies continued to mature.
At first glance, the report from Hired looked useful. It confirmed some big trends that I’ve witness firsthand, such as engineers in the UK generally earning less than engineers in the USA, and that salaries are still on the rise, despite a cooling hiring market.
But these two points of confirmation weren’t enough for me to assume credibility for the rest of the report. I always approach free reports with a lot of skepticism, and you should too. Here’s why.
You’re the product
This report, and dozens like it, are a marketing tool. Sure, it’s a marketing tool that does provide some value for you.
Engineering leaders are notoriously a logic-driven group of people, and I am a skeptic at heart. I question the validity of everything, from a charity soliciting donations door-to-door to the nutrition score on my box of crackers (does it benefit the government to trick me into eating more wheat?!). Oddly though, as soon as something is both free and useful, many of us suspend that skepticism because there’s a job to be done, and this resource is helping us do it faster.
My rule when using free benchmarking data is the same as when I’m using a free service or tool:
If you’re not paying for anything, you’re the product.
The first question to ask yourself when looking at free benchmarking data — or any free reports published by a for-profit company — is “what are they selling?” In the case of Hired, they sell hiring and talent acquisition services, but also quite a handful of features like “salary bias alert,” which will tell you when you’re making an offer that’s out of range. It directly benefits Hired for you to look at this benchmarking data, think to yourself, “oh man, I think our salaries might be out of calibration with the market.” This makes you a more qualified lead for them.
Other questions to ask:
Am I their target customer? If so, there’s added incentive for the company to present the data in a way that makes you feel visceral pain.
How could my demographic data or questions benefit their product? Free product tiers or free whitepapers can be used to validate product ideas, or collect more data on a persona or demographic.
Where is the company located? In the case of Hired, they’re headquartered in the USA. Know that the data you’re getting is going to be skewed with that perspective. In this report, the data is coming from the US, Canada, and Western Europe, but presented as “worldwide.”
Understand your sphere of competition
If you are using salary benchmarking data to determine your own salary bands, or to verify that your offers are competitive, it’s imperative that you understand whose salaries are being reported, and if they are representative of the candidate pool for your open roles. Related to the last question above, “where is the company located?”, knowing the source and potential biases or omissions of your data is important.
Let’s see how this can play out:
Company A is a remote-first, internationally distributed startup. At Series C, they’ve gained some significant momentum both as a top-tier engineering brand, but also as a company with a high valuation and potential huge upside for equity. Because of this, they are competing on a global scale for talent. Specifically, a candidate in their hiring funnel can easily get an offer at another top tier company, including ones that have salaries normalised around San Francisco rates.
Company B is headquartered in Cleveland, Ohio. They are a publicly traded healthcare company, and they’re looking to expand their global offices by opening a satellite engineering office outside of London. Though most teams are flexible, employees are required to be in the office some of the time, so they hire only people who are local to the office, or who agree to relocate.
Company A and Company B are not competing for the same talent, and looking at two such stark examples can show how free salary benchmarking data isn’t always specific enough to know if you’re looking at salaries of people who are competitive on a global scale, on a local scale, or something else.
Is currency conversion important?
If it’s not obvious, I’m not an economist. I can’t speak to the textbook correctness and/or economic implications of comparing salaries in Europe to the USA with both salaries converted to USD, or if it’s reasonable to look at the base salary in their respective currencies. But, I can point out that it’s something you need to watch out for.
Check to see how your benchmarking data is reporting this, and how useful it is for your business. If your business is headquartered in the UK and you’re looking to hire engineers in the USA, it’s obviously going to be very useful to look at US salaries converted to GBP for your bottom line. But if you’re looking to understand what a competitive offer is in Berlin, knowing the USD equivalent isn’t as useful. The market rate for a job isn’t completely dependent on the exchange rate to USD.
My take on this topic is likely a bit different from other people’s, given that worked for about a decade in the USA and then moved to Europe in 2015. Unsurprisingly, my mortgage and car payments do not change based on the value of the Dollar against the Euro. For this reason, I tend to grit my teeth a bit when hearing the argument that “well, an engineer in the UK is making 85k GBP but that’s 95k USD.” Factually true, but the cost of living isn’t necessarily tied to the value of GBP against the Dollar (though it’s been pointed out that inflation is driven by global economics, which does influence cost of living). For a long time (and arguably still today), 1 EUR and 1 USD had the same buying power in local markets. Specifically, an iPhone was 899 USD or 899 EUR. My rent for a similarly-nice apartment in Berlin was 1500 EUR a month, about the same as what I paid for my condo in Chicago. But, things like out-of-pocket healthcare expenses do vary extensively from country to country.
Oddly though, I’ve never seen a company adjust base salary in the employees favour based on changing global exchange rates. However, it is interesting to mention that some companies — including one I formerly worked at — do give employees the option to select which currency they want to be paid in. I had some team members in countries with very volatile currencies elect to be paid in USD.
I asked Deel and Remote if they also noticed this trend:
In short, know how currency plays a role in the questions you’re trying to answer with benchmarking data.
Where I do look for trustworthy salary benchmarking data
I’ve spent a lot of time pointing out reasons not to trust the data, so to round it out, here are some places I do look to for trustworthy data around salary benchmarking.
Paid sources. Again, if you use free stuff, you’re using advertising. Many larger companies use a tool like the Radford Compensation Database for salary benchmarking. There are a handful of other tools like Figures that are positioned more for startups and small to medium businesses.
Published salary calculators. These only give you insight into one company, but the depth that they offer is valuable. GitLab’s famously used to be public but now it’s available only to employees, but Buffer’s is still out there. Codacy has a thorough writeup of how they’ve structured their calculator, as does Whereby.
Self-reported data. Glassdoor, Levels.fyi, and others are all good ways to get some data points about salary, but they never paint the whole picture, so keep that in mind. I really like the approach taken by TechPays, because it’s easier to differentiate whether the salary is for a global company, local company, which level, and the balance of cash vs. equity vs bonuses (Levels.fyi does a good job here to). These can never offer the depth and breadth of either paid benchmarking data or salary calculators from individual companies, but I do find them helpful to round out a picture.
Gem. Gem is a talent engagement and insights company, so a lot of the rules about "using advertising” and “you’re the product” definitely still apply here. But, I’ve followed the reports that Gem uses for years, and I have found them to stand out as unbiased, not using scare tactics, and transparent about what they are and what they aren’t. I’ve never felt tricked or that I was being sold to when referencing any of their materials, which I can’t say for other companies in the same space. They’ve got a bunch of reports that I’ve found useful when doing research, as well as a metrics calculator and lots of really solid blog posts about all things people.
