A survey of CTOs: how flat should your organization be?
A survey of CTOs: how flat should your organization be?
As your company grows you will need to add more structure, but knowing when to do so is a subtle art.
A few bad ideas have recently become fads in the tech world, in particular the idea that all managers are useless and that not hiring managers therefore leads to efficiency. In other words, it’s become fashionable to believe that a flat organization is automatically an efficient organization. This idea is pure poison, especially when inexperienced entrepreneurs hear it, because they will naturally be nervous when they are told that it is time to add structure to their organization, so if someone tells them that they don’t have to add any structure to their organization, they will be eager to believe that they can avoid what might otherwise be a difficult task.
Recently I’ve written a lot about this topic, see for instance:
However, you are probably sick of hearing this from me, so this time I surveyed several of the best CTOs that I know, and I’ll here present what they wrote.
We start with a quote from the weblog of Schuyler Bishop, and his essay “True engineering leaders should not code,” where he attacks a different aspect of the “keep everything flat” mantra.
This is from Schuyler Bishop:
The second intended outcome from expecting a leader to be “in the code” is the thought that you have to be an engineer to create value. I vehemently disagree with this reasoning and my example here is Steve Jobs. …He was definitely a once in a lifetime leader. He lead Apple out of the doldrums of the early-mid 1990s and to be the most valuable company ever, beating Microsoft in 2024. Steve Jobs didn’t write a line of code or provide engineering for a single product. I will grant you the possibility that Jobs was the exception of exceptions, but I have also worked in organizations where leaders were not direct engineers but provided immeasurable value. And in every case, those leaders were one thing — they were leaders. They required focus on leadership issues to be successful.
Leadership is its own skill. It is the most difficult for humans to learn, which is why great leaders are rare, and why we admire great leaders. Your managers should not be asked to take on tasks other than managing.
But this was a good starting point for my conversations with other CTOs.
When I asked “How flat should an organization be?” Lokesh Kumar responded:
I think it depends on the stage of the company a lot. When I scaled the startup that I co-founded from myself to nearly 100 engineers and 5 PMs, I went from complete flat to 4 teams (each with 8 engineers) to 10 teams of 10 each. When it was 4 teams, all those team leads reported to me. When I went to 10 teams, I had one more level for few teams plus I still kept some of the core teams with me. This way, I had a mix of both.
This gave me the visibility where I needed just that and hands on where I needed to be.
I am sure if it scaled beyond that, I would have to rethink the entire organization.
Ying Wang, another senior level tech leader, wrote:
Not sure where I heard this, but my rule of thumb is to try and align the unofficial power structures (who do your people actually turn to for help / guidance) with the official power structure (what's written on the org chart), to minimize the number of side channels that have to take place for a business to operate. Nothing's perfect of course, but it makes sense that if people trust the org chart to actually represent how the organization behaves and works, it's much easier to scale without unforeseen difficulties.
Steve Rubin wrote:
The main problem, as always, is that a good idea gets taken too far.
Flattening can be good. We don't need a manager for 1-2 people. They can be rolled up into the next layer. Great!
But then someone looks to push out all the middle managers, leaving just a VP with dozens of direct reports. It doesn't work long term. Can this work in very specific situations (small size, temporary, consulting, people/teams with decades of experience)? Sure. That's a handful of situations. For normal companies, IT DOES NOT SCALE. Every company who has tried to squeeze out the middle management layers has reverted (or failed).
So, flatter yes, but flat no.
From Alexander Kalinovsky:
In my experience 8 – 10 people is the maximum span of control. And I wouldn’t get hung up on the word “control”. It is about coaching, resolving conflicts, being able to spot signs of dysfunction. Whether it is a line manager or lead on a particular project or technical lead for a product team depends on a specific organization and what it does.
Among my favorite thinkers and speakers on this issue is Peter Bell whose answer was comprehensive:
I think this is broadly settled (although would appreciate dissenting voices).
- Default span of control for a line manager is ballpark 5-8 - slightly less for Software Engineering Manager (SEM)+ managing managers
- If you’re looking for a player coach that moves the roadmap forward, that becomes 3-6
- You can have a smaller span of control in hypergrowth (hire a director with an Engineering Manager (EM) and two directs and they’ll have a team of 35-50 in 18 months - we need them to drive the initiative and hire a bunch)
- You can have a bigger span of control in any of the following situations:
- - It’s a temporary period with a highly aligned and motivated team and you’re all willing to do what it takes for the next 6-18 months to ship, even if you don’t get to talk with your manager about career progression, quality of life or when your cat gets sick.
- - You have a really senior, self motivated team that doesn’t need handholding - replace 1:1’s with team checkins and other lightweight mechanisms for keeping alignment and visibility
- - You have a smaller org to align across - EM at Stripe is 35 hours a week of alignment across who knows how many teams. If it’s a 30 person engineering org, you can have more directs because there are less cross functional initiatives and collaborations to take care of.
I’d also say:
- Player coach team leads and EMs are both plausible strategies
- Once you get a level up from that (sometimes SEM - often a director with 3-6 EM or team lead reports) they should not be picking tickets from the backlog. There is value in them being technical, sometimes they’re architectural advisors, often they help test and level up their team, and they are probably reviewing and engaging with PRs, but they shouldn’t be pulling all nighters to ship mission critical features - otherwise they’ll end up blocking the team (and being lousy managers with excess regrettable attrition over time) [[ as opposed to “unregretted attrition,” a metric popularized by Amazon for when an employee quits but you were happy to see them go — lawrence ]]
If you take those general numbers and play with them, it gives a pretty good sense of minimum levels of management that are possible before things start to break.
Holocrary didn’t scale. GitHub thought you didn't need managers and found out they were wrong. There are lots of ways of tweaking the span of control numbers for a given situation, but I think you ignore them at your own peril.
Team of 10? Report to CTO, but consider adding a VPE or a director or maybe an EM.
You hit 20 ICs and you probably want 3 EMs and either a VPE or CTO keeping the trains running on time (sometimes the VPE does that while the CTO is in sales, evangelism, a hardcore architect, etc).
If you’ve got 50 IC’s in 7 teams, you’re right at the limit of what a single VPE can manage and you’re probably late in hiring 1-2 directors or SEMs under them to manage the load.
At 100 ICs, if that’s ~14 teams, if a director has 5 teams (a lot) you have at least 3 SEMs or Directors under what is probably a VPE
In my experience you actually end up adding more middle managers because you want them to own and focus on data science, platform, a product line, an important new initiative, etc, but having much less in the way of middle management than this is hard as a long term strategy.
If you actually deconstruct the elements of leadership/management, you end up with things like:
- Architectural leadership (advice on approaches to technical architectures and tooling)
- Coaching on coding (test infecting, leveling up feedback on PRs etc)
- Performance management
- Project management
- Career support/alignment with organizational goals
- Personal emotional support
- An interface to HR processes and systems
Etc
Some of these are temporary, some of them are ongoing. The needs depend on the individual, and the allocation of these roles doesn’t need to be “an EM does most of this and a Staff+ does the rest”
For example, consulting orgs often decouple project management from the line management of individuals.
Dynamic re-teaming is a way to have a larger group with a few shared managers who keep line management responsibilities for the same individuals but not necessarily the individuals that they’re working on a team with.
You can democratize or share responsibilities for some of the elements such as performance reviews
So there are ways to reduce the load, allowing for a larger span of control.
But personally I would only leave the default path to solve a specific problem that was worth solving within my org. I would rather innovate on features than organizational design unless a boring organizational design wasn’t working for some reason. For me it’s the leadership equivalent of “use boring technologies."
I personally prefer the traditional hierarchy being advocated by Peter Bell, but it is true that you could take aspects of a managers jobs (for instance, mentoring) and give it to someone outside of the hierarchy (for instance, a free floating coach). This means the manager does not have to mentor, so they have more time for other things, so they can handle having more employees under them. That is the approach advocated by Kevin Goldsmith:
I’ve seen companies give a manager very large spans of control by pushing more of the manager's duties into other roles. This will depend a lot on the company culture and somewhat on how other functions are organized (if Product is command-and-control and engineering is sociocracy, it will probably implode eventually).
The way I tend to think of it is that there are certain things individuals in the organization need: development/mentoring, direction, and performance management. In a traditional EM role, these are all done by the same person. Those tasks will limit how many people that one person can manage. The “rule of thumb” is 5-7, but personally I disagree with that as it is highly contextual on the team and the manager, I’ve managed up to 14 directs. It worked because several of them were fairly senior and didn’t need as much of my time.
If you want a flatter organization, you can figure out how individuals can get the things they need from someone other than their manager. Explicit mentoring programs, internal career coaches, and peer-based performance management frameworks would allow managers to be responsible for a much larger number of individuals. Companies like Google (at the large scale) and Qumulo (at the mid-side startup) have used these techniques.
…Our organization at DistroKid today is fairly boring, but for us it is a way point on coming from a very unstructured team that had gotten too big to maintain as is. I'm not sure where we'll evolve to, it will depend on what issues we face and what we think is the best way to solve them...
Those of us with an interest in organizational theory are exploring and experimenting with some of the ideas that Goldsmith is suggesting here. Does it make a more powerful organization when you take one aspect of management and give it to some specialist who is outside of the normal hierarchy? These ideas are worth exploring, but it is important to note that there is no success story that we can point to and say “That company was wildly successful because of their unusual arrangement of free floating specialists who took on aspects of being a manager.” And therefore, as Peter Bell said, if you don’t have a strong reason to experiment in this area, it is best to fall back to the traditional structures, which he did an excellent job of outlining.