Our most effective ICs all share a common trait: they’re former managers.
We’re aggressive early adopters of AI-augmented engineering at Tolan, even in places like our native iOS app. Like others we saw a step-change in the efficacy of long-running, concurrent agents starting in early December.
Surprisingly, the engineers who benefited most from that step change weren't necessarily the ones who had already pushed hardest on incorporating AI into their work. Instead, the ones with management experience – people like Dan Federman and Aseem Kishore, former eng managers who were early on our small team – suddenly found themselves with a new superpower.
We quickly realized that running concurrent agents was a totally different skillset than other forms of AI augmented engineering. It turns out that managing agents requires the same skills and judgment that it takes to manage people well.
Given the slope on the METR time horizon evals, we’re betting that this trend of effective concurrent / parallelized agents will only accelerate, so we’ve pivoted how we’re hiring by creating a new role: Agent Engineering Manager. It's an IC position, but we're recruiting, compensating, and evaluating performance on the basis of how well you manage AI agents.
This post is about how we got there: how we define this role, how our agent engineering managers work today, and why we think other teams will gravitate toward a similar approach.
The role
The thesis behind the role is straightforward: If the most important skills for working with AI agents are management skills, then we should hire for management skills. The best fit is almost certainly a tech lead or someone in a similar scale management role who still loves to write code themselves.
The role does not come with any human direct reports, and we intend to keep our team quite small, so folks who join us today might never have a team of junior engineers to manage.
And yet - the role requires every skill that makes someone a great engineering manager. During our interview loop, we evaluate Agent Engineering Managers on their ability to:
- Break down ambiguous product problems into well-scoped tasks
- delegate those tasks with appropriate milestones and planned checkpoints
- Coach and redirect agents quickly and effectively as they surface for help
- Conduct thorough code reviews of completed work
- Intervene manually when needed to ship well-tested features
If you’re curious for more detail, we published an IC mobile engineering oriented flavor of this interview loop here.
The work
Like any other leader, Agent Engineering Managers carefully balance their time across three competing demands:
Setting up the team for success: creating and maintaining the infrastructure and documentation that makes it possible for AI agents to contribute to the codebase sustainably.
Directly managing: living within the loop of a set of agents working concurrently, prompting / coaching / redirecting as needed throughout the day.
Jumping in: on a small team, managers often roll up their sleeves and contribute directly. Agent Engineering Managers often do the same to get features over the finish line.
Our iOS tech lead, Dan, recently published more context on how he works. It’s a great preview into the day to day role of an agent engineering manager.
Beyond our team
As each generation of technology matures, we tend to create new titles and teams that reflect the new skills required to operate successfully. Many folks working today lived through the shift from SysAdmin to DevOps as the frontline infrastructure engineering role. Similarly, the shift of many companies to React Native and similar frameworks meant that many mobile engineers today do not necessarily write much native code, and roles / job descriptions evolved accordingly.
Similarly, we expect that traditional titles and leveling around both backend and product engineering should evolve to map to the new reality. Creating a role dedicated to managing agents is a first step toward treating this shift as a unique category of work: One that needs its own best practices, its own evaluations, and its own ways of thinking about performance.
How do you evaluate someone's ability to manage agents? How do you distinguish between someone who uses agents to ship fast and someone who uses agents to ship well? What does it mean to scale your impact when your “team” is a fluid number of agents? How do we evaluate individual engineering excellence as discrete from agent management excellence? Does that distinction even matter today?
These are the questions everyone in our industry will be answering in the coming years.
Get in touch
If you're an engineering manager who's been quietly experimenting with agent-assisted development and wondering what it would look like to go all in, we'd love to talk - send me a note at quinten@portola.ai
