Building AI Products with a Team of Five

When I tell people I work at a startup building photorealistic AI avatars, they imagine a large team: researchers, engineers, designers, a product org. The reality is that for most of the time I've been at Anam, the core team has been about five people.

I mention the number because it explains everything that follows: this is what building AI products looks like when there's no one to delegate to.

Everyone does everything

At a team of five, your job title is a suggestion. The work is defined by what needs doing today, not by what's on your LinkedIn profile.

In the last month, I've written product specs, debugged WebRTC issues, designed onboarding flows, run customer calls, and reviewed model evaluation results. Everyone on the team has a similar range.

The upside is that you develop a deep understanding of the entire product. There are no handoff points where context gets lost. When I write a spec, I understand the engineering constraints because I was debugging the system yesterday. When I'm on a customer call, I can speak to technical details because I reviewed the model outputs that morning.

The downside is that nothing gets the depth of attention it deserves. Every task competes with every other task for the same small pool of people. You're constantly making prioritisation decisions on the fly, not in a roadmap meeting but in the moment, choosing which Slack message to respond to first.

The AI-specific challenge

Building AI products with a small team has a particular difficulty that I don't think is well understood from the outside.

Traditional software has well-understood patterns for small teams. You can build a web app, a mobile app, even a reasonably complex SaaS product with five good engineers. The tools and frameworks have matured to the point where small teams can be remarkably productive.

AI products are different. The surface area is enormous. You need ML expertise for model development and evaluation. Systems engineering for real-time inference pipelines. Frontend engineering for the client experience. DevOps for GPU infrastructure. Product thinking for the actual user experience.

In a large company, each of these is a team. In a startup of five, each of these is a hat that someone wears on Tuesday afternoons.

What we got wrong

I want to be honest about the mistakes, because the startup narrative tends to skip to the lessons without acknowledging the pain.

We had a prolonged release process. Rather than shipping and iterating, we kept trying to build more of the product before putting it in front of users. We should have focused on growth systems earlier instead of adding features to something nobody was using yet. The feedback we got in the first week of production was worth more than the previous month of internal iteration.

Infrastructure is harder than it looks. We have a world-class infra engineer, which helps, but the challenges are enormous. GPUs are hard to get at a sensible price, and you end up with customers who are physically far from your datacenters, which directly impacts performance. When you're building real-time AI, infrastructure is part of the product, not a supporting function.

We tried to do too many things. For a while we were simultaneously building a consumer product, a developer platform, and a custom enterprise solution. For three different markets. With five people. It took an embarrassingly long time to accept that we needed to pick one and do it well.

What we got right

Hiring for breadth. Hiring has been hard, honestly. We've had several people not work out. But on the tech side, we've focused on finding the best people we can who can take on a wide range of specialities. Every person on the team can operate outside their primary skill set. That was a deliberate criterion, and I think it was the right call for this stage.

Shipping constantly. We ship tens of times per day now. We streamlined our release processes and testing pipelines so we can release in minutes with low risk. That velocity only works because the team is small enough that everyone understands the full system.

The most valuable discipline in a small team is the ability to stop working on something. Every hour spent on a lower-priority task is an hour not spent on the thing that matters most. This sounds obvious. In practice, it's one of the hardest things to do because every task feels urgent when you can see the full picture.

Talking to users constantly. With no dedicated sales or customer success team, engineers are on customer calls. This is inconvenient but incredibly valuable. There's no filtering layer between what users say and what the team hears. When a customer says "the avatar feels slow," the engineer who can fix it is in the room hearing the frustration directly.

The ownership trade-off

The flip side of a small team is ownership. You can trace the impact of your work directly, and there's no ambiguity about whether it matters.

That clarity is addictive, and it's what keeps people at startups despite the hours and the uncertainty.

Not unconditionally. Building AI products with a tiny team only works if you have the right people, and skill is the smaller half of that; they have to be comfortable with long stretches of ambiguity. At this size, one wrong hire is an existential risk rather than a management problem.

But if you have those people, there's a speed and coherence to the work that larger teams can't match. Decisions happen in minutes, not weeks. The product reflects a unified vision because there's no organisational distance between the vision and the code.

Five people building something that shouldn't be possible with five people. Most days, it feels exactly as difficult as it sounds.