The Narrative vs. The Numbers

Every major AI coding tool vendor publishes productivity claims in the 2x-10x range. GitHub says Copilot users complete tasks 55% faster. Cursor reports 2-3x speed improvements for certain task types. The enterprise pitch decks all have the same hockey stick slide.

Then there’s what engineering teams actually measure when they track before-and-after carefully.

LeadDev’s latest analysis of real production data across multiple engineering teams found that the median productivity improvement from AI coding tools is 10-30% — not 2-10x. The 10x cases exist, but they’re narrow: specific categories of tasks (writing tests from existing code, generating boilerplate, translating code between languages) where the AI is almost entirely removing human work rather than augmenting it.

For most engineering work, the improvement is real but measured. Understanding why the gap exists between the claimed and measured numbers is more useful than either dismissing the tools or believing the marketing.

Why Code Velocity ≠ Team Productivity

The most important insight from the data: code accounts for roughly 16% of a software engineer’s time.

That’s not a guess — it’s from multiple time studies done over the past decade. The rest of an engineer’s time goes to:

  • Understanding requirements (reading docs, asking questions, attending meetings)
  • Code review (reading others’ code, writing feedback, re-reviewing)
  • Testing and debugging (finding the thing that broke, not writing the test)
  • Deployment and operations (watching deploys, debugging production incidents)
  • Planning and communication (estimating, writing tickets, syncing with stakeholders)

If AI tools make you 10x faster at writing code — an extreme assumption — but leave everything else unchanged, total engineer productivity improves by roughly 10% × 10 = a 90% speedup on the 16% slice, which is about a 14% improvement in total throughput.

That’s actually close to what the data shows. The math works.

# Rough model of AI productivity impact
engineer_time_breakdown = {
    "writing_code": 0.16,
    "code_review": 0.20,
    "debugging": 0.18,
    "meetings_planning": 0.25,
    "testing": 0.12,
    "deployment_ops": 0.09,
}

def calculate_productivity_gain(ai_speedup_on_coding=10.0):
    """
    Even with 10x speedup on code writing,
    total productivity gain is much smaller
    """
    coding_fraction = engineer_time_breakdown["writing_code"]
    
    # Time saved on coding = fraction * (1 - 1/speedup)
    time_saved = coding_fraction * (1 - 1/ai_speedup_on_coding)
    
    # Overall speedup 
    new_total_time = 1.0 - time_saved
    overall_speedup = 1.0 / new_total_time
    
    return overall_speedup

# With 10x speedup on code writing → 1.19x total speedup (19% faster)
# With 2x speedup on code writing → 1.09x total speedup (9% faster)
print(f"10x coding speedup → {calculate_productivity_gain(10):.2f}x overall")
print(f"2x coding speedup → {calculate_productivity_gain(2):.2f}x overall")

The Hidden Costs in the Productivity Equation

The math above is generous. It assumes AI saves time on code writing with no additional costs. In practice, there are three hidden costs that eat into the gains:

Context switching overhead. Using AI tools well requires a different workflow than typing code. You write a prompt, evaluate the output, iterate if needed, verify the result. For simple tasks this is faster. For complex tasks where the AI output is wrong in subtle ways, the iteration cost can exceed just writing the code yourself. Studies suggest engineers spend 15-25% of their “AI-assisted” coding time in prompt iteration rather than the task itself.

Verification overhead. AI-generated code requires different review practices. You can’t just glance at it — you need to understand what it does, whether it handles edge cases, whether it matches the intent of the ticket. Teams that trust AI output without rigorous review accumulate bugs faster than teams who wrote the code themselves. The verification cost is real and often underestimated.

Knowledge decay. This one is slow and invisible until it isn’t. When engineers stop writing code in a domain because AI handles it, they stop building expertise in that domain. A year later, when the AI generates something wrong, no one on the team has the context to catch it. The “cost” of this doesn’t show up in short-term productivity metrics.

What the 10x Cases Actually Look Like

The genuine 10x improvements from AI tools are real — they’re just narrow. Here’s where the data shows outsized gains:

Test generation from existing code. If you have well-structured code and want to generate a comprehensive test suite from it, AI can genuinely do this 5-10x faster than writing tests manually. The AI is excellent at covering edge cases, generating parametric tests, and ensuring coverage of error paths. This is a task where the output is largely deterministic and verifiable.

Boilerplate generation for established patterns. CRUD endpoints following your team’s conventions, migration scripts, configuration files for new services — anything where the pattern is well-established and the AI just needs to fill in names and types. Measured speedups in this category are often 5x+.

Language/framework translation. Porting code from one language to another, or from one framework version to another, is dramatically faster with AI. The pattern-matching quality for translation tasks is genuinely high and the output is verifiable against the original.

Documentation and test data generation. Tasks that humans find tedious and tend to under-invest in: writing inline documentation, generating realistic test fixtures, creating API documentation from code. AI at 10x speed here is plausible and genuinely valuable.

The common thread: these are tasks where the output is verifiable against a clear specification, the task is isolated (doesn’t require deep system context), and the work is repetitive in structure even if the details vary.

What To Measure Instead

If you’re a Tech Lead trying to honestly evaluate AI tooling in your team, here’s what I’d track:

Task-level cycle time, not commit volume. Story points or ticket completion time from start to “done” in production. This captures the full cost including review, bugs introduced, and rework. Teams often see AI increase raw commit volume while story completion time stays flat or increases because reviews take longer.

Review comment density on AI-generated vs human-generated code. Are reviewers leaving more comments per line on AI code? More bugs found in review? This is a leading indicator of quality issues that will show up as bugs in production later.

New capability velocity vs. maintenance burden. Track how much engineering time goes to net-new features vs. fixing/updating existing functionality. AI can accelerate building new things while simultaneously generating technical debt that slows you down. If your maintenance burden is growing faster than your feature velocity, the productivity calculation is net negative even if individual tasks are faster.

Domain expertise retention. This is the hardest to measure but most important. Survey engineers about their confidence in specific technical domains over time. Teams that AI-delegate whole domains eventually lose the ability to evaluate AI output quality in those domains. That’s a strategic risk, not just a productivity metric.

The Right Mental Model

The 10x narrative isn’t wrong — it’s describing the ceiling of what AI tools can do in specific conditions. The problem is that ceiling gets mistaken for a floor or average.

The right mental model: AI coding tools are amplifiers for specific task types. They dramatically accelerate tasks that are pattern-based, self-contained, and verifiable. They provide modest speedups for complex tasks that require system context, creativity, and judgment. They can slow you down on tasks where the verification cost exceeds the generation savings.

Building a team that actually captures the gains means:

  1. Identify which tasks in your team’s work are in the “AI sweet spot” (pattern-based, contained, verifiable) and invest in tooling and workflow specifically for those.

  2. Don’t measure success by AI adoption rate. Measure by outcomes. A team that uses AI 20% of the time but perfectly targets the high-value cases will outperform a team that uses AI for everything and manages the resulting quality issues.

  3. Keep humans writing code in critical domains. Expertise requires practice. If AI handles all the code in a domain, no one on the team can evaluate whether the AI is producing good code in that domain. Deliberate human coding assignments aren’t inefficiency — they’re knowledge capital investment.

  4. Build verification capacity, not just generation capacity. The bottleneck shifts from “can we write this code” to “can we verify this code is correct.” Teams that invest in testing infrastructure, code review practices, and automated verification will capture more AI productivity gains than teams that just add AI generation tools.

The companies telling you their engineers are 10x more productive with AI are either measuring something narrow, have extraordinary implementation discipline, or are telling you what their investors want to hear. The companies actually capturing meaningful AI productivity gains are investing in the verification and knowledge systems to make AI output reliable — which is harder and less marketable than “we use Cursor.”

Thuận Lương is a Technical Lead with 15+ years of experience in .NET, cloud architecture, and AI systems. He writes about lessons learned building real production systems.

Export for reading

Comments