Low Margins, High Potential: Rethinking AI App Economics

Founders building AI applications are often told their gross margins should look like SaaS. But is that realistic? Or even healthy?

The assumption is that AI apps should hit 80%+ gross margins fast. But AI plays by different rules.

Gross margin is one of the simplest ways to tell how efficiently a company turns revenue into usable profit. The higher the margin, the more scalable the business because it means each new dollar of revenue costs relatively little to deliver.

This is why the market loves software. Build it once, and when you scale, you keep a massive portion of every sale. 

Traditional software on-premises companies already had this advantage from selling the same product repeatedly with minimal delivery cost. Then SaaS came along and made those economics even more appealing by turning one-time licenses into predictable, recurring subscriptions and centralizing delivery through the cloud. 

Margins don’t change much in software because the core direct costs - cloud hosting, customer support, and third-party software - are low as a percentage of revenues from the start. Once the core product is built and hosted, the incremental cost of serving each new customer is minimal so gross margins remain high and stable.

These SaaS leaders launched with very high margins, then maintained or slightly improved over time. 

Investors didn’t just see margins; they saw machines for scalable, compounding growth. Gross margin became a shorthand for quality, capital efficiency, and long-term value creation.

But for AI applications, gross margin can be misleading.

The whole promise of AI is that it serves the specific user. It learns from your data, tunes to your behavior, and gets better the more it’s used.

To deliver that kind of personalization, AI applications require a different set of costs.

These additional costs make the product smart, sticky, and hard to replicate.

A low gross margin today doesn’t mean low gross margin at scale. It can be a signal you’re investing in the right things - differentiation and future operating leverage.

Trying to hit arbitrary margin benchmarks too early can lead to one of two things:

• Overpricing your product to justify your economics

• Underinvesting in what makes your product useful

Either one kills momentum.

So let’s reset the expectation. Gross margins matter. But if you don’t read them in the right context, you’ll miss the whole picture.

Gross Margin Isn’t a KPI. It’s a Design Choice.

Gross margin reflects the cost of delivering your product. But what goes into that cost isn’t standardized - it depends on how you build.

• Are you using large, general-purpose foundation models or small, task-specific ones?

• Do you have human-in-the-loop validation steps?

• Are you onboarding or configuring for each customer?

• Are you delivering through dynamic interfaces like real-time voice or video?

Each of these decisions shapes your margin profile. More “intelligent” models are more expensive. And greater personalization or manual involvement drives up delivery cost. 

These aren’t flaws - they’re strategic choices that define the product experience and long-term defensibility.

And crucially, two of the most margin-dragging costs in AI applications get better with time:

• Model costs are falling. Foundation model pricing has dropped significantly over the last year, and competition among providers continues to push prices down. Companies that fine-tune smaller models for specific use cases will benefit even more.

• Human-in-the-loop dependence fades with maturity. Early on, humans may validate or curate outputs to ensure quality. But over time, feedback loops, improved training data, and automation reduce that need. Every new customer, and every interaction improve the product and lower the unit cost to deliver.

And costs will continue to fall:

✅ Moore’s Law of AI: New chips and inference-optimized hardware (e.g. NVIDIA H100s, Google TPUs) drive down compute cost per token. 

✅ Model optimization techniques reduce model size and inference cost without significant performance loss.

✅ Open-source innovation: Smaller, cheaper open-source models (e.g. Mistral, Phi-3, LLaMA) are catching up in quality.

✅ Use-case specific models: Companies are increasingly training or fine-tuning models on proprietary data, allowing them to use smaller, cheaper architectures without sacrificing accuracy.

✅ Marketplace competition: Model providers are racing to lower prices as a differentiator, leading to steep pricing drops.

✅ Inference providers & infra commoditization: New players like Together.ai, Abacus, and Deep Infra are offering dramatically cheaper hosting and inference options.

Together, these forces point to a future where delivering AI products becomes significantly cheaper, even as the user experience gets more sophisticated.

This isn’t to say that AI apps can’t eventually reach SaaS-like margins. As models are fine-tuned, infrastructure gets optimized, and onboarding becomes more automated, the cost to serve each additional customer drops, sometimes dramatically. But that takes time, iteration, and scale.

Case in Point: Duolingo’s AI Flywheel

In Q4 2024, Duolingo’s gross margin dipped 120 bps year-over-year, driven by increased AI inference costs. And yet, its enterprise value continued to rise, driven by record user growth and an expanding premium subscriber base.

Duolingo isn’t just a language-learning app, it’s one of the clearest examples of what a scaled AI application can look like. Leveraging generative AI, Duolingo launched 148 new language courses in a single year - a pace that previously took over a decade.

Today, Duolingo trades at over 25x forward revenue, one of the highest multiples in consumer tech. And yet, its gross margins hover around 71%, lower than traditional software businesses. 

Why? Because the market sees what’s being built: a product with deep personalization, habit-forming user experiences, and long-term monetization leverage. The lower margin is not a flaw, it’s a reflection of deliberate choices: inference costs, dynamic content generation, and user-specific tuning. These are the things that make the product smarter and harder to replicate.

A big part of that margin dip came from Duolingo Max, the company’s premium subscription tier that includes AI-powered features like Roleplay and Explain My Answer, built on GPT-4. These features require more costly inference, particularly for real-time, interactive content.

But Duolingo’s leadership is clear-eyed about the tradeoff. As CFO Matt Skaruppa noted on a recent earnings call, “that will change over time as the price of the generative AI used for video call comes down. And we saw in Q1 new models released with lower pricing so we would expect  the gross margin of Max to increase.”

Margin expansion is a function of scale and model evolution, not the primary focus today.

Investors also looked past the short-term margin compression. They saw the flywheel: AI-powered content, rising engagement, stronger retention.

Duolingo Valuation vs Gross Margin %

Don’t Confuse Gross Margin With Business Quality

Here’s the problem: when some investors see 50% gross margins instead of 80%, they assume something’s wrong.

But that’s like criticizing Amazon in 2001 for warehouse costs. Or Netflix in 2012 for streaming bandwidth.

Gross margin is an output of how your product is built. It’s not a measure of success. The better question is: are you building toward defensibility and operating leverage?

Don’t Chase SaaS Margins, Instead

1. Tell the story of your cost structure Show what’s in COGS. Separate variable vs. fixed costs. Flag what scales and what doesn’t.

2. Focus on where margins go, not where they start Duolingo’s team doesn’t obsess over gross margin today. They’re investing in engagement and defensibility. That’s how you get to strong unit economics later.

3. Price for strategic position, not spreadsheet aesthetics You don’t win AI markets by squeezing early margins, you win by building indispensable products and accumulating user data, behavior, and feedback that compound over time. That may mean pricing aggressively today to accelerate adoption, knowing the economics improve as model costs fall, automation increases, and operating leverage kicks in.

4. Avoid vanity metrics A clean-looking P&L with 85% gross margins doesn’t help if your product can’t scale. A messy one might be better if it reflects the right foundation.

If you’re Pitching Investors, use gross margins to show intentional design, not performance. You don’t need to apologize for costs that will shrink over time.