This means that as we go from hundreds of millions to billions of users talking to their favorite AI chatbots, running countless prompts and queries galore, the variable costs for the providers, both directly and indirectly, add up quickly. And thus even a flat subscription price of $20/month let’s say, quickly goes into the red.
And it’s accelerating as both businesses and consumers ramp up their AI use. All that ‘AI Slop’ and more quickly adds up.
Yes, the models themselves are getting cheaper to run, both via software and hardware innovations (Nvidia, and AI data center efficiencies beneath the hood). And at impressive rates every few months. BUT, the overall volumes of usage are growing much faster in the short term than the costs at times. And it’s a tough task managing through the ups and downs of AI quality and costs.
“The overarching narrative of the past two years has been that AI models are getting cheaper for customers.”
“Just take a look at this chart, showing how the cost of purchasing access to a GPT-4 level AI model from OpenAI has dropped more than 85% since it was released in early 2023. And that’s without getting into the rise of cheap, open-source AI like DeepSeek which offers more low-cost alternatives to developers.”
“So it’s interesting when we hear from AI application developers that the models they buy are still just too darn expensive. As a result, many app developers are struggling to get their gross profit margins anywhere near 70% or 80%, the kinds of margins enjoyed by traditional software businesses.”
It’s tough when customers when they can make a choice on the LLM AI model to use on OpenAI, Anthropic Claude, Google Gemini, xAI Grok, Perplexity etc., more often than not will pick the latest models, both for AI reasoning and regular AI Agentic queries. So even when the older models are far cheaper, the costs for the latest models are higher and ramp up quickly.
“There are a couple potential reasons for this. Crystal Huang, a general partner at GV, told me that, while the costs of older AI models are dropping, AI labs can still charge a pretty penny for each new state-of-the-art model they release, sometimes dozens of times higher than older models.”
“And startups often rush to incorporate the newest models into their product, hoping to attract new customers. For instance, just hours after Anthropic’s Claude 4 was released in May, a number of coding assistant startups including Anysphere and Cognition said their customers could use the model to power their coding assistants.”
Obviously, older models, and open source models especially from providers like Meta with Llama, DeepSeek with R1 and soon R2, etc., all make a difference. But it’s the mix shift that matters. And complicated while the underlying AI applications scale at various levels.
“You might think that, over time, the AI startups that buy models from Anthropic and OpenAI would find cheaper, older models to save on costs—and that is true. However, because these markets are competitive, startups are also under a lot of pressure to use the state-of-the-art models, even if they’re only marginally better, to eke out whatever small performance gains are possible for their customers.”
“The expenses of viral AI features can also sneak up on developers. One founder of an inference provider told me that their customers have to be very careful when releasing new AI features because if they become extremely popular, the costs of tapping the AI models that power the features can quickly exceed the monthly subscription fee the startup is charging its customers.”
“For example, a simple AI chatbot feature in a moderately popular consumer app can easily cost between $500,000 to $2 million per year, even with a cheap model, they said.”
That makes for tougher budgeting, especially for the small to medium sized AI applications and services provider.
“To counteract this, developers are trying a number of approaches including using small, cheap models like Alibaba’s Qwen models to power as many features as possible. And makers of mobile apps powered by AI are trying cheap or free on-device models to handle some features, thanks to new iPhone developer tools that have generated excitement, as my colleagues recently wrote.”
“For the time being, it seems that as long as AI startups can continue raising funding from overeager VCs, they don’t have to worry too much about operating with poor margins. But don’t be surprised to see more companies switch to usage-based pricing from fixed subscription pricing as they seek to cover a greater portion of their costs.”
All this means that the variable cost conundrum is a fixture of AI companies as they feel their way to the best pricing at various tiers and bundles. And a lot of uncertainty for investors along the way at this stage of this AI Tech Wave. Stay tuned.
(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here)
