AI: About those AI costs again

…yellow flag on financial growth metrics

A few weeks ago, I wrote about the relatively high costs of the race to deliver LLM AI and Generative AI services to both businesses and consumers. Especially given the unique role of high computation intensive AI ‘reinforcement learning cycles’ for ever better results. And the trial and error process that will likely take at least three years to figure out pathways to sustained revenue, margin and profit growth. Those were yellow flags on the path to eventual AI markets, revenue and profitability growth.

I’ve highlighted that as in previous PC, Internet and other tech cycles, the AI Tech Wave too will take years to figure out the best products and services, and their eventual ‘product-market-fit’. 

This was underlined by a WSJ piece this week “Big Tech Struggles to Turn AI Hype into Profits”:

“Tech companies are touting new AI technology that can spit out business memos or computer code. They are still figuring out how those products will generate a profit.

“Generative artificial-intelligence tools are unproven and expensive to operate, requiring muscular servers with expensive chips that consume lots of power.”

“Microsoft, Google, Adobe and other tech companies investing in AI are experimenting with an array of tactics to make, market and charge for it.”

“Microsoft has lost money on one of its first generative AI products, said a person with knowledge of the figures. It and Google are now launching AI-backed upgrades to their software with higher price tags.”

“Zoom Video Communications has tried to mitigate costs by sometimes using a simpler AI it developed in-house. Adobe and others are putting caps on monthly usage and charging based on consumption.”

“A lot of the customers I’ve talked to are unhappy about the cost that they are seeing for running some of these models,” said Adam Selipsky, the chief executive of Amazon.com’s cloud division, Amazon Web Services, speaking of the industry broadly.” 

“It will take time for companies and consumers to understand how they want to use AI and what they are willing to pay for it, said Chris Young, Microsoft’s head of corporate strategy.”

None of this is surprising given the relatively high capital expenditures and investments in AI infrastructure companies in building up Foundation LLM AI and related hardware infrastructure. As discussed before, we’re talking investments in the hundreds of billions of dollars. And the fact that white hot demand for AI GPU hardware infrastructure, companies like Nvidia with over 80% of the current market and others, are in the catbird’s seat in terms of GPU allocations and availability at scale. This will likely remain tight for a year or two if not more.

Thus the pricing and profitability yellow flags.

Again, as the WSJ piece reiterates:

“AI often doesn’t have the economies of scale of standard software because it can require intense new calculations for each query. The more customers use the products, the more expensive it is to cover the infrastructure bills. These running costs expose companies charging flat fees for AI to potential losses.” 

“Microsoft used AI from its partner OpenAI to launch GitHub Copilot, a service that helps programmers create, fix and translate code. It has been popular with coders—more than 1.5 million people have used it and it is helping build nearly half of Copilot users’ code—because it slashes the time and effort needed to program.”

“It has also been a money loser because it is so expensive to run.”

“Individuals pay $10 a month for the AI assistant. In the first few months of this year, the company was losing on average more than $20 a month per user, according to a person familiar with the figures, who said some users were costing the company as much as $80 a month.”

The issue remains the same for AI driven productivity software, as I’ve discussed before. The WSJ again highlights:

“Microsoft is going with a higher price for its next AI software upgrade. On top of regular monthly charges—starting around $13 for the basic Microsoft 365 office-software suite for business customers—the company will charge an additional $30 a month for the AI-infused version. The AI-powered feature can be instructed to compose emails, create PowerPoint presentations and build Excel spreadsheets independently.”

“Google, which is releasing a similar AI assistant feature for its workplace software, will also be charging $30 a month on top of the regular subscription fee, which starts at $6 a month.”

“Microsoft, Google and others have gone with a flat monthly rate, betting that the higher additional charges will more than cover the average expenses of running the technology.”

The picture isn’t different with other software vendors trying out AI driven products like Adobe:

“Adobe uses a system of credits to help ensure its AI image generator, Firefly, won’t put it into the red. Once Adobe customers go over their allotted monthly credits, the company slows down the speed of the service to discourage overuse.” 

“We are trying to provide great value but also protect ourselves on the cost side,” Adobe CEO Shantanu Narayen said. “

Again, the WSJ underlines:

“Companies expect generative AI to get cheaper over time, as happened with many technologies, including cloud storage and 3-D animation. New chips and other innovations will likely drive the processing costs down.”

“OpenAI lowered the price it charges for using its older AI earlier this year. Its free viral chatbot ChatGPT uses an older version of its software. Customers who want a ChatGPT powered by its latest version have to pay $20 a month. “

None of this is surprising in terms of high starting costs for ground-breaking new technologies in the early stages of a major tech cycle, especially as the core LLM AI services go ‘multimodal’.

The key difference to keep in mind here with AI products and services, is the high amount of continued ‘Compute’ costs for both training and inference of AI ‘reinforcement learning loops’, which uniquely deliver better, more relevant, reliable and safer results to users over time. And of course that each AI computation requires thousands to millions of time greater computation cycles than traditional software. On expensive AI hardware, both in the cloud and on local devices.

Those costs will come down but will take years most likely. So the experimentation with costs and pricing for AI products and services will continue. And this will remain a gating factor for how fast these services ramp in usage by consumers and businesses. It will all likely happen, but with more headwinds in this wave vs waves in the past. It may all take a bit longer than usual too ‘get there’.

The financial and secular cycle correlations will have to be watched closely, especially given the current private and public market enthusiasm vis a vis AI valuations, especially for Foundation LLM AI companies and pending transactions.

Thus the yellow flag, despite the secular need to accelerate ahead in these early days.

Stay tuned.

(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here)