AI: Sticker Shock

...Pricing AI Apps & Services

While developers, businesses and customers large and small are all excited about LLM AI based applications and services, the pricing models for many have not been clear. This week we saw the proposed pricing for one of the most anticipated AI augmented services, CoPilot for Microsoft Office 365 As The Verge reports:

“Microsoft is putting a price on the AI-powered future of Office documents, and it’s a steep one for businesses looking to adopt Microsoft’s latest technology. Microsoft 365 Copilot will be available for $30 per user per month for Microsoft 365 E3, E5, Business Standard, and Business Premium customers.”

“That’s a big premium over the cost of the existing Microsoft 365 plans right now. Microsoft charges businesses $36 per user per month for Microsoft 365 E3, which includes access to Office apps, Teams, SharePoint, OneDrive, and many other productivity features. A $30 premium for access to Microsoft 365 Copilot will nearly double the cost for businesses subscribed to E3 that want these AI-powered features. For Microsoft 365 Business Standard, that’s almost three times the cost, given that it’s $12.50 per user per month.”

There’s been a fair bit of excitement around CoPilot for Microsoft Office 365 since it was unveiled a few weeks ago. But as the above report highlights:

“Microsoft is trying to overhaul its Office apps with its AI-powered Copilot service, allowing businesses to instantly summarize documents, generate emails, and speed up Excel analysis. Microsoft 365 Copilot certainly looks like a very compelling feature addition, and many genuinely believe it will change Office documents forever, but the cost could put a lot of existing Microsoft 365 businesses off adopting Copilot in the short term.”

As Axios explains further, this move by Microsoft is driven both by the currently much higher costs of AI services, and the incremental revenue opportunities to please Wall Street:

“Why it matters: That will add up to a hefty chunk of change, representing the most significant new revenue opportunity for Microsoft's Office business since it switched to a subscription model.

Details: Microsoft announced the Microsoft 365 Copilot pricing at its Inspire partner conference on Tuesday, along with a business version of its GPT-4-powered Bing Chat, which will sell for $5 per user per month on its own, and also be included in some of the company's subscription bundles.

• Bing Chat Enterpise adds protections designed to ensure that confidential business data doesn't get leaked out into the world.” (MY NOTE: THIS MEANS MICROSOFT IS CHOOSING NOT TO TAKE USER QUERIES AND USE IT FOR REINFORCEMENT LEARNING LOOPS ON THE UNDERLYING GPT4 LLM AI)

“Between the lines: That could add upwards of $5 billion to $16 billion in additional revenue for Microsoft next year, Ivana Delevska, chief investment officer at asset manager Spear Invest, told Axios. Her revenue estimate assumes 5% to 16% of Office 365 users sign up for Copilot.

• The $30 monthly per-user price was higher than the $5 to $20 per user per month many analysts had expected, Delevska said.

• On the flip side, Delevska noted it also costs Microsoft a lot to power its AI Copilots — on the order of $2 to $5 per user hour for the compute capacity needed to provide the service.

• "We do believe that this creates an opportunity for Microsoft, but it remains to be seen what value it will provide for its customers," Delevska said.”

“Yes, but: Generative AI services are pretty compute-heavy today, so there's considerable cost involved as well. Microsoft can draw on its existing Azure cloud computing infrastructure which is already providing AI services for OpenAI and others.”

Hundreds of business customers have been testing the product, and its release date is yet to be announced.  And as the piece above reminds us there is pending competition, 

“The software giant will face competition from Google, too. Microsoft’s Copilot announcement came just days after Google announced similar AI features for Google Workspace earlier this year, including AI-assisted text generation in Gmail, Docs, and more. Zoomand Salesforce have also been adding AI-powered features, so all eyes will now be on how Google, Zoom, and Salesforce handle pricing for their AI additions going forward.”

The prices for some of those applications and services are also not yet clear, and the market of course will sort out the levels that businesses and customers will ultimately bear.

But these are early days for LLM AI and Generative AI applications and services, and the underlying costs of developing and running the Nvidia and other GPU driven Compute around these services, are high right now given GPU chip shortages and the AI Gold Rush environment, and cloud datacenter, hardware and software vendors are trying to sort out their ultimate cost and pricing structures. As Runtime highlights here:

“Over the last 15 years cloud providers got very good at commoditizing the enterprise infrastructure needed to launch and run a business in the 21st century. This year, however, the generative AI boom has knocked them out of their comfort zone.”

“The cost of providing high-performance AI cloud computing workloads is surging, from the expensive (and hard to find) Nvidia AI chips where the magic happens to the energy needed to run the whole show. It's also clear that AI workloads are evolving differently on the cloud and that the infrastructure used to dial in general-purpose CPU-driven workloads at scale might need to be reinvented for the AI era.”

“AWS, first in traditional cloud computing but scrambling to catch up in AI, signaled this week that it plans to compete for AI business on price.”

“These models are expensive,” Dilip Kumar, vice president of AWS Applications, told Reuters this week. “We’re taking on a lot of that undifferentiated heavy lifting, so as to be able to lower the cost for our customers.”

• “He was referring to AWS's custom chips for AI workloads, such as its Inferentia and Trainium chips that, you guessed it, process inference and model-training tasks.”

• “AWS was the first cloud provider to design its own CPU for low-cost cloud workloads, and it's clearly hoping it can pull off the same trick to help it avoid paying as much of the Nvidia tax as possible.”

• “Building and maintaining a chip design team is not exactly cheap, but should customers find the results useful the effort could save AWS a ton in the long run.”

“But everyone is trying to get a handle on AI infrastructure costs.”

• “Google Cloud has been working on custom AI chips for several years, and probably has the most experience of the Big Three with running AI workloads outside Nvidia's orbit.”

• “Two University of Washington researchers with ties to Microsoft just published a description of a "chiplet" architecture that could dramatically reduce the cost of running AI models at scale.”

• “Even IBM is kicking the tires on chips designed internally for AI workloads in hopes of reducing operating costs.”

“It's still hard to tell how much real end-user demand for AI services, however, and that will obviously have a big impact on the price.”

• “CIOs recently surveyed by Jeffries ranked AI well below kitchen-table enterprise spending priorities like security and application development.”

• “But it's clear there is an AI startup arms race, with money pouring into the sector amid an otherwise dismal time for venture-capital investment.”

• “Startups developing these models can't get their hands on enough AI computing capacity, which combined with GPU rationing should keep prices relatively high for the rest of the year.”

“Still, regular businesses might balk at the current price tag for those services, especially coming out of a year in which they've been asked to scrutinize every dollar spent on technology.”

So it’s early days, for costs and prices around AI applications and services, but be prepared for sticker shocks and vigorous price discovery in the markets ahead. 

Ultimately, the AI industry will sort out their lower cost pricing Flywheels, and optimized pricing strategies. Competition will come from companies large and small, and open source AI hardware and software will also be a big factor as in cycles past. Stay tuned.