AI: Going Small to go Big

...local AI hardware and software a big driver

The Bigger Picture, Sunday November 5, 2023

The financial momentum and secular AI tech enthusiasm continues for Tech stocks big and small, as the AI Tech Wave enthusiasm continues to drive investor optimism. In this Sunday’s ‘The Bigger Picture’, I’d like to underline again that both ‘Big AI’ driven by Foundation LLM AI technologies will be meaningfully catalyzed by ‘Small AI’ applications and services. They’ll be driven increasingly by AI hardware and software working closer together in a multimodal way on our local devices as well as multi-billion dollar AI cloud data centers. They’ll have far humbler ambitions than grander general AI ambitions, but will accelerate us getting there more safely and reliably. Let me explain.

Barron’s underlined this point this weekend with their cover story “AI is about to remake the PC”:

“‘The killer app of AI,”’says Dell Vice Chairman Jeff Clarke, “will be that you’ll love your PC again.”

Again, it’s Box number 6 at the top of the AI Tech Wave value chart I’ve often discussed:

“The working theory of AI has been that it requires big, powerful computers, driven by hard-to-find graphics processors, primarily from Nvidia (NVDA). All of that computing—the creation of large-language models plus their continuing use—happens in the cloud. Meanwhile, laptops, desktop PCs, and even mobile phones become simply access points to the cloud, where AI services like ChatGPT do their computationally intensive magic.”

“Even before AI, consumer and business laptops had largely become dumb terminals for using online platforms from Amazon.com (AMZN), Microsoft (MSFT), Alphabet‘s (GOOGL) Google, Meta Platforms (META), and Apple (AAPL). Documents are in the cloud, email is in the cloud, photos are in the cloud, music is in the cloud. “The network is the computer,” Sun Microsystems computer scientist John Gage presciently said 40 years ago.”

“PC makers are looking to change that paradigm. They are readying AI personal computers, with the first models set to arrive in the next few months. The microprocessor companies are excited, too. The common goal is to enable PC users to run generative AI applications right on their desktops, whether connected to the network or not. “

The key technical change that’s accelerating this trend are specialized AI chips that will run smaller, efficient LLM AI models directly on our phones and PCs, as the Barron’s piece continues to highlight:

“AI PCs won’t look different than the one now sitting on your desk. But on the inside, they’ll have a system-on-a-chip that includes three important elements: a conventional CPU, or central processing unit; a GPU, or graphics processing unit; and—here’s the new wrinkle—an NPU, or neural processing unit. Neural processing is a way of processing very large data sets efficiently. And the NPU will pick up most of the AI-specific computing requirements. “

As I’ve highlighted before, Apple has led on this trend with over two billion devices in user hands:

“Apple has been including a neural engine as part of its homegrown processors since it introduced the A11 processor in the iPhone 10 in 2017.”

“This past Monday night, Apple unveiled its latest MacBooks during a rare prime-time event, a sign of its renewed focus on its own PC division. The latest MacBook Pros, along with an upgraded iMac, will all use a version of the company’s new M3 processors. Apple said the high-end version of the chip, called the M3 Max, will be capable of running complex AI workloads.”

Chip makers AMD, Intel, Qualcomm, Micron, and others are all accelerating their efforts with AI chips particularly optimized for local ‘reinforced learning loops’ for AI inference at ‘the Edge’, which are going to be the primary driver of how users augment themselves with AI going forward:

“Intel recently unveiled an “AI PC Acceleration program,” which is intended to jump-start applications leveraging local use of AI techniques, in areas like content creation, security, audio effects, and video collaboration. “AI is good at generating something from nothing, or saving time in laborious human tasks,” Intel’s Hallock says.”

“IBM CEO Arvind Krishna says the rise of the AI PC is “a given.” In an interview with Barron’s, he explained the history of the PC industry as a set of major chip upgrades that revolutionized the device and opened the way to new applications.”

“About 35 years ago, he says, Intel rolled out a chip called the 80387 that worked with x386 processors for computationally intensive applications like spreadsheets.”

“PCs later added GPUs, enabling games and video editing. And now PCs are adding NPUs, for artificial intelligence. Krishna thinks every PC before long will have the ability to run AI models with 10 or 20 billion parameters. He imagines PCs that run electrocardiograms to check users’ heart function at home without connecting to external networks for analysis, or that use voice conversations without the need for any text input.”

“It changes the market for the chip makers,” he says.”

“Alex Katouzian, a senior vice president at Qualcomm, says AI PCs based on Qualcomm’s Snapdragon X Elite processors will be able to run multiple AI models in the background at the same time. He imagines, for instance, the ability to summarize text and compile a list of action items from an ongoing meeting.”

“Qualcomm claims that the new Snapdragon processor is faster—while using less power—than chips from Intel and AMD. The first PCs using the new Qualcomm chips should be available next year. “

Apple in particular in my view will lead the AI wave, despite the industry’s comparative concern of their fortunes vs the rest of the tech companies driving AI:

“Apple’s latest quarterly report highlighted a concerning issue for the world’s most valued company: It simply isn’t growing. And no one can say when the malaise will come to an end.”

“In the latest quarter, Apple (ticker: AAPL) recorded revenue of $89.5 billion, down 1% from the year-ago period. That marked its fourth consecutive quarter of declines. Apple’s latest results are a contrast from the other Magnificent Seven tech giants. In the September quarter, Tesla ’s (TSLA) revenue was up 9%; Alphabet (GOOGL) grew 11%; Microsoft (MSFT) and Amazon.com (AMZN) both expanded 13%; and Meta Platforms (META) increased 23%. Nvidia (NVDA) blew away everyone, with 88% growth in its latest quarter.”

Apple has a lot to leverage with AI hardware and software at the edge on local devices in my view, and as Barron’s again highlights:

“To be clear, Apple has built a fantastic business. The company has more than one billion subscribers to its various services, ranging from music and streaming video to financial services and data storage. And it has an installed base of more than two billion hardware devices.”

So overall, it’s encouraging to see the industry focused on AI tech and infrastructure on both ends of the ‘barbell’. From the big cloud AI data centers, to the smallest of devices running AI on the user’s persons in the form of ‘Wearables’ and beyond.

From OpenAI, Microsoft, Google, Amazon/Anthropic, Meta, Nvidia, Apple and Tesla, along with a host of private AI companies, like Humane AI, Rewind AI and many others. Despite sometimes overwrought regulatory concerns and actions around AI, we’re seeing a positive set of secular technology innovation rise up to supplement the financial enthusiasm around AI.

It’s going to be both ‘Big and Small AI’ making it all happen on both ends of the infrastructure spectrum, the cloud and the devices around us. It’s accelerating, particularly driven by open source AI initiatives.

A lot to look forward to with open eyes and contained existential and geopolitical fears. Big things to come from Small things ahead on all things AI. Stay tuned.

(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here).