Semiconductors Remain a Key AI Gating Factor. ARD #104
...OpenAI, SK Hynix & the Market around chips and more.
Today’s theme: semiconductors remain a key AI gating factor — whether it’s the AI GPUs or the memory chips, especially on the local-computing side, where computer and smartphone prices are all heading in one direction for the next couple of years. Even Apple has signaled it won’t be immune, despite its huge global supply-chain advantages. Three events I’d like to dig into for the AI Tech Wave — each with my Take first, then my Overall Take.
(1) OpenAI’s ‘Jalapeno’ AI Chip — An AI-GPU Diversification Side Quest for Inference Compute
MP TAKE: OpenAI fired up ‘Jalapeno,’ its first homegrown AI chip, co-designed with Broadcom and optimized for LLM inference — not training. It’s the latest tell that every top Nvidia customer remains focused on its own ASIC AI-chip alternative, both for real and for show — even while staying dependent on Nvidia for 60-70-80% of its AI-chip needs, at least through the end of this decade.
Most of these in-house efforts target the inference side — the compute that creates the tokens when you and I prompt and converse with AI. Note that Nvidia isn’t ceding this: Jensen makes the point at every keynote that on inference, Nvidia’s chips (the Vera Rubin class) remain state-of-the-art on efficiency per token — the metric that matters when tens of billions are going into data centers. So while alternatives like Jalapeno are interesting technically, they’re an add-on to the broader show. For now the market is big enough for all of it — the in-house ASICs and Nvidia’s majority supply. To widen the lens: these are complements, not replacements — for OpenAI this is a logical step that will still take a lot of investment and time to bear fruit at scale.
Sources, in narrative order: Axios — OpenAI fires up ‘Jalapeno,’ its first homegrown AI chip, with Broadcom. OpenAI — OpenAI and Broadcom unveil LLM-optimized inference chip. For longtime readers, in narrative order: OpenAI’s ongoing mega AI deals, now with Broadcom in AI-RTZ #874; and ‘Nvidia vs its Top Customers for AI GPUs’ — frenemies to the end in AI-RTZ #574.
(2) SK Hynix Strikes the US Market While the Iron Is Hot
MP TAKE: A timely move by SK Hynix — out of South Korea, now the biggest of the memory names, having just overtaken its trillion-dollar cousin Samsung — targeting a $29-billion-plus US listing via a Nasdaq ADR, reportedly as soon as early next month. It’s consistent with companies in the US and elsewhere tapping equity and debt markets for tens of billions in additional AI capital, echoing Nvidia’s recent debt raise and Google’s equity-side move.
The three core global memory companies — SK Hynix, Samsung and Micron (the third, in Idaho) — now each trade at market caps over a trillion dollars, a sharp break from their historically cyclical position at the lower end of the tech stack. They’re posting software-like margins of 80%-plus, levels never seen before this multi-trillion-dollar AI-infrastructure wave, as they shift wafer capacity to the higher-margin high-bandwidth memory (HBM) for AI data centers. That same shift is why the average memory share of an iPhone or laptop’s bill of materials has gone from ~5% to 30-40% and rising — and why Apple is signaling price increases. The new SK Hynix supply will help on the margin, but it takes years to come online. To widen the lens: directionally right, with higher volatility in the near term — memory has been cyclical for forty years, and old habits can reassert before the decade’s crunch eases.
Sources, in narrative order: Reuters — SK Hynix targets $29B-plus US listing as AI demand surges. CNBC — SK Hynix plans to raise $29 billion via Nasdaq listing as soon as July 10. For longtime readers: ‘SK Hynix ready for its AI close-up’ (December 2024) in AI-RTZ #557.
(3) Chip Stocks Ebb With Tech Stocks After a Historic Year-to-Date Run
MP TAKE: On the seventh day, metaphorically speaking, the AI and chip stocks rest a bit — a retracement across the broader tech tape and chip stocks in particular, amid the latest round of ‘AI bubble’ fears. But the fundamental, bottoms-up demand drivers remain intact, especially for the global memory oligopoly — SK Hynix, Samsung and Micron — with other names like Sandisk, Kioxia and others not far behind globally.
Given the multi-hundred-percent moves in these names in recent months, some downside volatility is par for the course (and none of this is stock advice). On the competitive, extra-supply side, there aren’t many alternatives — other than ramping memory makers in China, which currently face US government restrictions, even as US companies like Apple have said they could use that extra DRAM supply. How that sorts out may ride on the Xi-Trump meeting in DC this September and the administration’s desire for a broader tech deal. Worth noting the irony: Microsoft, a core catalyst of the AI wave with OpenAI, is now seeing its Windows PC empire hit by a 10-20%-plus unit retracement on exploding memory and storage prices — even its Xbox consoles are seeing 20-30%-plus price increases. To widen the lens: a pullback after a run this steep is healthy, not thesis-breaking — the demand wave and the supply bottleneck are both still in force.
Sources, in narrative order: Axios — Tech stocks slump as AI bubble fears loom. For longtime readers, in narrative order: ‘The investment FOMO around AI supply chains’ in AI-RTZ #1083; and ‘Big $$ private/public cross-over FOMO momentum’ in ARD #72.
MP OVERALL TAKE
All three events highlight the gating factor for all things AI: chips. Not just the AI GPUs that have boosted Nvidia and the Google TPUs to date, but also the memory-chip companies that are now upending the local computer- and smartphone-pricing picture globally.
The short supply across all these chips going into the end of this decade means a reversal of the multi-decade Moore’s Law trend of ever-lower tech-equipment prices with ever-improving capabilities. That trend is on a global ‘pause’ for at least two to three years — until more supply and/or competition comes online at scale, with the only near-term relief likely coming from geopolitical progress on US companies accessing extra memory supply in China, still in its early stages. Watch this space, through this AI Tech Wave.
Gadget AI — Google Back in the Home-Speaker Game With a New $100 Gemini Speaker
MP Take: Google’s re-entry into the home-speaker market — following its earlier Nest speakers — gets lukewarm reviews, more for its audio quality versus both its own earlier offerings and comparable devices from Amazon and Apple. Gemini AI is the key feature, but Gemini is already available across Google’s other devices — on smartphones and other channels. So the notion of a $99 device that leans on a subscription to unlock the higher-end Gemini features you can already get elsewhere makes for a less compelling offering in this version.
If only it had a display and a clock — like the Amazon offering. Expect Apple to announce upgrades to its home-speaker lineup soon, with the new Siri AI launching to the mainstream this fall. It’s Google’s shot ahead of that fall rush — but a home-speaker market that’s still trying to figure out which AI features actually stick to the wall.
Sources, in narrative order: Bloomberg — Google’s new $100 speaker makes a weak case for Gemini AI in the home. For longtime readers: ‘The Glass Half Empty on AI Devices’ in AI-RTZ #872.
Questions
Q1 — What’s MP’s favorite feature with these home devices?
Still the prosaic features — time, weather, timer, and music playback — across the devices from Amazon, Apple and Google over the last few years. Boringly enough, I don’t use them much beyond those categories, despite the tens of billions invested to date — which is exactly why companies like Google keep coming in and out of the market, and why they’re back in now with AI and Gemini.
Q2 — What’s MP’s least favorite feature with these devices?
The wake words — the lack of multi-device awareness. Even though they’re supposedly ‘always listening,’ none of these devices reliably knows which one to respond with when you say “Hey S,” “Hey A,” or “Hey G.” More than half the time it’s the device on the other side of the room rather than the one in your hand. I’ve gone deep on the technicals — it’s a tough problem, made worse because there’s no horizontal standard across vendors, so each company’s own approach mucks up the fix. It’s one of the core problems to solve before AI devices can succeed — the biggest frustration coming out of the reviews, and they need to fix it.
Full Source Reading —
For the broader context, see the canonical sources for ARD 104 — in today’s narrative order:
Event 1 — OpenAI’s ‘Jalapeno’ AI Chip
Axios — OpenAI fires up ‘Jalapeno,’ its first homegrown AI chip, with Broadcom
OpenAI — OpenAI and Broadcom unveil LLM-optimized inference chip
AI-RTZ #874 — OpenAI’s ongoing mega AI deals, now with Broadcom
AI-RTZ #574 — Nvidia vs its Top Customers for AI GPUs (frenemies to the end)
Event 2 — SK Hynix US Listing
Reuters — SK Hynix targets $29B-plus US listing as AI demand surges
CNBC — SK Hynix plans to raise $29 billion via Nasdaq listing as soon as July 10
AI-RTZ #557 — SK Hynix ready for its AI close-up (December 2024)
Event 3 — Chip Stocks Ebb With Tech Stocks
AI-RTZ #1083 — The investment FOMO around AI supply chains
Gadget AI — Google’s $100 Home Speaker
Bloomberg — Google’s new $100 speaker makes a weak case for Gemini AI in the home
AI-RTZ #872 — The Glass Half Empty on AI Devices
Shorts Clips from today
Clip 1 — AI Wake Word Challenges
MP’s biggest gripe with home AI devices: they can’t reliably tell when, and which, device is being called. They answer when not asked — they’re ‘always listening’ — and often respond on the device furthest from you rather than the one in your hand.
MP Take: The companies still haven’t solved the multi-device, multi-direction wake-word triangulation problem. There’s no horizontal standard across vendors, and each one’s own approach mucks up the fix. It’s a core problem to solve before these can be truly useful home or work ‘companions.’
Clip 2 — SK Hynix’s $29B US Listing
SK Hynix is striking the US market while the iron is hot — targeting a $29B-plus listing via a Nasdaq ADR, reportedly as soon as July 10, to fund memory-capacity expansion.
MP Take: A timely move, consistent with companies tapping equity and debt markets for tens of billions in AI capital — echoing Nvidia’s recent debt raise and Google’s equity move. The proceeds expand supply, which takes years, against extraordinary demand for high-bandwidth memory in AI data centers.
Clip 3 — Memory’s Trillion-Dollar Trio
The three core global memory companies — SK Hynix, Samsung and Micron — now each trade at market caps over a trillion dollars, re-rated from cyclical commodity names into AI Tech Wave growth stories as they shift wafer capacity to high-bandwidth memory for AI data centers.
MP Take: They’re now posting software-like margins of 80%-plus — levels never seen before this multi-trillion-dollar AI-infrastructure wave. That’s a sharp break from their historically cyclical position at the lower end of the tech stack. Directionally right, with higher near-term volatility — memory’s 40-year cyclicality can still reassert.
Clip 4 — AI Home Speaker Market Heats Up
Google is back in the home-speaker game with a new $100 Gemini speaker — its shot ahead of a fall rush of AI home devices from Amazon (Alexa+) and Apple (new Siri AI). Early reviews call the audio quality middling versus rivals.
MP Take: Gemini is the key feature, but it’s already on Google’s other devices — so a $99 box that leans on a subscription for higher-end Gemini is a less compelling offering this version. If only it had a display and a clock, like Amazon’s. A market still figuring out which AI features actually stick.
About AI Ramblings Daily (ARD), and AI-RTZ
Both are daily. Both are free. Both are about AI. But they’re different mediums carrying different messages.
AI-RTZ is the morning text — a deeper written take on one idea, published by at least 5 AM EST. Today: post #1127.
AI Ramblings Daily is the afternoon video + podcast — my ad hoc takes and perspective on the day’s AI issues & news flow, around 20 minutes, with short 1-2 minute clips for quick topic views. Today: episode #104.
Subscribe to either or both on michaelparekh.substack.com. They run as separate Sections you can opt into or out of.
Links used in today’s show (already embedded inline above; listed here for reference)
Take 1 — OpenAI’s ‘Jalapeno’ AI Chip:
Axios — OpenAI fires up ‘Jalapeno,’ its first homegrown AI chip, with Broadcom
OpenAI — OpenAI and Broadcom unveil LLM-optimized inference chip
AI-RTZ #874 — OpenAI’s ongoing mega AI deals, now with Broadcom
AI-RTZ #574 — Nvidia vs its Top Customers for AI GPUs (frenemies to the end)
Take 2 — SK Hynix US Listing:
Reuters — SK Hynix targets $29B-plus US listing as AI demand surges
CNBC — SK Hynix plans to raise $29 billion via Nasdaq listing as soon as July 10
AI-RTZ #557 — SK Hynix ready for its AI close-up (December 2024)
Take 3 — Chip Stocks Ebb With Tech Stocks:
Gadget AI — Google’s $100 Home Speaker:
Q1 + Q2 — MP’s favorite + least-favorite features of home AI devices:
(no external sources — MP’s own analyst view)
Companion text:
AI Ramblings Daily on AI-RTZ is here to think through AI and reset. Together.
Today’s AI-RTZ #1127 — Microsoft picks a side vs the frontier AI models — Microsoft is increasingly picking its spots versus the frontier AI models, balancing its OpenAI relationship against a broader multi-model, enterprise-first strategy. Recommended as today’s reading post.
Tomorrow — ARD 105 on AI-RTZ 1128.
Thanks for joining us today, AI Curious Folk. Stay tuned.
(NOTE: The discussions here are for information purposes only, and not meant as investment advice at any time. Thanks for joining us here.)
Subscribe to AI: Reset to Zero for daily AI Ramblings + Sunday Bigger Picture posts.
Yeah, the memory squeeze gets underplayed next to the GPU story. Every wafer that shifts to high-bandwidth memory for AI data centers is one not making consumer DRAM, and it shows up fast: memory is now around 35% of a PC's bill of materials, up from the mid-teens. The AI buildout is landing on the price of every laptop and phone, not just the data center bill.