The AI revolution is pushing the boundaries of chip networking, creating an urgent demand for speed and innovation. In the heart of Silicon Valley, it's not just about who you know on LinkedIn, but how fast your chips can connect and communicate.
Tech giants are pouring billions into AI data centers, and chip manufacturers are responding with cutting-edge networking solutions. From the interconnects between transistors on a chip to the external connections between server racks, networking is integral to the world of semiconductors.
Chip leaders like Nvidia, Broadcom, and Marvell have long established their dominance in this space. However, the AI boom has sparked a search for new networking approaches to handle the massive data flow in data centers. This is where innovative startups like Lightmatter, Celestial AI, and PsiQuantum step in, utilizing optical technology to accelerate high-speed computing.
Optical technology, or photonics, has been a game-changer. Once considered niche and uneconomical, it has found its moment in the sun with the AI boom. According to PsiQuantum's cofounder and chief scientific officer, Pete Shadbolt, the technology has gone from being "lame, expensive, and marginally useful" to a key player in the AI era.
Venture capitalists and institutional investors are betting big on these startups, hoping to ride the wave of chip innovation or find the next acquisition target. They believe that traditional electron-based interconnect technology is no match for the growing demand for high-bandwidth AI workloads.
"Networking was once seen as boring, just switching packets of bits," says Ben Bajarin, a tech analyst and CEO of Creative Strategies. "But with AI, it's about moving robust workloads, and that's where the innovation in speed comes in."
The Big Chip Players
Nvidia gets credit for its foresight in networking. Its acquisitions of Mellanox Technologies and Cumulus Networks were turning points, allowing them to cluster GPUs and create powerful data centers.
Broadcom, meanwhile, has become a key player in custom chip accelerators and high-speed networking. With a market cap of $1.7 trillion, they work closely with tech giants like Google, Meta, and OpenAI on data center chips. They're also at the forefront of silicon photonics, with their new networking chip, Thor Ultra, designed to link AI systems to data centers.
ARM, a semiconductor design giant, announced plans to acquire the networking company DreamBig for $265 million. DreamBig specializes in AI chiplets, small circuits designed to be packaged together, and has an "interesting intellectual property" for scale-up and scale-out networking, according to ARM CEO Rene Haas.
The Light of Innovation
Lightmatter CEO Nick Harris highlights the rapid growth of AI's computing power, doubling every three months, far outpacing Moore's Law. Computer chips are getting bigger, and linking them together becomes crucial for performance.
Lightmatter's approach is unique, using silicon photonics to link chips. They claim to have the world's fastest photonic engine for AI chips, a 3D stack of silicon connected by light-based technology. With investors like GV and T. Rowe Price, they've raised over $500 million in the last two years, reaching a valuation of $4.4 billion.
"The future of computing is about light," Harris says. "Electronics and software are critical, but at this level, you need new ideas, and light is a big part of the computer frontier."
Other startups like Celestial AI and PsiQuantum are also making waves with their optical interconnect technology. Celestial AI raised $250 million from investors like Fidelity Management and BlackRock, while PsiQuantum, valued at $7 billion, raised $1 billion from BlackRock and others.
The Challenge of Optical Networking
While optical networking offers exciting possibilities, it's not without challenges. It's expensive to build, requires specialized equipment, and must integrate with existing electrical systems.
Companies like Broadcom and Marvell, with their expertise and resources, can cater to the specific needs of hyperscalers in both AI chips and networking. They know how to scale, which is crucial in an industry moving towards customization.
"Networking is essential for computers to function," says Bajarin, "but the industry is moving towards customization, which might be a challenge for smaller players."
Despite these challenges, the demand for faster data speeds and better networking tech is only increasing. While the payoff for experimental startups might be years away, the future of computing is bright with possibilities.
"We believe in a photonics future," Bajarin concludes, "but it's still a journey in the making."
This article is part of the Model Behavior newsletter, exploring the latest in AI and its impact on our world.
