Nvidia is building an IP licensing empire on the back of NVLink

The proliferation of custom AI ASICs, or XPUs, from OpenAI, Meta, Microsoft, and others has led many to question Nvidia's grip on the market. After all, if everybody's building their own, who needs the GPUs the AI arms dealer has made its fortune on?

But Nvidia isn't concerned in the least and has instead invited its competition to raid – or, rather, license – the GPU giant's IP holdings, particularly those related to networking.

On Monday, MediaTek became the latest to embrace Nvidia's NVLink Fusion high-speed interconnect technology for its fledgling datacenter XPU offering. In exchange, Nvidia has invested $3.5 billion in convertible bonds issued by MediaTek. The GPU giant will also continue to license the Taiwanese SoC provider's own designs for use in future DGX and RTX Spark systems.

If you're curious, we explored MediaTek's datacenter ambitions last month. But the designer, best known for its Arm-based smartphone and tablet processors, is only one of several high-profile chip designers that plan to integrate the tech. Amazon, Fujitsu, Qualcomm, Arm, and Marvell are all on the same list.

NVLink Fusion started as a commercialized version of Nvidia's high-speed inter-GPU interconnect, introduced early last year. Initially, it was offered in two varieties: a chip-to-chip (C2C) variant for connecting CPUs to GPUs (or XPUs) in a memory coherent fabric, and a switched fabric used to stitch together multiple accelerators into one big logical rack-scale chip. It's since expanded to become the blanket offering by which Nvidia licenses its semiconductor IP.

Given how long it's taken Broadcom and others to develop competitive alternatives to NVLink, it's not surprising companies like MediaTek would license the tech, rather than reinventing the wheel, or piping alternative interconnects, like UALink, over regular old Ethernet.

By licensing Nvidia's NVLink Fusion tech, companies can focus on building competitive accelerators without worrying about how they're going to scale in production.

And because Nvidia's MGX rack designs are part of the Open Compute Project, MediaTek and its partners not only benefit from Nvidia's scale-up networking tech, but also can essentially take existing NVL72 racks and slot in their compute blades. This should dramatically reduce the system design and mechanical engineering experience required to go from silicon to AI racks.

And this isn't theoretical. Amazon is doing just that with its Trainium series of AI accelerators. As we wrote at the time, Amazon used Nvidia's MGX NVL72 reference design for its Trainium3-based rack systems launched last year. Meanwhile Trainium4, expected late this year, will ditch Amazon's in-house NeuronLink interconnect tech for NVLink Fusion.

Nvidia's cut

So, what does Nvidia get from letting rival chip designers piggyback off its hard-won networking tech?

A lot more than licensing fees: It's also a way to keep companies hooked on its other products.

When NVLink Fusion was first announced, customers had two options. They could pair their XPUs with Nvidia's CPUs or their CPUs with Nvidia's GPUs. Customers also were on the hook for NVSwitches if they wanted to take advantage of the tech's scale up networking capabilities.

So, on top of licensing the tech, Nvidia had the potential to sell roughly two GPUs per partner CPU, a Grace or Vera CPU for every two partner XPUs, and up to nine NVSwitches for every 72 or so accelerators, and that's just the rack.

If you're already buying NVLink, why not use InfiniBand or Nvidia's Spectrum-X Ethernet kit to switch those racks together to train and serve even larger models at scale?

Nvidia has since softened its stance and now seems content with the combination of NVSwitch sales and NVLink Fusion licensing. If it hadn’t, Amazon couldn’t have paired Graviton with Trainium.

"The beauty about this platform is it provides the flexibility and the fungibility in order to meet the customers where they are. And so, leveraging the NVLink Fusion platform, they can adopt anywhere from the CPU to the GPU to the scale-up fabric to the scale-out networking across our Spectrum-X and InfiniBand, as well as our DPU within the scale-in fabric to do the north-south storage management as well as security isolation," Dion Harris, Nvidia's senior director of HPC and AI Hyperscale Infrastructure, told press on Monday. "So, the five different networking technologies and topologies that we offer in our technology stack are all available to customers via NVLink Fusion."

Nvidia isn't quite ready to talk about the licensing structure just yet, but Harris says they won’t be giving it away.

"…It is safe to assume that there will be some licensing elements in place in that Nvidia NVLink Fusion is an Nvidia technology," Harris said.

It's not just about networking

Nvidia is already the largest datacenter Ethernet networking provider in the world, thanks in no small part to the AI boom. Its quarterly networking revenue now exceeds $15 billion, and NVLink Fusion licenses and associated NVSwitch sales will only bolster that. However, Nvidia's licensing ambitions don't stop there.

Last week, Nvidia detailed its latest IP offering in the form of NVHBM, essentially a custom base with an integrated memory controller.

According to Nvidia, the approach frees up 25 percent of the die for compute while cutting HBM power draw by 15 percent and boosting effective bandwidth by 30 percent.

Amazon's Annapurna Labs will be among the first to field the tech with its Trainium family of AI ASICs.

But while Amazon's adoption of NVHBM is a clear win for Nvidia, it should be noted that the idea isn't unique to Nvidia. As we understand it, both Broadcom and Marvell are developing similar base-die tech as well. But the tech is another example of how Nvidia is leveraging its engineering might to extract revenues from everything associated with AI, including custom ASICs from others.

A deal with the devil or mutually beneficial coexistence

While Nvidia's fledgling IP offerings are rather narrow for the moment, it's not hard to imagine Nvidia competing with rivals like Broadcom or partners like Marvell and MediaTek on XPU design before long.

Depending on how things play out, Nvidia, Broadcom, Marvell, and MediaTek could end up looking more like frenemies.

XPUs are quickly becoming big business for both Broadcom and Marvell, but the networking required to scale that compute and system design required to bring it to market remains a competitive advantage for Nvidia.

Sure, both have competent Ethernet switching appliances capable of going toe-to-toe with NVSwitch, but even then you still have to integrate that into a rack. The advantage of being able to slot a custom compute blade into a generic NVL72 system and for it to work out of the box is hard to ignore.

If NVLink Fusion helps the XPU designers sell more chips, it could end up being a net benefit even if it means losing out on switch sales. Nvidia is clearly aware that while the barrier to entry for custom silicon has fallen through the floor, scaling that compute remains the hard part. ®