Nvidia’s lightning speed in driving change

CEO Jensen Huang has ambitious plans for ‘full-stack computing’.

In the years it has taken Nvidia’s graphical processing units, first developed for video gaming, to find their way to the centre of the computing world, chief executive Jensen Huang has been nothing if not consistent. His long-running prediction that the need for new forms of “accelerated computing” would demand a new chip architecture has come to spectacular fruition with the advent of generative AI.

This week, as the tech world’s attention swung fully on to Nvidia for its annual technology conference, Huang had a new mantra: “Full-stack computing”. The world may know Nvidia as a chip company, but its CEO’s eyes have been set on the full computing “stack” required for AI — not just processors, but complete hardware systems and the software needed to optimise their performance and make them useful.

As the battle lines around AI chips harden, with rivals like AMD and customers like Microsoft and Amazon developing their own specialised processors, this week’s event was a reminder that the focus of competition has already moved well beyond the chips.

Nvidia’s “full-stack” approach begins with the fact that most of what it sells are not individual GPUs but complete systems built around its chips, which are designed to optimise the performance of its components. As important as the processors are the interconnects between its chips, shuttling data around faster, and the central processing units needed to manage the process.

Huang’s technology “stack” also extends to software, in the form of the programming models and libraries of code to make it easier for developers to tap into the power of its chips. Known as Cuda, this has long been recognised as an important competitive moat for Nvidia — though competition is finally emerging in the form of rival software ecosystems.

For most developers, already steeped in Nvidia’s technology, the costs of switching don’t yet match the benefits. And the more tools that Nvidia and its partners create to take its technology deeper into individual industries — something currently happening at lightning speed — the more invested they will be in its underlying chip architecture.

Nvidia has also been moving fast to extend its software beyond Cuda. Notable among the latest crop of announcements is Nvidia’s plan to offer pre-trained AI models, which it calls NIMs. Acting as chatbots, or “co-pilots”, these industry-specific smart assistants are designed to be integrated into IT processes — much like an enterprise version of the AI-infused consumer apps, called GPTs, that OpenAI is developing.

This is the latest example of Nvidia moving “up the stack”, reaching for higher-value services that both make its technology more useful and expand the potential market it can reach. It also strengthens the company’s connection with the AI developers. It is a truism of the tech world that the focus of development constantly moves to a higher level of abstraction, as developers look for tools that make it easier to draw on the burgeoning computing power available to them.

Other leading AI companies are pursuing a similar goal. At its first developer day last year, OpenAI laid out its own case for why developers should look to build directly on top of its large language models.

All of this has had Wall Street analysts salivating about which Big Tech markets Nvidia will turn to next. There has been plenty of talk about whether it will carve out software as a new line of business, potentially adding a new engine to its growth. Huang also faced questions this week about whether Nvidia would take on companies like Amazon and Microsoft by becoming a cloud service provider, running its own network of datacentres to host its AI services.

So far, Huang has avoided responding directly, suggesting he is focused on the current wave of technology and not yet ready to lay out his plans for the next. Deploying Nvidia’s technology through the computing clouds of companies like Amazon is a faster way of getting to market, rather than building its own. And Nvidia is looking at software as another way to boost its chip sales, not as a separate revenue source.

If Huang is right, some of the uses of AI it showed off this week — things like manufacturing simulations and robotics — will become giant markets in their own right. But for now, focusing the entire computing stack on satisfying the red-hot demand for AI chips makes sense. For would-be chip rivals, this week’s event was a daunting reminder of the speed at which Nvidia is moving and the sheer range of complementary technologies it is building.