AI-acceleration deployments curbed by datacentre design considerations

The latest financial results from PC server manufacturers show sales have been boosted by demand for artificial intelligence (AI)-accelerated servers. There is pent-up demand as manufacturers continue to experience supply chain issues, but this does not appear to be a hindrance for customers as projects to deploy AI-accelerated servers appear to be taking longer than expected.

HPE’s results show a backlog in the production of graphics proessing unit (GPU)-powered servers needed to run for AI training workloads, but it does have servers ready for customers to deploy.

According to the transcript of HPE’s first-quarter 2024 earnings call posted on Seeking Alpha, CEO Antonio Neri said: “We had a couple of deals that slipped from Q1 into future quarters because customers are taking a little bit longer to prepare the datacentre space, getting the power and the cooling ready.”

He described the GPU supply issues as “a tight environment” but said it was “seeing some improvements”.

According to Neri, HPE has a stock of GPU-based servers already built, but “customers will take time to accept those systems”. But as of the end of the first quarter, he said there is a $3bn backlog.

Neri said HPE customers faced a 20-plus week lead time for GPU-equipped servers. Specifically, he said customers were adopting servers equipped with the previous generation of Nvidia GPUs, the H100 architecture. “The majority of the demand today is on H100. Going forward, we’re going to have the Grace Hopper H200 and others.” These, he said, will include AMD’s recently introduced GPU, the Radeon Instinct MI300X.

Dell Technologies has also seen demand for GPU-powered servers continue to outpace supply. In its third-quarter filing, Dell reported it had experienced consecutive growth in traditional servers.

In a statement accompanying the earnings call for the quarter, Dell Technologies vice-chairman Jeffery Clarke said: “AI-optimised server orders increased by nearly 40% sequentially. We shipped $800m of AI-optimised servers, and our backlog nearly doubled sequentially, exiting the fiscal year at $2.9bn. We are also seeing strong interest in orders for AI-optimised servers equipped with the next generation of AI GPUs, including the H200 and the MI300X.”

Like HPE, most of Dell’s customers are still in the early stages of their AI journeys. “We are helping them get started and work through their use cases, data preparation, training and infrastructure requirements,” said Clarke.

Discussing the opportunity for Dell going forward, which reflects how it engages with customers across its product portfolio, consulting and finance businesses, he said: “Progress in this space won’t always be linear, but we are excited about the opportunity ahead.”

Beyond the GPUs, AI puts new demands on enterprise storage. Clarke said organisations that are building AI systems have initially trained their AI models using text-based data. Dell is betting on its customers moving to richer enterprise datasets, which aligns with its storage strategy.

He said the new PowerEdge F710 and 210 storage products offer increased performance around high-concurrency, latency-sensitive workloads. “It’s aligned with what is going to be the growing need as this gets deployed inside the enterprise,” he added.