Support needs for AI workloads

Many enterprises need far more than that in terms of support, however, especially with the rise of AIOps and MLOps. Most organisations looking to benefit from AI and machine learning (ML) need a partner that can supply the required level and cadence of support. “There’s a consultancy and professional services element to consider,” says Boost. “And sovereignty is becoming a bigger and bigger thing. People have been burned. They crave control.” 

In summary, organisations need transparency around how data will be managed, stored and priced. They need to tread carefully when choosing cloud providers.

Neoclouds can raise the same sovereignty questions as traditional clouds. Do you really control your data?

Enrico Signoretti, vice-president of product and partnerships at cloud storage firm Cubbit, adds that many neoclouds are just specialised clouds, operated or using a tech stack that’s largely based overseas. “[This means] they can raise the same sovereignty questions as traditional cloud,” he says. “Do you really control your data?”

For sovereign AI, you need “home-grown champions”. European countries need to scale and fund their own new AI factories. The viable path is architectures that keep data sovereignty next to the GPU through encryption and the right data orchestration and governance. Otherwise, an enterprise’s data, which is its most important asset, remains exposed to risks linked to extraterritorial laws, he says.

Thomas King, chief technology officer of internet exchange DE-CIX, says neocloud providers have competed so far by offering cheap GPUs for AI training. Rapid innovation in AI servers travels hand-in-hand with depreciation, which is estimated to be three to five times faster than for traditional hardware.

“Usually, they are a lot cheaper because they focus on AI workloads only. They are not general-purpose cloud providers,” he says.

The risk to the customer partly depends on the risk of provider lock-in that restricts long-term agility. That said, modern IT infrastructures usually have a lot of virtualisation in place. Moving from one provider to another is a lot easier than it was 10 years ago, says King.

Additionally, moving to AI inference workloads instead of training is likely to prove more profitable. Training can be done cheaply, where land and power are affordable and datacentres are easy to build. But when you’re doing more, you need quality connectivity.

“When it’s about using the AI models, neoclouds supported very closely can provide inference with very low latency,” he says. “In this case, you are usually also in an environment where you’re not only going with one AI provider anyway. You need to find the right mix to serve your customers best.”

In addition, organisations do not usually go out of business overnight, with many neocloud firms publicly traded, which means regular market announcements. Warning signs, such as not keeping up with new GPU versions, mean you could start migrating elsewhere.

“If you do your IT infrastructure right, and build in the risk that your neocloud provider might go out of business, it shouldn’t be too hard to move your infrastructure,” he says.

With the European Union’s proposed Cloud and AI Development Act, which is set to come into effect this year, neocloud providers may be able to offer control of data processing locations and ensure jurisdiction-aware interconnection and data pathways, he adds.

Expansion tipped to continue

Estimates by Synergy Research suggest the doubling of the neocloud sector in the past year could be followed by further expansion at 69% per year through to 2030.  

“AI is a killer application for edge computing,” DE-CIX’s King says. “You have complex AI models. [Applications] need to be close to the user, because working on doing the calculations on the AI model already takes time. You can’t spend a lot of time on the transmission of the data back and forth.”

Traditional hyperscale providers are also moving in a similar direction because a new market is developing, even if not as fast, with the return on investment (ROI) not being realised as quickly as many had hoped. 

“There are a lot of pros, including high margins in the inference space,” says King. “Not everyone will survive. But, in the end, everybody is looking into how we can make use of AI, and we are still in the beginning.”

Suresh Vasudevan, CEO of AI platform provider Clockwork.io, notes that datacentre lifecycles run to 10 or 15 years, while GPU technology depreciates in four to six years. However, long-term contracts with foundation model builders or hyperscalers may reduce any risk.

In many cases, neoclouds can offer lower GPU pricing, more predictable access to high-end capacity in a supply-constrained market, and sometimes bare metal environments where enterprises can bring and tune their own software stack for higher utilisation. When GPU supply is tight, guaranteed access to capacity and cost control can outweigh ecosystem convenience – although integration friction and enterprise readiness requirements cannot be underestimated.

“Ultimately, the choice comes down to workload profile and economics,” adds Vasudevan.

Consider independent benchmarks

Every neocloud will describe itself as enterprise-grade, so look for measurable operating data on the infrastructure reliability, utilisation, power, cooling and the like. Consider independent benchmarks like ClusterMAX from SemiAnalysis for useful comparative transparency, Vasudevan urges. “Enterprises should press for hard numbers,” he says. “What is your measured cluster-level availability? How often do interruptions occur at 1,000-GPU scale? What does your SLA truly guarantee?” 

Enterprises should press for hard numbers. What is your measured cluster-level availability? How often do interruptions occur at 1,000-GPU scale? What does your SLA truly guarantee?

Four or five nines availability is expected in traditional central processing unit (CPU) environments, he points out. However, large GPU clusters can experience multiple disruptive interruptions per day. Failures are part of operating at scale, but must be consistently and efficiently managed. “The second differentiator is diagnostics. When jobs slow down or fail, does the provider offer deep, actionable telemetry to isolate the problem quickly? Without strong observability, GPU hours are lost and ROI erodes,” says Vasudevan.

Hyperscale won’t be going away. For organisations with multi-year cloud commitments and significant data gravity, there are financial and practical incentives to continue building within that environment. “Hyperscalers bring breadth. They offer a deeply integrated ecosystem of microservices – identity, databases, security, networking and observability – that already sits alongside an enterprise’s existing data estate,” adds Vasudevan.  

CBRE’s dataset also recorded notable activity in the Nordics, where there are lower-cost renewable energy options. Power requirements may have more influence on the lease structures than square footage, and CBRE has also noted that neoclouds are attracting more interest where there are fewer hyperscale availability zones.

Kevin Restivo, director and head of datacentre research for Europe at CBRE, says that generally, colocation providers may be offering space to neoclouds under different terms than those offered to hyperscalers. “The deals we see in the market between neoclouds and datacentre providers are typically shorter in length,” he says. “And contract terms change depending upon the amount of capacity contracted.”

Meanwhile, rent prices of late are sometimes well above inflation. So it can be worth paying a premium and having shorter-term deals, the pay-off being greater flexibility and ability to migrate, as well as speed to market. “Neoclouds are trying to build out their infrastructure,” he says. “They need their kit in datacentres, and they need to do it quickly. Capacity is, as I like to say, an increasingly precious commodity in Europe and worldwide.”

Through 2026, the supply bottleneck for compute-intensive workloads looks confirmed, relative to the perceived demand for access to GPUs, he adds. 

Of course, if that demand does not eventuate, there may be a need for fewer providers down the track. For now, neocloud will continue to play a key role in the datacentre landscape, by virtue of the capacity in train – that is, under construction for cloud purposes. “The real question is what enterprises make of AI services,” says Restivo. “Because there is great anticipation about investment in AI services on the part of European enterprises.”

For most enterprises to move forward and begin employing AI at scale, the markets are going to need to see more early adopters succeed, demonstrating benefits and productivity.