What Is a NeoCloud Provider? GPU Cloud vs Hyperscalers Explained

What Is a NeoCloud Provider?

A neocloud's primary product, often its only product, is GPU compute. Hyperscalers like AWS and Azure offer hundreds of managed services with GPUs listed somewhere in the catalog. Neoclouds don't bother with the rest. Dense GPU clusters, high-bandwidth interconnects, and NVMe storage tuned for AI: that's it.

McKinsey defines the category as "independent GPU-as-a-service providers" that emerged in response to global scarcity of high-end compute hardware and GPU manufacturers' push to move product outside of direct hyperscaler agreements. McKinsey, Equinix, ABI Research, and Cisco all use the term in their infrastructure research.

Where the term came from

"Neocloud" gained traction in 2023-2024 through SemiAnalysis GPU supply chain reporting and McKinsey's generative AI infrastructure work. Before that, these companies were called "GPU cloud" or "GPU-as-a-Service" providers. The category is older than the label. CoreWeave started in 2017 doing rendering and moved to AI compute around 2020. Lambda Labs was making ML hardware in 2012 and shifted to cloud rental as research demand grew.

What a neocloud does and does not offer

The row that matters most is managed services. When an AI team needs only to run a training job or a batch inference pipeline, hyperscaler complexity and pricing overhead do not help them. Neoclouds remove both. When that same team needs a database, a queue service, and a GPU cluster on the same bill with unified identity and networking, a hyperscaler is the more practical choice.

NeoCloud vs Hyperscaler: What Actually Differs

The difference between a neocloud and a hyperscaler isn't mainly about price. Hyperscalers added NVIDIA hardware to platforms built for web services, databases, and virtual machines. Their networking, storage, and management layers came from that context and got adapted. Neoclouds started from the GPU cluster.

How the architectures diverge

Neoclouds run H100 and A100 GPUs on InfiniBand fabric at 400 Gbps per link. In distributed training, gradient synchronization between GPU nodes moves terabytes of intermediate model state, and that traffic is latency-sensitive. AWS EFA and Azure's selective InfiniBand instances handle many workloads fine but aren't uniformly as fast as the InfiniBand setups neoclouds run by default.

Neoclouds also lean toward bare-metal or thin-VM access. Less hypervisor overhead between the workload and the hardware. For training jobs that run days, that efficiency difference shows up in cost.

The managed services trade-off

"For most AI teams, neocloud providers deliver 40-85% lower GPU compute costs than hyperscalers with comparable or better GPU availability in 2026." (Thunder Compute, June 2026)

AWS has 200+ distinct services. A team already running their application on AWS can add a p5 instance and keep everything in one place: data, networking, identity, monitoring. Moving the training workload to a neocloud means data transfer overhead, a new auth layer, separate monitoring, and a second vendor relationship to manage.

For teams that only need GPU compute and handle their own orchestration anyway, that's workable. For teams deep in a hyperscaler ecosystem, the friction may outweigh the savings.

The Major NeoCloud Providers in 2026

Six companies dominate the neocloud market in 2026. CoreWeave and Lambda Labs operate centralized, curated GPU clusters with enterprise-grade SLAs. RunPod and Vast.ai run marketplace models where independent hardware operators list capacity on a shared platform. Together AI and Crusoe Energy occupy focused niches: inference APIs, and low-cost stranded-energy sourcing respectively.

Pricing from Spheron Network and Thunder Compute, May-June 2026. Marketplace providers show typical ranges; actual prices vary by host.

CoreWeave

CoreWeave is the largest neocloud by managed fleet size and the only one to have signed a verifiable multi-billion-dollar contract with a major AI lab. In 2023, the company announced an $11.9 billion, five-year compute agreement with OpenAI, according to press reporting from that period. CoreWeave began as a rendering service in 2017 and pivoted to AI compute around 2020. By 2025, its fleet spanned hundreds of thousands of GPUs across US data centers.

For the full CoreWeave pricing breakdown, GPU availability, and who the platform serves, see our CoreWeave review.

Lambda Labs

Lambda Labs started as a hardware company selling GPU workstations to machine learning researchers and launched its cloud service in 2019. It operates centralized clusters with dedicated InfiniBand fabric, making it the most consistent neocloud choice for multi-node jobs requiring guaranteed inter-GPU bandwidth. H100 on-demand in Q2 2026 runs $2.49-$3.44 per GPU-hour depending on configuration (Spheron Network, May 2026).

RunPod

RunPod operates a marketplace where independent GPU operators list their hardware under two tiers: Community Cloud (cheaper, lower guaranteed uptime) and Secure Cloud (stricter quality requirements, higher prices). H100 PCIe starts at $1.99/hr on Community Cloud and $2.39/hr on Secure Cloud as of Q2 2026. Networking quality and reliability vary by host rather than being uniformly guaranteed across the platform.

Vast.ai

Vast.ai runs the most aggressive pricing in the market, using a spot-style auction marketplace where hosts bid for workloads. H100 instances average $1.50-$2.58/GPU-hr depending on host and availability window. Preemption risk is real: spot instances can be reclaimed by the host with limited notice. For the full breakdown of Vast.ai's strengths and limitations for production workloads, see our Vast.ai review.

Crusoe Energy

Crusoe Energy differentiates on power source rather than price. The company builds GPU clusters at natural gas flare sites and data centers powered by stranded renewable energy, positioning itself as the sustainability choice for companies with carbon reporting requirements. Its enterprise H100 clusters use custom negotiated pricing.

How NeoCloud Pricing Works

Neocloud pricing divides into three tiers: spot or interruptible instances at the lowest cost with preemption risk, on-demand hourly instances at list price, and reserved contracts at significant discounts for 3-month or 12-month commitments. Hyperscalers use the same three tiers, but their list rates for identical GPUs are materially higher.

The pricing spread across tiers (H100 80GB, Q2 2026)

For a complete listing of GPU cloud providers including non-neocloud options, see our cloud GPU providers comparison.

The Number Most Guides Don't Show

Compare the total compute cost of a single H100 training run across providers. 1,000 GPU-hours is a reasonable proxy for a 7B LLM fine-tuning job running 10 hours across an 8-GPU node, with several iterations.

The CoreWeave on-demand rate sits close to AWS, which surprises users expecting all neoclouds to be dramatically cheaper. CoreWeave competes on managed services and SLAs rather than raw spot pricing. Its reserved contracts for enterprise customers, typically multi-month commitments, bring the effective rate into the $2.50-$3.50 range. That is where the substantial savings appear relative to AWS.

McKinsey's finding that neoclouds are "up to 85% cheaper" applies most directly to the marketplace tier (Vast.ai, RunPod spot) versus Azure on-demand pricing: the two extremes of the market. The median saving for equivalent on-demand GPU hours when comparing leading neoclouds to AWS is more typically 40-60%.

When to Use a NeoCloud Instead of AWS or Azure

It comes down to whether the workload needs GPU compute only, or GPU compute wired into a broader cloud stack.

Use a neocloud when:

• The workload is GPU training, fine-tuning, or large-batch inference and needs no managed cloud services alongside it
• Your team already manages its own storage (S3-compatible or local NVMe), orchestration (Kubernetes or Slurm), and monitoring
• The compute job runs for hours or days and the cost difference is material (at $3,890-$5,380 saved per 1,000 GPU-hours vs AWS, this threshold arrives quickly for teams running regular training jobs)
• You need InfiniBand networking for multi-node distributed training and your hyperscaler cannot provide it consistently at your instance size
• GPU availability is the constraint: neoclouds often have shorter provisioning queues for H100 hardware than hyperscalers during periods of high demand

Stay with a hyperscaler when:

• The application uses GPU compute alongside managed databases, Lambda functions, S3, or other services with data staying in one VPC
• Your organization has enterprise agreements with AWS, Azure, or GCP that include committed use discounts making the effective GPU rate competitive
• Compliance requirements (SOC 2 Type II, HIPAA, FedRAMP, ISO 27001) rule out providers that have not completed those certifications
• You need 30+ global regions for latency or data residency requirements
• The team lacks capacity to manage a separate cloud vendor relationship alongside an existing hyperscaler stack

What most enterprise teams actually do

"Neoclouds complement the big clouds by filling a gap: ultra-high-performance computing for AI, rather than replacing them." (NEXTDC, 2025)

Most organizations with mature cloud setups keep their application infrastructure, databases, and user-facing services on a hyperscaler and send GPU-intensive training jobs to a neocloud. It requires planning for cross-cloud data movement, but on multi-week training runs the compute savings are usually large enough to cover that overhead with room to spare.

For context on when training a model makes sense versus calling an inference API, the AI training vs inference explained guide covers the cost and performance trade-offs in full.

The Technical Stack Behind a NeoCloud

Neoclouds build their performance advantage into three infrastructure layers: the networking fabric, the storage system, and the GPU density of each rack. All three are configured from the ground up for distributed AI workloads rather than adapted from a general-purpose design.

InfiniBand vs Ethernet

InfiniBand HDR delivers 400 Gbps per port with single-digit microsecond latency. This is what NVIDIA NVLink multi-GPU clusters use internally and what multi-node training clusters require for gradient synchronization. AWS and Azure offer InfiniBand selectively through specific instance families (AWS p5, Azure ND H100 v5), but not uniformly across all GPU instance types. CoreWeave and Lambda Labs run InfiniBand as the default fabric for H100 deployments.

For a full breakdown of NVIDIA H100 bandwidth specifications and NVLink architecture, see our H100 GPU specs and pricing guide.

NVMe storage for training throughput

Training large models requires reading and writing checkpoints and dataset batches at high throughput. NVMe-over-Fabrics storage delivers sequential read speeds of 5-7 GB/s per drive versus 0.5-1.5 GB/s for the SATA SSDs common in general-purpose cloud VMs. The difference is invisible on light workloads but becomes the throughput ceiling on data-intensive training pipelines.

GPU rack density

Standard enterprise server racks handle 10-20 kW of power draw. A rack of eight NVIDIA H100 SXM5 GPUs draws approximately 56 kW at full load (eight GPUs at 700W TDP each). Neoclouds design and build their facilities to handle this density from day one. Power, cooling, and physical rack structure are all specified for it. Hyperscalers host these racks as well, but dense GPU configurations are not the default infrastructure specification the way they are at a neocloud.

The NeoCloud Market in 2026

The GPU cloud market is growing at 30-40% annually through the 2020s, per IDC and Gartner projections from 2025. The neocloud segment hit an estimated $4-7 billion in 2025. At current growth rates, $10 billion by 2028 is achievable.

What drove the neocloud wave

When H100s became scarce in 2023-2024, AI labs took GPU capacity wherever they could find it. Neoclouds had supplier relationships with NVIDIA before the boom and could provision hardware that hyperscalers were quoting months out. The cost gap did the rest. A company paying $200,000 per month on AWS GPU instances could run the same workload for $80,000-$120,000 on Lambda Labs or RunPod. For a startup, that's not a rounding error; it's runway.

Enterprise teams started treating GPU purchasing as a separate line item, negotiating neocloud capacity outside their existing hyperscaler deals once training budgets crossed the threshold where managing two vendors was worth it.

Where this is heading

"The long-term viability of neoclouds depends on moving up the stack into AI-native services, putting them in more direct competition with hyperscalers." (McKinsey, 2025)

Renting bare-metal GPUs by the hour is a thin business. Hyperscalers are locking in better NVIDIA contracts, and as supply normalizes, the on-demand price advantage will compress. McKinsey's 2025 analysis is clear on where neoclouds need to go: AI-native managed services, training pipelines, inference endpoints, model serving, and MLOps tooling carry better margins than raw compute.

CoreWeave's $11.9 billion OpenAI contract points the direction. That deal isn't on-demand GPU hours. It's a long-term managed infrastructure relationship with pricing and capacity locked in. Lambda Labs has been building managed ML tooling alongside its GPU rental business for the same reason. The neoclouds with real staying power in 2026 and beyond are the ones adding services on top of hardware, not just renting racks.

Frequently Asked Questions

Related Articles