If vLLM already solved LLM serving, why did SGLang appear?
For most of 2025, the H200 question was about supply. Could you get one at all?
Then Blackwell arrived and the answer looked obvious. The B300 started shipping in Q1 2026. TrendForce put Blackwell at over 70% of NVIDIA’s high-end GPU shipments for the year, with Rubin slipping. The H200 was a generation old. It should have become the cheap, boring option.
It got more expensive instead. The median on-demand H200 price rose about 18% between July 2025 and July 2026, from $3.50 to $4.14 per GPU-hour (getDeploying, 29 Jul 2026). Hopper demand came back faster than HBM3e supply.
So rent or buy is a live question again. It also has an answer. The answer is a utilization number, and you can work out yours in ten minutes.
The short version
| Access route | Price | Commitment | Best for |
| On-demand GPU VM | $4.47 / GPU / hour | Monthly hours minimum | Bursty R&D, fine-tuning, evaluation |
| Committed VM | $2,263 / GPU / month, down to $1,767 at 36 months | 1 to 36 months | Steady production inference |
| Dedicated 8-GPU node | $16,758 / month, down to $13,084 at 36 months | 1 to 36 months | Multi-GPU training, single-tenant work |
| Buy outright | ~$370,000 for an 8-GPU HGX H200 server | 3 to 5 years of capital | Saturated 24×7 workloads, in-house infra team |
An owned 8-GPU H200 node costs about $2.12 per GPU-hour if you keep it busy every hour of every day. Run it below roughly 47% sustained utilization and hourly rental wins.
That is before you count the capital, the wait, or the resale risk. Measured enterprise GPU utilization sits between 30% and 50%. That gap is the whole argument.
1. A generation-old chip that still costs a lot
The H200 is the same GH100 die as the H100 with a bigger, faster memory system. It carries 141 GB of HBM3e at 4.8 TB/s. The H100 carries 80 GB at 3.35 TB/s. Same CUDA cores, same Transformer Engine, same FP8 path, no FP4.
That memory is the whole reason to pick one. It holds a 70B model in FP16 on a single card, with no tensor parallelism. It also leaves room for a long-context KV cache. Push past 70B and you are sharding across cards again.
What changed in 2026 is the H200’s position in the stack. It now sits below the B200 and B300. The honest case for it is availability and cost per token on memory-bound inference, not peak throughput.
What stayed the same is the price. HBM3e supply and CoWoS packaging remain the binding constraints. If you waited for Blackwell to make the H200 cheap, that wait was wasted.
2. What buying an H200 costs
Two numbers get quoted. Only one of them reaches a purchase order.
The card price for an H200 SXM5 runs $32,000 to $40,000 (Mercatus, verified 8 July 2026). A bare SXM module on its own is of no use to you. It needs a baseboard, a chassis, and integration.
The system price is the one that matters. A complete 8-GPU HGX H200 server runs $320,000 to $420,000. The typical integrated price from a major OEM is about $370,000. That works out to roughly $46,000 per GPU once you count the chassis, the NVLink baseboard, the NICs and the integration work.
Then the running costs. An 8-GPU H200 node draws 5.6 kW at the GPUs and needs about 8 kW of billed capacity once cooling overhead is counted. Indian colocation runs $126 to $263 per kW per month. Call it $1,000 to $2,100 a month for space and power. Add spares, operations, and the people who carry the pager.
On a 36-month straight-line write-down:
| Line item | Per month |
| Hardware, $370,000 over 36 months | $10,278 |
| Colocation and power, 8 kW at $190/kW | $1,520 |
| Operations, spares, support | $580 |
| All-in | ~$12,400 |
That is about $2.12 per GPU-hour if the node runs every hour of every month. It excludes the inter-node fabric you need the moment you buy a second node. It also excludes the engineers.
Assumptions, so you can re-run them: $370,000 system price, 36-month life, $190/kW/month colocation, straight-line depreciation, no financing cost. Change any one and the number moves.
3. What renting costs
Start with the global market, for calibration. Across 56 tracked on-demand H200 listings in July 2026, prices run $2.30 to $7.91 per GPU-hour. The median is $4.14 and the average is $4.45 (getDeploying). AWS sits near the top at $7.91. The cheap end is short-notice capacity from providers with no India footprint.
4. Rent or buy is a utilization question
Here is what the $X-per-hour versus $Y-per-unit tables leave out. An owned node costs the same whether you use it or not. A rented one only costs you when it runs.
| The short version: Buying a node for bursty work is like buying a truck to move house twice a year. You will use the truck. You will also pay for it on every day it sits parked. |
An 8-GPU node offers 5,840 GPU-hours a month. Take the owned all-in of $12,400 a month and divide:
| Sustained utilization | Owned, per useful GPU-hour | Neysa on-demand |
| 100% | $2.12 | $4.47 |
| 70% | $3.03 | $4.47 |
| 50% | $4.24 | $4.47 |
| 40% | $5.30 | $4.47 |
| 25% | $8.48 | $4.47 |
The lines cross at about 47%. Below that, hourly rental is cheaper, and the gap widens fast as utilization drops.
So what is real utilization? Measured numbers land lower than most teams assume. Meta’s own Llama 3 405B training run sustained about 38% model FLOPs utilization on a cluster built for that one job. Broader enterprise studies put GPU utilization between 30% and 50%. Cast AI measured production Kubernetes clusters in 2026 and found an average near 5%, driven by defensive over-provisioning and whole-GPU-per-pod scheduling.
Meta got 38% on purpose-built hardware running one workload. A bursty fine-tuning schedule will land lower.
The other half of the finding matters just as much. Once you are reliably above that line, a commitment beats hourly rental. Neysa’s 36-month dedicated node works out to about $2.24 per GPU-hour at full utilization. That sits within a few percent of owning, with none of the capital.
That is the real shape of the decision. Bursty work goes on-demand. Steady work goes on a term. Very little justifies a purchase order.
5. Three costs that never reach the quote
Lead time. Data-center GPU procurement ran 32 to 52 weeks through 2025 and 2026 for H100, H200 and Blackwell server configurations. Add the DGFT authorization and the rack build on top. A 90-day procurement cycle breaks against a 32-week lead time. Every week of that wait is dead time for the team that needs to run the job.
Obsolescence. NVIDIA ships on an annual cadence: Hopper in 2022, Blackwell in 2024, Rubin in 2026. Each generation lands with roughly twice the performance per watt. The older card keeps working perfectly while its cost per token roughly doubles. For frontier AI silicon that means an 18 to 36 month economic life, well short of the 4 to 6 year refresh cycle finance teams expect from HPC hardware. GPU depreciation schedules became a board-level argument in late 2025 for this reason.
Idle. Covered above. It belongs on this list because teams model it last and pay for it first.
Owning is a defensible choice. It is a bet on sustained utilization, and it deserves to be made on purpose rather than by reflex.
6. Where Neysa fits
Neysa Velocis is an AI Acceleration Cloud built and run in India, and a SemiAnalysis ClusterMAX Recommended cloud. There are three ways onto an H200.
On-demand GPU VMs. A 1 x H200 SXM instance with 141 GB, 24 vCPU and 224 GB RAM runs at $4.47 an hour. 2x and 4x configurations are available. Metering is minute-level, so a 61-minute run bills as 61 minutes. A monthly hours minimum applies, so treat this as elastic capacity rather than a card you swipe for.
Committed capacity. The same VM starts at $2,263 a month and tapers to $1,767 on a 36-month term. Predictable pricing with discounts for long-term commitments, which is the right structure once a workload settles.
Dedicated bare metal. An 8 x H200 SXM node carries 1,128 GB of GPU memory, 112 cores and 224 threads, 2,048 GB of RAM, and 8 x 3.8 TB of local NVMe, on a 3,200 Gbps fabric. It runs $16,758 a month, down to $13,084 at 36 months. Single tenant, and the local NVMe is in the price. Storage beyond that, the VPC and bandwidth are metered separately, so ask for the full quote rather than the node rate.
Around the card: managed notebooks with an idle killer that stops billing when a workspace goes quiet. Dedicated single-tenant inference endpoints where you pick the quantization, FP8, INT4 or AWQ. VKE for self-serve Kubernetes, where you own the control plane. WEKA performance storage and Arista networking. Minute-level metering with invoice drill-down by project, so spend is attributable to a team.
Onboarding is assisted rather than self-serve. H200 capacity starts with a conversation, and a Neysa engineer helps you size it and stand it up.
Final verdict: rent, unless you can prove the node will stay busy
Buy an H200 if all four of these are true. You have a saturated 24×7 workload that will not change shape. You employ the infrastructure staff to run it. You already hold a data-center or colocation contract. Your finance team is comfortable putting $370,000 of capital against silicon with an 18 to 36 month economic life.
Rent if any one of those is missing. For most Indian AI teams, all four are.
If you sit in between, split the workload. Put steady production inference on a committed term. Put bursty experimentation on-demand with the idle killer running. That combination beats both extremes, and it is the only one that survives a utilization audit.
See predictable GPU cloud pricing for your workload. Talk to an AI Expert about deploying on Neysa Velocis.
