If vLLM already solved LLM serving, why did SGLang appear?
Updated on
Published on
By
Table of Content
About the author
India went from having no real high-performance computing option to having a genuine choice in about eighteen months. L&T is building a 10,000-GPU NVIDIA B300 cluster in Chennai for Together AI on an order worth roughly $1.57 billion. Google broke ground on a gigawatt-scale AI hub in Visakhapatnam under a $15 billion commitment. Microsoft committed $17.5 billion. Yotta already runs more than 16,000 H100s, and Neysa is contracted to over 25,000 GPUs by 2027.

India is now one of the largest AI infrastructure build-outs outside the United States. These are different kinds of commitment, so read the note before comparing the bars.
So availability is no longer the constraint. Completion is. The question now is whether a multi-week parallel run survives the fabric, the checkpoint traffic, the scheduler and the node failures that come with it.
On that test the shortlist splits three ways:
Neysa leads this list because it is the only India-headquartered provider in the SemiAnalysis ClusterMAX recommended tiers, and the only one that lets you choose silicon, fabric, storage, scheduler and SLA independently rather than accepting a fixed design. If you need a small cluster this month against a published rate card instead, E2E Networks is the fastest route.
GPUs and pricing at a glance
| Provider | GPUs available in India | India locations | Pricing |
| Neysa | H100, H200, MI300X live. 8,192 MI350X and 9,216 B300 landing | Mumbai, NCR, Bengaluru, Kolkata | On request. On-demand or 12 to 36 month commitment |
| Yotta Shakti Cloud | 16,000+ H100, L40S, sliced GPUs. B200 planned | Yotta-owned data centres | Full public SKU list, INR |
| L&T Vyoma | 10,000 B300 at Chennai, building | Chennai, 250MW Phase 1 | Not published |
| Together AI | 10,000 B300 announced for Chennai | No live India region today | Public list, USD |
| Oracle OCI | Confirm India SKUs | Mumbai, Hyderabad | Public list, USD, per-second on many shapes |
| Microsoft Azure | Confirm India SKUs | Pune, Chennai, Mumbai, plus Hyderabad mid-2026 | Public list, USD, per-second |
| AWS | Confirm India SKUs | Mumbai, Hyderabad | Public list, USD, per-second |
| Google Cloud | H100, L4, T4 in Mumbai. H200, RTX PRO 6000 in Delhi | Mumbai, Delhi NCR | Public list, USD, per-second |
| Tata Communications Vayu | H100, H200, L40S on bare metal | India | On request |
| E2E Networks | B200, H200, H100, A100, L40S | India | Public INR feed. B200 at INR 290/hr on commitment |
| IBM Cloud | Confirm India SKUs | Confirm with IBM | Public list |
Storage is priced separately almost everywhere. Two published reference points: E2E lists its ultra-high-performance parallel file system at Rs 70 per GB per month, and Yotta lists SSD file storage at Rs 8 per GB per month. On a cluster that checkpoints hourly, that line is not a rounding error.

Only three providers publish an India GPU count. E2E Networks, Tata Communications and the global clouds do not, which is not the same as not having one.
How we ranked. Contracted India capacity with a date attached. Non-blocking fabric. A parallel file system that survives checkpoints. Scheduler configuration on delivery. Independent verification rather than vendor claims. India residency and India-hours support. Announced megawatts did not count: a campus that energises in 2028 does not help a run that starts in March.
If you want the reasoning behind those six criteria, it is in the buyer’s guide below the list.
Best for: dedicated single-tenant clusters in India where silicon, fabric, storage and SLA are specified to the workload, with operations included.
At a glance: H100, H200, MI300X live · 8,192 MI350X and 9,216 B300 landing · Mumbai, NCR, Bengaluru, Kolkata · Pricing on request, on-demand or 12 to 36 month · ClusterMAX recommended tier
Neysa is the only India-headquartered provider currently in the recommended tiers of the SemiAnalysis ClusterMAX rating.
Nothing about the cluster is pre-decided. Most providers make the architectural choices for you. Neysa makes them with you.
Operations, which is what decides long runs.
Proof from production. ITQ Technologies: about 141 billion tokens a month, 40% lower cost, P99 latency from 14 seconds to under 2. TIFIN: 65% reduction in GPU cost. Innoviti: 60% lower TCO across 50,000+ merchants.
Best for: the largest deployed NVIDIA fleet in the country, from an operator that owns the buildings as well as the racks.
At a glance: 16,000+ H100, L40S, sliced GPUs · B200 planned · Yotta-owned data centres · Full public SKU price list · TOP500 listed
Best for: campus-scale capacity from 2027, from a builder that has delivered national infrastructure for decades.
At a glance: 10,000 B300 at Chennai, building · Chennai gigawatt campus, 250MW Phase 1 · Pricing not published · Not rated
Best for: a mature training and inference software stack, once its India capacity is live.
At a glance: 10,000 B300 announced for Chennai · No live India region today · Public list pricing, USD · ClusterMAX Silver
Worth confirming: Together’s documentation names no live India region today, and data residency is currently an enterprise option. For a 2026 start, plan on their existing regions and treat India as forthcoming.
Best for: regulated enterprises already standardised on Oracle that want cluster networking without adding a vendor.
At a glance: Confirm India GPU SKUs · Mumbai, Hyderabad · Public list pricing, USD, per-second on many shapes · ClusterMAX Gold
Worth confirming: the Gold rating reflects the global fleet, so confirm contiguous multi-node GPU capacity inside the India regions in writing. Slurm is orchestrated rather than a first-party managed service, so budget configuration time. Oracle is US-headquartered, so CLOUD Act jurisdiction applies alongside India residency.
Best for: enterprises with an existing Microsoft agreement, where procurement speed outweighs unit cost.
At a glance: Confirm India GPU SKUs · Pune, Chennai, Mumbai, plus Hyderabad from mid-2026 · Public list pricing, USD · ClusterMAX Gold
Worth confirming: no bare-metal GPU size appears on the VM sizes page and GPU availability on Dedicated Hosts is limited, so ask how single-tenant isolation is delivered. India GPU quota competes with global demand. Microsoft is US-headquartered, so CLOUD Act jurisdiction applies. Comparison: Microsoft Azure alternatives in India.
Best for: clusters that need to sit inside a much larger managed estate.
At a glance: Confirm India GPU SKUs · Mumbai, Hyderabad · Public list pricing, USD · ClusterMAX Silver
Worth confirming: newest accelerators typically reach India regions later than US regions, so check your SKU is available locally before planning around it. Egress is chargeable, which matters if your training data lives elsewhere. AWS is US-headquartered, so CLOUD Act jurisdiction applies. Comparison: AWS alternatives in India.
Best for: TPU and JAX workloads, and organisations planning for 2028 as much as this year.
At a glance: H100, L4, T4 in Mumbai · H200, RTX PRO 6000 in Delhi NCR · Public list pricing, USD · ClusterMAX Silver
Worth confirming: sole-tenant nodes give physical isolation but there is no bare-metal GPU product, and Slurm is deployed via Cluster Toolkit rather than offered as a managed service. Vizag is a 2026 to 2030 build. Google is US-headquartered, so CLOUD Act jurisdiction applies. Comparison: GCP alternatives in India.
Best for: clusters that move large volumes of data between India sites and an existing estate.
At a glance: H100, H200, L40S on bare metal · India · Pricing on request · ISO/IEC 42001 certified AIMS
Worth confirming: the published fleet is Hopper-generation, and fabric design, parallel file system options and maximum contiguous cluster size are not detailed publicly. Ask for all three before a large multi-week run.
Best for: small and mid-sized clusters where a published rate card and a fast start carry the decision.
At a glance: B200, H200, H100, A100, L40S · India · Public INR price feed, B200 at INR 290/hr on commitment · ClusterMAX tested
Worth knowing: L&T acquired a 21% stake in E2E in late 2024 as a strategic partnership. L&T has since launched its own AI cloud under Vyoma and won the order to host Together AI’s India capacity. E2E operates independently. Useful context when planning a multi-year commitment.
Worth confirming: ask for the fabric oversubscription ratio, and for benchmark access on a short commitment so you can validate collective performance on your own model before scaling.
Statuses come from a first-party documentation audit. Not published means we could not find it in the vendor’s own material. That is not the same as absent.
| Provider | Bare metal | Slurm | Parallel file system | Single-tenant | Live India capacity | Independent signal |
| Neysa | Yes, first-class | Yes, pre-configured | WEKA, VAST, DDN or BYO | Yes, by default | Mumbai, NCR, Bengaluru, Kolkata | Only India-headquartered provider in ClusterMAX recommended tiers |
| Yotta Shakti Cloud | Yes | Yes | Shared file storage published | Yes | Yes | TOP500 listed |
| L&T Vyoma | Yes | Not published | Not published | Yes | Chennai campus building | Not rated |
| Together AI | Yes | Yes, managed | Yes | Yes | Announced, not live | ClusterMAX Silver |
| Oracle OCI | Yes | Orchestrated | Yes, Lustre | Yes | Mumbai, Hyderabad | ClusterMAX Gold |
| Microsoft Azure | Partial | Via CycleCloud | Yes, Managed Lustre | Partial | Pune, Chennai, Mumbai, +1 in 2026 | ClusterMAX Gold |
| AWS | Yes | Yes, ParallelCluster | Yes, FSx for Lustre | Yes | Mumbai, Hyderabad | ClusterMAX Silver |
| Google Cloud | Partial, sole-tenant | Via Cluster Toolkit | Yes, Managed Lustre | Partial | Mumbai, Delhi NCR | ClusterMAX Silver |
| Tata Communications Vayu | Yes | Not published | Not published | Yes | Yes | ISO/IEC 42001 certified AIMS |
| E2E Networks | Yes | Yes, Slurm-native | Yes, published parallel FS | Yes, private clusters | Yes | ClusterMAX tested |
Everything below is the reasoning behind the ranking. Read it if you are running the evaluation rather than skimming the shortlist.
Three scales of GPU work, and why only one of them is HPC
A GPU cloud sells you the first two. An HPC cloud sells you the third. Once you cross that line, four things decide your throughput.
1. The fabric, because leaving the node costs an order of magnitude
Every training step ends the same way. Each node computes gradients on its slice of the batch, then all nodes agree on a single averaged set before the next step starts. That operation is an all-reduce, and it is a collective: every participant must finish before any participant moves on.
A collective runs at the speed of its slowest path.

Figure 2: Inside a Blackwell node, GPUs talk at 1,800 GB/s. The moment traffic leaves the node, the best available link drops to roughly 100 GB/s.
Work it through. A 70-billion-parameter model in bf16 holds 140 GB of gradients. A ring all-reduce moves roughly twice that across each node’s uplink, about 280 GB per node per step.
Frameworks overlap communication with computation, so you never pay the full number. But you cannot overlap bandwidth you do not have. On the wrong fabric the GPUs sit waiting, and your monitoring reports it as low GPU utilisation rather than as a network problem. That is a common misdiagnosis in multi-node training.
Two terms you will meet in a proposal:
Ask for the oversubscription ratio in writing. It is a number, and a provider running a non-blocking fabric will be glad to give it to you.
| NVIDIA Quantum InfiniBand | RoCE v2 over open Ethernet | |
| Congestion handling | Managed in the fabric, mature, predictable at scale | Requires careful DCQCN and PFC tuning to match |
| Ecosystem | NVIDIA reference architectures assume it | Multi-vendor, standard switches, no single supplier |
| Typical speed in India today | 800G | 400G and 800G |
| Best when | You want the validated NVIDIA design | You want vendor choice, or you are running AMD Instinct |
A provider that offers only one of the two has made that decision for you. More in HPC architecture explained.
2. The file system, because checkpoints stall the whole cluster
A parallel file system lets thousands of processes read and write the same namespace at once, striping data across many storage servers. WEKA, VAST, DDN and Lustre are the common implementations.
A 70-billion-parameter model does not checkpoint 140 GB. It checkpoints the full training state: bf16 weights, an fp32 master copy, and two Adam optimizer moments. That is roughly 12 to 14 bytes per parameter, so about 1 TB per checkpoint.
Checkpoint hourly, normal for a run this size, and over seven days you write it 168 times.

Figure 4: The same 168 checkpoints, four different storage tiers. Storage throughput is a GPU cost line, not an IT line.
The GPUs are identical in both cases. The bill is not.
Object storage over a standard network is the wrong tool here. It is excellent for datasets and archives, and it is not built for thousands of ranks writing a coordinated checkpoint. See object vs block vs file storage.
3. The scheduler, because placement follows topology
Slurm is the standard batch scheduler for HPC. It decides which physical nodes your job runs on.
By default it does not know how those nodes are wired. You tell it through a file called topology.conf, which describes which nodes sit under which switch.
Without it, Slurm can scatter 64 ranks across the fabric in a way that pushes every collective through the busiest uplinks in the building. The job runs. It is simply slower than it should be, on every step, forever, and nothing in your logs says why.
Three ways providers hand you a cluster:
For the Slurm option, ask specifically whether topology awareness is configured on delivery or left to you.
4. Failure detection, because long runs fail and short runs do not
GPUs rarely fail cleanly. They throttle under thermal pressure, log correctable ECC errors, or slow just enough to lag their rank. In a synchronous training job, one slow rank sets the pace for every other rank in the cluster.
Then there is outright node loss, and the arithmetic is unforgiving.

Figure 3: Interruption risk compounds with cluster size. A seven-day run on 512 nodes is a coin flip even at a modest per-node failure rate.
A single node might be reliable. Five hundred of them, running for a week, are not. Two contractual numbers matter more than the hourly rate:
The term for what you are protecting is goodput: the fraction of allocated GPU time that produces work you keep. Everything above is a goodput argument. See GPU monitoring and ML observability.
What ClusterMAX is, since it appears throughout this list
SemiAnalysis rents clusters from GPU providers, runs real workloads on them, and publishes what happened. Providers are placed in tiers from Platinum down through Gold, Silver, Bronze, Underperforming and Unavailable. Bronze is the lowest tier SemiAnalysis still recommends.
It is the only independent, hands-on evaluation this market has, which is why it carries weight here.
One structural difference worth naming
AWS, Azure, Google and Oracle all keep data in the India region you select. All four are US-headquartered, so CLOUD Act jurisdiction applies on top of Indian residency.
Residency answers where the bytes sit. Jurisdiction answers whose courts can compel access to them. They are different questions, and for most workloads only the first matters. Where the second matters, as it often does in banking and financial services or insurance, an India-headquartered provider answers something the others structurally cannot.
Four clauses decide how an HPC engagement goes. Each maps to one section above.
A dated delivery schedule with a slip consequence. Cluster size, silicon, fabric, storage, in service by a date. Capacity that arrives two quarters late is capacity you did not buy.
The fabric oversubscription ratio, in writing. The difference between the bandwidth you were quoted and the bandwidth you receive during an all-reduce.
A GPU replacement SLA in hours. Everyone quotes uptime. On a 500-node run, this single number caps your exposure.
Proactive monitoring with a defined escalation path. Ask what share of incidents the provider catches before the customer does, and who you reach at 3am.
Getting a cluster in India is now solvable. Getting a run to finish on it is the purchase that matters.
Deploy, run, and scale on infrastructure designed to keep your AI moving forward.

The workloads driving AI infrastructure today look nothing like what the previous generation of
GPUs was designed for. Reasoning models, agentic pipelines, long-context inference at scale –
these aren’t just faster versions of what came before. The NVIDIA B300 is NVIDIA’s answer to
that shift, and it’s built differently from the ground up.