The same H100 chip rents for $2.69 an hour on RunPod and $12.29 on Azure
The same H100 costs $2.69 or $12.29 an hour depending where you rent it
A data-receipts analysis normalizes on-demand GPU prices to dollars per GPU-hour, comparing only identical chips. For an H100 SXM 80GB, RunPod Community charges $2.69 an hour while Azure's ND96isr_H100_v5 runs $12.29 — a 4.57× spread, or $1,964 versus $8,972 for a month of continuous use. The gap isn't pure margin: Azure's ND series bundles 400 Gbps InfiniBand with GPU Direct RDMA, a datacenter SLA, and enterprise compliance that single-node fine-tuning never touches.
If you are fine-tuning on one node, you are paying for an interconnect you will never use. If you are training across 64 nodes, the cheap row is not an option.
- plantain
You are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VM's (you can see it openly discussed in their Discords).
- noodlesUK
One thing that I'm not seeing mentioned very frequently is that it's really hard to actually get the GPUs that hyperscalers advertise depending on your region, use of cloud credits etc.
On Azure if you have credits (even hundreds of thousands of USD of credits through programmes like MSFT for startups) you just can't get any GPU quota in any region, even if you beg and plead with support reps. Even if you have cash it's often difficult to get any of the cheaper cards. You can forget about spot instances etc because the regions are essentially all at 100% capacity.
I've heard on other hyperscalers that it's also challenging.
This is super chilling for anything other than GenAI inference/training because if you have some more traditional workload that requires GPUs (traditional ML, other kinds of modelling) you're SOL.
- itkovian_
One interesting thing going is device fraud on the 3rd party market places - i.e changing the clock speed, dividing devices up and renting same one at same time and hoping rentees won’t notice, everything you can think of someone is trying.
lots of interesting dynamics in the preemptible (spot) markets as well. If you’re smart i’d say there is a >10x cost advantage available over hyperscalar OD pricing with basically no downside. All easy with agents now as well, so it will likely soon be competed away (you already see this with spot rates increasing significantly in last 3 months and converging to just below the od prices)
- buredoranna
There are a couple of new indices out there.
[1] "Silicon Index". Built against pricing observations, including listed prices and transaction data.
[2] "Ornn Compute Price Index (OCPI)". Claiming to use only executed transactions.
Your mileage may vary, but I find these new indices to be quite compelling.
[1] https://www.silicondata.com/products/silicon-index
And yup, its not just how much these resources cost, but how much people are willing to pay for them.
(edit: formatting)
- uneekname
> 4.57× between cheapest and dearest, for the same chip
> What the 4.57× is not
> - InfiniBand
> - A datacentre SLA
> - Enterprise compliance, committed capacity...
ok