Nvidia sells plenty of GPUs for actual money, they are good for it.
No, the real issue is the depreciation for the people owning GPUs. Your GPU will be usable for 4-6 years, and 2-4 of those years will be spent as ”the cheap old GPU. After that time, you need new GPUs. (And as the models are larger by then, you need moahr GPU)
How the actual fuck do these people expect to get any ROI on that scale with those timeframes? With training, maybe the trained model can be an asset (lol), but for inference there are basically no residual benefits.
I feel like what sounds personally insane to us (and is, don’t get the wrong idea), to the people making such decisions the situation is more like -
“Emerging market with unknown upside thanks to new and evolving capabilities, exploration and competitive advantage shaped and constrained, globally, by hardware capability. Not my money I’m betting, ‘risk’ is extreme opportunity for me, negative consequences borne by others. Let’s go”
Calculate the total power cost of running it at 100% load since 2014
Calculate Flops/Watt and compare with modern hardware
Calculate MTTF when running at 100% load. Remember that commercial support agreements are 4-5 years for a GPU, and if it dies after that, it stays dead.
In AI, consider the full failure domain (1 broken GPU = 7+ GPUs out of commission) for the above calculation.
You’ll probably end up with 4-6 years as the usable lifetime of your billion dollar investment. This entire industry is insane. (GTX 1080 here. Was considering an upgrade until the RAM prices hit.)
Nvidia sells plenty of GPUs for actual money, they are good for it.
No, the real issue is the depreciation for the people owning GPUs. Your GPU will be usable for 4-6 years, and 2-4 of those years will be spent as ”the cheap old GPU. After that time, you need new GPUs. (And as the models are larger by then, you need moahr GPU)
How the actual fuck do these people expect to get any ROI on that scale with those timeframes? With training, maybe the trained model can be an asset (lol), but for inference there are basically no residual benefits.
enron sold plenty of gas and real things too: it’s the double handling that’s the problem; not the nature of the goods or services
The relative size of the double handling is the potential problem. I think Nvidia is just trying to extend the gold rush for a bit longer.
I feel like what sounds personally insane to us (and is, don’t get the wrong idea), to the people making such decisions the situation is more like -
“Emerging market with unknown upside thanks to new and evolving capabilities, exploration and competitive advantage shaped and constrained, globally, by hardware capability. Not my money I’m betting, ‘risk’ is extreme opportunity for me, negative consequences borne by others. Let’s go”
I’m still rocking a GTX970 from 2014
Do this:
You’ll probably end up with 4-6 years as the usable lifetime of your billion dollar investment. This entire industry is insane. (GTX 1080 here. Was considering an upgrade until the RAM prices hit.)