Can they power with solar? Nuclear? Hell, even a natural gas plant? Nope, the data centers need the power right this second, so they get gas turbines on site. Same with cooling; evaporative is just the quickest and cheapest to set up.
Same with its architecture. There’s no time to fix temperature/sampling issues, no time to try bitnet or any of a bazillion interesting papers that came out. A shippable product (model) is needed yesterday; just scale up what we have. “Fail” a single experiment? Your team is fired, which is exactly what happened at Meta.
Everything has to happen right now because of corporate FOMO. So, while this is an interesting musing and maybe Intel or someone will play with it, the actual AI labs could not care less because they can’t get it immediately.
The issue with AI is “now”
Can they power with solar? Nuclear? Hell, even a natural gas plant? Nope, the data centers need the power right this second, so they get gas turbines on site. Same with cooling; evaporative is just the quickest and cheapest to set up.
Same with its architecture. There’s no time to fix temperature/sampling issues, no time to try bitnet or any of a bazillion interesting papers that came out. A shippable product (model) is needed yesterday; just scale up what we have. “Fail” a single experiment? Your team is fired, which is exactly what happened at Meta.
Everything has to happen right now because of corporate FOMO. So, while this is an interesting musing and maybe Intel or someone will play with it, the actual AI labs could not care less because they can’t get it immediately.