• 3 Posts
  • 1.04K Comments
Joined 2 years ago
cake
Cake day: August 24th, 2023

help-circle






  • The power usage isn’t even that much on an individual basis once it’s trained, it’s that they have to build these massive data centers to train and serve millions of users.

    I’m not sure it’s much worse than if nvidia had millions of customers using their game streaming service running on 4080s or 4090s for hours on end vs less than an hour of AI compute a day.

    It’d be better if we could all just run these things locally and distribute the power and cooling needs, but the hardware is still to expensive.

    You have apple with their shared GPU memory starting to give people enough graphics memory to load larger more useful models for inference, in a few more generations with better memory bandwidth and improvements, the need for these data centers for consumer inference can hopefully go down. These are low power as well.

    They don’t use CUDA though so aren’t great at training, inference only.














  • Australia isn’t the greatest spot to run a data centre in general in terms of heat, but I do understand the need for sovereign data centres, so this obviously can’t work everywhere.

    What makes you think $3.5 million can’t be profitable? A mid sized hospitals heating bill can get into the many hundreds of thousands or into the millions even. Especially if it’s in a colder environment. A 5-6 year payback on that wouldn’t be terrible and would be worth an upfront investment. Even a 10 year payback isn’t terrible.

    These colder locations are the ideal locations for the data centres in the first place because they generally want a cooler climate to begin with, so they will gravitate to them when possible.

    Edit: And if you build a data centre with this ability to recoup heat, you could start building further commercial things in the area and keep the heat redistribution very close. You don’t need to travel very long distances. You do need to put some thought into where they go through and whats around or will be built around.