Training is a spike. Answers are the bill.
The energy cost of AI
Training a large model spends a great deal of electricity once. Every reply after that spends more. Cooling the building is part of the same bill. Efficiency is a design choice.
What it is
Training is the expensive pass that sets the weights. It happens rarely, on a cluster, and it is easy to point at.
Inference is the electricity of each answer, all day, on whatever is serving the model. For a system people actually use, this is the load that does not stop.
Data centers do not only power chips. A large share of the site's electricity is cooling, power conversion, and the building around the racks.
How it works
A bigger model, a longer context, and a hotter sampling pass all cost more per reply. Recomputing work you already did costs again. Reuse, when the situation has not changed, is one way to spend less.
A model that runs only when something happens spends less than one that stays busy while the room is quiet. Sparsity and event-driven hardware are research answers to that waste. They are not free, and they are not the same as a smaller bill on a normal GPU.
Numbers move fast and get misquoted. If a post claims a percentage cut, check whether that figure is in the paper or only in the headline.
Why it matters
If energy is an afterthought, the system grows until the power bill or the heat is the limit. Putting the cost in the design — what to compute, what to reuse, what to drop — is how a system stays runnable.
Local machines and far clusters spend differently. Doing a small job on a small machine can be cheaper than shipping it across a continent. Doing a huge job on a tiny machine can be worse. The honest question is the whole path, not the chip alone.
Sources
- International Energy Agency, Energy and AI (2025), is a public overview of data-centre electricity. Use it for scale. Do not treat a blog's percentage as the IEA's number.
- This page does not claim a measured watt figure for any Violet Echoes system. The City's energy rule is a design stance, not a lab result.
See it in the City
- The Development Divergence
One of the five principles is energy as a first-class constraint. Sustainable cost counts. Peak capability does not win by default.
- Public Services Grid
The Grid's version of the same rule is energy honesty: no hidden losses, no deferred critical work, continuity over applause.