Root Causes 433: Will AI Eat All the Electricity?

Hosted by

Tim Callan

Chief Compliance Officer

Jason Soroko

Fellow

Original broadcast date

October 17, 2024

News reports claim Chinese researchers broke AES with a quantum annealing computer. We clarify the details and talk about the implications of this reported discovery.

Podcast Transcript

Tim CallanSo there was a recent event, I forget what it was, some kind of conference where Elon Musk made headlines with his very - what I want to say? Firm pronouncement, as he likes to do - that AI is going to run into a problem because it's going to run out of suitable components. I think he was talking about transformers in particular - in order to be able to build the AI systems that we're trying to build. And I think perhaps also related to that, there's a concern about the amount of electricity that it's going to use. So here's probably the question for us to discuss today. Is AI going to be badly hamstrung by an inability to literally, physically build the computers we need because we don't have core components?

Jason SorokoSo there's a lot that you just said, and let's tease them out. There are limitations everywhere. But let's call out what the limitations are. Therefore nobody can get enough Nvidia chips because it's now becoming very clear that Apple - Microsoft just made an announcement as well - being able to do GPT, not in the cloud, but right on your hardware. So in other words, AI optimized hardware is gonna become a real thing soon. Therefore, hey, big demand on AI capable hardware. Optimized hardware.
All right. But let's talk about AI in the cloud. Let's start good old-fashioned ChatGPT, which, as you know, Tim, shocking amount of adoption when it first came out. Well, here's what else is shocking behind it. Data centers take up a lot of electricity. That's not a surprise. You know that. I know that. The amount of compute that goes on in the public clouds equals, let's just call it a lot. But it's not something we were like, oh my god, the lights went out because Azure kicked on today. You don't hear news items like that. However, if you're talking about, what does each ChatGPT request take in electricity? It's higher than you think. And it's because a lot of very, very smart hardware behind the scenes is chewing up electricity and, putting its circuits through the paces in order to give you an answer, not just fast, but more creatively through time.
So there's a lot of electricity used for any form of AI. And so let's break down what Elon Musk talked about again. So, yes, he did talk about the lack of components. That's a limitation. But that’s a limitation that just make more chips, make more factories, make more whatever. Is it finite? Yes. Will it bottleneck? Yes. But if there's a demand for more chips, I guarantee they'll go off and make some more chips. It just takes time. He was making a joke, a pun, about the fact that we need to build more transformers for the transformers. And so his joke was literal electrical substations need to be built. In other words, more electrical infrastructure, electric generation infrastructure needs to be built in order to power all of the ChatGPT AI transformers. Which is a terminology within the guts of how AI works. So he was doing a clever pun for nerds.
So the heart of it is, and I think this is the real question that you and I are going to talk about here, is, is there a finite amount of just electricity? Let's say you and I wanted to turn the innovation dial. Let's say the innovation dial right now is at one. Let's say you and I wanted to turn it to 11. How much electricity would that take? And this is very, very oversimplified and just mental model kind of thinking. Could you turn it to 11? Or would we run out of electricity? Never mind, components. Electricity.

Tim CallanThat's an interesting problem. This actually strikes me as being at least rhyming with the problem around cryptocurrency.

Jason SorokoSo if you have proof of work based cryptocurrencies, Bitcoin being an example of one that, will it ever move to a proof of stake and away from proof of work? I think that there's a lot of people heavily invested in Bitcoin mining based off of burning cycles of computers ultimately chewing up a lot of electricity.
And if you've heard even Elon Musk himself and others, they've talked about how, yes, if you look at their projections of Bitcoin usage potentially ending up requiring not just, 1% of the world's electricity, but like 10% or 20% or 30%, or 90%

Tim CallanWhich is shocking.

Jason SorokoThis what we're talking about now with AI. And I love the analogy you just brought up, because in reality, it's the exact same problem. You just have a lot of silicon burning a lot of electricity in order to do work.
In the case of cryptocurrency, it's basically to be able to have that Bitcoin mining process, which is what allows things to occur securely. But in the case of AI, it's, again, just work where silicon is burning electricity. So the reason I use that analogy of turning the dial to 11 is, would it be acceptable, Tim, if turning the innovation dial to 11 on AI, if it required 90% of electricity? And would you be willing to, like, not have light for 90% of your day? I don't think anybody would accept that.
Therefore, I'm just saying that there's a certain level of speed and innovation within AI that I think electricity could be some kind of a gating factor. Let me throw you the arguments against that.
There will probably be efficiencies made in newer and newer hardware. There will be advancements in the way AI is done that will require less electricity. This is where the proof of work analogy breaks down, because Bitcoin assumes there cannot be an innovation to use less electricity. It's a proof of work. AI is not a proof of work.

Tim CallanAt all.

Jason SorokoAt all. It can tolerate, and, in fact, will benefit from efficiencies. Therefore everybody's motivated to have efficiencies. Nvidia will make newer hardware that takes less electricity. And I'll tell you where the next innovation could come from. What happens when quantum computers enter the AI game and efficiencies in decision making and thinking neural networks? What happens when quantum computers, which could potentially be incredibly efficient in making simultaneous thoughts.
A quantum field being able to think in many, many things at the exact same time, whereas a traditional computer needs to brute force itself through all kinds of decision making, chewing up a lot of electricity. Could quantum render this conversation kind of a moot point. It's just a thought.

Tim CallanI don’t know what we do about it.

Jason SorokoWell, what we do about it is we watch in that we are in a point in time and the amount of electricity being chewed up by AI is growing and growing.
Llike I think there are people who have done formal measurements on this, and it's shocking the amount of terawatts of electricity or whatever the heck it is. I think right now, at this very moment, I think some of the numbers that were quoted like Estonia plus, a few other countries’ worth of electricity, just traditional consumption is the equivalent of what AI is taking up right now.

Tim CallanIn its early days.

Jason SorokoAnd within a couple years, it'll be the equivalent of Japan's usage of electricity. So it will be significant. In other words, I think within a short period of time, we're going to go from the onesie-twosie percentages to 10% and then once I think we start hitting 20% things are going to be like, oh boy.

Tim CallanBut to your point, you're right. There's probably a lot of efficiency in front of us.

Jason SorokoThat's the argument against this whole thing, which is, guys, guys relax. Well, this is not a proof of work cryptocurrency story. This is a we're all motivated.

Tim CallanWe’re gonna be well motivated to make it more efficient. There's gonna be a lot of focus on that.

Jason SorokoThere it is. There it is.

Tim CallanAll right. Well, there you go.

Jason SorokoThanks, Tim.

Tim CallanThank you, Jay. This has been Root Causes.

Stay informed with expert insights

Subscribe to Root Causes for engaging discussions on PKI, digital security, and best practices for protecting your organization's critical assets. Don’t miss an episode!