Measuring efficiency: tokens and PUE

When we say 'the cloud' what we mean is 'the data centre'. Globally, data centres are projected to consume over 1000 terawatt hours in 2026. What does that mean for energy production, distribution, and consumption? Guest Phil Harris, Cerio President and CEO, joins thinkenergy to shed light on something we all rely on but may not fully understand. From efficiency to sustainability, environmental concerns to Cerio's role improving how data centres manage energy. Listen in for the future of cloud computing.

-

Related links

● Cerio: https://www.cerio.ai/

● Phil Harris on LinkedIn: https://www.linkedin.com/in/paharris/

● Trevor Freeman on LinkedIn: https://www.linkedin.com/in/trevor-freeman-p-eng-8b612114

● Hydro Ottawa: https://hydroottawa.com/en

To subscribe using Apple Podcasts: https://podcasts.apple.com/us/podcast/thinkenergy/id1465129405

To subscribe using Spotify: https://open.spotify.com/show/7wFz7rdR8Gq3f2WOafjxpl

To subscribe on Libsyn: http://thinkenergy.libsyn.com/

---

Subscribe so you don't miss a video: https://www.youtube.com/user/hydroottawalimited

Follow along on Instagram: https://www.instagram.com/hydroottawa

Stay in the know on Facebook: https://www.facebook.com/HydroOttawa

Keep up with the posts on X: https://twitter.com/thinkenergypod

---

Transcript:

Trevor Freeman 00:07

Welcome to think energy, a podcast that dives into the fast, changing world of energy through conversations with industry leaders, innovators and people on the front lines of the energy transition. Join me, Trevor Freeman, as I explore the traditional, unconventional and up and coming facets of the energy industry. If you have any thoughts, feedback or ideas for topics we should cover, please reach out to us at thinkenergy@hydroottawa.com. Hi everyone, and welcome back. Data centres have come up a number of times on this show, and for very good reason, they have become a key underpinning technology for so much of our lives, every time we pull out that phone from our pockets to pull up directions or buy something online or doom, scroll on your social media or new site of choice, every time you use your phone stream a movie, leverage an AI model, whatever you end up using it for, it's funny as I read this list, I'm sure there's like some university student out there who's thinking, man, what is this old man talking about? We don't use our phones for that, whatever the kids are doing these days, whatever we're doing these days with our phones, with our computers, our tablets, et cetera. All of that leverages infrastructure that most of us have never seen and, quite frankly, probably don't really understand we talk about the cloud like it's this amorphous, nebulous thing, but in reality, we're talking about real hardware in a real building that uses real energy, mainly electricity, a lot of water. And this isn't really new, like we've been leveraging centralized data centres for many years now, but what is changing is the scale of the data centres that we're seeing now, and the pace of growth in computing power that we need to do, the things that we want to do, and that our data centres are able to deliver. So just to throw a few numbers at it, the traditional data centre servers that maybe power the early days of on demand online streaming services, for example, they used anywhere from five to 15 kilowatts per rack. But modern server racks that are used to power AI searches, for example, can hit anywhere from 60 to 100 kilowatts per rack. This is great from a power output per rack perspective, but it means massive energy needs, and that is showing up in the size of load requests that we're seeing from new data centres. New data centres today are asking for service connections that are orders of magnitude higher than those built even just five years ago, globally, data centres are projected to consume over 1000 terawatts in 2026 or terawatt hours, sorry, in 2026 and just a quick kind of refresher from high school or wherever you would have learned this, a terawatt is 1000 gigawatts, which is 1000 megawatts. So 1000 terawatt hours, which is roughly equivalent to the annual electricity demand from the country of Japan, an entire country. So given all of this, there are a lot of incentives to find ways to maximize efficiency and reduce some of that energy demand, and that's where my next guest, Phil Harris and his company Cerio come into play. I'll let Phil get into the details of exactly what Cerio does, but essentially, their goal is to reimagine the data centre to maximize sustainability and reduce energy needs. Phil is Cerio's President and CEO, and has been in the networking and data centre industry for over 35 years, including at well known companies like Intel and Cisco. And I'm really excited about this conversation. One to understand, how do we make data centres a little bit more efficient, or maybe a lot more efficient, but also just to really understand, like, what are we talking about when we talk about a data centre? What is actually happening, what is physically inside these buildings, and we'll get into a little bit of that in our conversation. So Phil, welcome to the show.

Phil Harris 04:13

Well, thanks, Trevor. I appreciate it.

Trevor Freeman 04:13

So Phil, obviously we're here today to talk about your work building sustainable data centres, or trying to make data centres a little bit more sustainable. But before we get into that. You know, you've spent your career, you know, decades of your career at different tech giants. Let's call them in telecisco to to mention, you've seen quite a bit of change. No doubt, over your time, has that changed, like, does this industry change linearly? Does it grow fairly steady, or is it kind of big jumps? And are we on the cusp of any major shifts? What can you kind of tell us about the future of this, this sector, data, tech, etc?

Phil Harris 04:48

It's interesting, I think, as companies start, and I was at companies like Cisco, for example, when it was a very small company to when it was very large company. And this should be no surprise for anybody, the bigger the company gets, the harder. It is to change, and they really find that the only way they change is when they absolutely have to, not because they want to, and that's a combination of just inertia and shareholders expectations and a whole bunch of things. So I would say that the bigger the company is, the harder is them, for them to react. And so I think small, nimble companies tend to do much better when there's a lot of transformational technology and development and changes in the overall ecosystem we live in. I think just the second part of your question, you know, I look at the current situation as a point in time where a lot of companies will have to make some significant changes, simply because we're hitting too many walls, technological walls, commercial walls, geopolitical walls, that are really sort of confining what people can do. So I think what's going to about to happen is we're about to see a significant change, and this is not atypical in the industry. If we think about back into the into the start of what we would think of today as computer science around mainframes that were happening in the 60s. You know, for about a decade and a half, two decades, there was a lot of dominance around a particular way of doing things. And then some new innovational technology came along that rapidly changed, that scaled out, and it went from a very dominant set of players to a much larger number of smaller players who could then provide more innovation and more scale and more choice. And I think we're about to see that transition occurring as well.

Trevor Freeman 06:25

So is this, is there sort of like an analogous time, 10 years ago, 20 years ago? Are we on the cusp of, like, the big, the big change that we've seen before? Like, what would you compare this to? You know, in the last 2030, years?

Phil Harris 06:40

Yeah. I mean, I think there's been eras of compute. And if we say, I mean, we can find analogies outside of the compute world, but let's just stay in the compute, computing science world. I gave the mainframe example as one, and then we went to what we call client server, which scaled out rapidly. Telephony. We went from large, big telephone exchanges that started in in the government space, went to very large organizations. Now, basically we've completely scaled out how we make phone calls to use that now 20th century as a terminology. Nobody really makes telephone calls anymore. And we went through this with cloud computing and the Internet, where there was a change in the approach to the way we did things that suddenly gave us a scale out mentality, rather than a scale up mentality. And I think that's what we have to key in on here. Is it that we can take some of you? I was on a panel yesterday where we were talking about scale, and I say, well, to scale or not to scale? That is not the question. It's how do we scale? Do we continue to scale up, which is the current model, or do we start to think about scaling out, which is a more distributed model? So we go from a small number of big things to a large number of smaller things. And typically in computer science, whatever you want to start, storage, compute, memory, telephony, everything we've ever done goes through this arc.

Trevor Freeman 07:59

Yeah, it's it's interesting, and it's, there's obviously my brain's gonna immediately try and find those, those similarities between my world that I live in on the energy side of things. And it's the same question, like, there, there's, there is no path where we're not expanding the amount of energy we need. We're not going to be using more energy. But there are different ways to do that, and there are different paths we can take the business as usual that just grow, grow, grow, decentralized energy production and large scale transmission. Or there's a combination of like, grow those things, but also find alternative methods. More ders more sort of like close to consumer energy sources and storage, et cetera, et cetera. And people that listen to this podcast know I kind of go on ad nauseam about this. So lots of similarities. There another kind of framing or foundational thing that I want to talk through before we really get into the meat of our conversation is helping ground both myself and our listeners, and what exactly we're talking about here. So we, we all use, whether we know it or not, we use, you know, like cloud computing constantly, whether it's in our calls, how we're using the internet, using AI, more, more frequently. Now, what is the physical reality behind that? What's actually happening? What is the term data centre? What is a data centre for our listeners here? What does that look like?

Phil Harris 09:26

Yeah, let's start there. That's a great question. We started recognizing that the amount of power and space required for computers in companies and government in all sorts of different applications was getting larger than we could put in a room, in a closet near maybe where people were using it. We had to sort of create dedicated space, because the power requirements, the cooling requirements, just the noise. You can't hear this, but just in my basement, I have a few different compute systems that my wife continues to tell me is keeping my neighborhood awake. The reality is the environmentals of these things became very difficult. So we created these purpose built locations that had then different requirements in terms of access and facilities and power and cooling and staffing. And so they became a new way of thinking about building compute infrastructure at a building level, not just at the individual computers themselves. So a data is usually a very large room or building, I should say that houses large amounts of compute and storage and other networking equipment. There's a whole range of different technologies that go into a data centre that allows us to process information. That's what a data centre is. To give you some analogies in the US, there's about nearly 6000 data centres, depending on how you measure a data centre. In Canada, we have about 400 in Europe, there's about 750 that we can identify as standalone data centres. You can probably find more places where computers are outside of people's homes, but that's about the ratio we're looking at.

Trevor Freeman 10:59

And we're seeing, I think, and tell me if I'm wrong here, like, all this talk about the AI proliferation, data centre proliferation, we're seeing an expansion of these. Is that we're seeing the size of these data centres expand, or we're seeing just more of them popping up. Like, what does it mean when we say we're seeing, like, data centre growth because of AI, what does that mean?

Phil Harris 11:24

Well, it's fascinating, because now our worlds collide, because the way we now think about how to describe a data centre isn't in the square footage or the number of computers, it's in how much power it consumes, and we now measure it in megawatts, and it starts in 10 megawatts, or single digit megawatts, very small data centres, into average size data centres in the 10s of megawatts, up to now the hundreds and the gigawatts of consumption that you look at these hyperscalers. But I think we have to put this into a sort of a human scale. It helps us to put this in human scale. If I were to go back to ChatGPT actually about now, 15 months ago. ChatGPT-4. If you were to put that data centre footprint into the province of Ontario, for example, where you and I both are right now, it would be the equivalent of a million internal combustion engine cars driving 30 kilometers a day, if you ever drive up the 401 you probably don't want to see another million cars on the 401 Yeah, but that's the amount of energy that we can think of in terms of a data centre of that scale.

Trevor Freeman 12:33

Yeah, and again, kind of putting it in the electrical industry's terms, what we consider as a large load so we have a specific designation of a large load request that is anything five megawatts and higher. And like, up until recently, we would get one or two of those every once in a while, like, it's pretty rare to get a large load request. We are seeing large load requests coming in at a near constant pace now, like the number of large load requests we're getting, and a lot of it is because of this, not all because of data centres or anything like that, but a lot of them are certainly driven by that need for more more computing power, more facilities that support that.

Phil Harris 13:18

That's right. And at the same time, we're seeing a demand on on energy around now home, EV charging, and other aspects of the general distribution of the power, everything's taking a step function. But if I could just say one thing to your point about before I was seven megawatts, was a high load, then we may need to change that scale. It's almost inefficient to build a data centre unless you're somewhere above the 10 megawatt range, because at that point, get somebody else to do it for you.

Trevor Freeman 13:42

Interesting, yeah, and that's where it's sort of like, almost like, renting space in a data centre for a request of that size. Interesting, something that you know, I've seen kind of in your in your writing, on your on your blogs, is the idea that traditional data centres are really built for peak capacity, which absolutely mirrors the power industry. We build our electrical grids for peak capacity, and obviously that leads to a fair amount of inefficiencies. So if you're building just a peak capacity, if you're not at peak capacity, there is an inefficiency happening. There something that you identified. It's a stat from your research talks about graphics processing unit usage rates as low as 20 or 25% so I'm assuming that means kind of like three quarters of that hardware is sitting idle or not being used valuably. Tell us a little bit about what, what Cerio what you're doing, what your composable architecture specifically is doing to reclaim that wasted power and cooling capacity,

Phil Harris 14:44

Yeah, and so it starts off with your the premise you correctly raised is that, if we think about the the equipment, the physical equipment, and how we put these devices and these components together in a data centre, the same model we've been using today is, is about 3035, Years old in terms of individual compute systems, where we run applications, software that has memory and central processing units, those typical things you have in a laptop, or you have every computer. But then we put these accelerators, these GPUs, companies like Nvidia now are the one most valuable companies on the planet, if not the most valuable planet company on the planet, because that's the technology they develop. But we're trying to put these new class of accelerators into an existing compute model which wasn't designed for this. So then itself now starts to fragment the ability to leverage those resources in a data centre. And as you accurately said, it's interesting. If I could geek out on this a little bit for the energy consumer in the room, please. Do we think? We think about the notion not only the megawatts of power going into the data but we we think about what we call power usage efficiency. And that basically says, whatever the power delivered to a data centre, how much of that is applicable to the IT systems in that data centre, a good, well run, efficient data centre is about 1.2 that means about 1.2 times the amount of power that's used is delivered. Your home, for example, is about 30 times the amount of power we use is what's delivered. We are very inefficient from our home use, by the way. But that's another problem to solve in another podcast, but in this case, that's all true until we then ask the question, but what's actually being used at that equipment? And that's now in that 25 to 30% range at any point in time, and we refer to that as stranded and idle assets that, for whatever reason, aren't where the application is or aren't applicable to be used for the application that moment because they're in some other box, or it's a time of day when people use equipment. And by the way, equipment like that isn't being used 24 by seven, but it's drawing power 24 by seven, right? So there's lots of inherent inefficiencies in that model. So what we do is we provide the ability to dynamically have pools of resources where we can dynamically attach resources to a compute system as required, at the scale you're required, and allowing you to be much more efficient in the timing of that and the amount of equipment required to meet your end solution. And by doing that, we can increase the number of accelerators that you apply to a compute system, which inherently means you are much more efficient in those compute systems, because it's not just the computers. As I said before, there's storage, there's firewalls, there's load balances, there's networking equipment, all of that can now be much more efficiently used. All of that is drawing power.

Trevor Freeman 17:35

So is the idea, then, that the equipment not being used, or when you're at a lower demand time in terms of computing power, you've got physical equipment idling, sort of in more idle mode, drawing less resources that you can then ramp up so the peak amount of equipment still there. You're just being more efficient with it when it's not being used. And you've developed a way to sort of dynamically pull that in. Is that what I'm hearing.

Phil Harris 18:00

Exactly, I'll give you an example. A data centre here in Toronto wanted to have a block of 128 GPUs. They could have, they could they could service their customers with, with the current systems they were using previously to deploying our infrastructure, they had to require deploy, actually, 200 GPUs and a very large number of servers in the to house those GPUs. By deploying this area technology, they brought that down to 136 actual GPUs, and they reduced the number of compute platforms by a factor of four. So they reduced it by 75%.

Trevor Freeman 18:35

Yeah, that's fantastic,

Phil Harris 18:36

With exactly the same outcomes to their customers. With no no contention for resources, no oversubscription of resources, just more efficient use of those resources.

Trevor Freeman 18:46

Gotcha. So still able to meet that peak demand, but not sort of firing up that equipment when it's not needed.

Phil Harris 18:53

Well, not just not firing it, not having to have as much stranded equipment, because we can use all the equipment all the time.

Trevor Freeman 19:01

Gotcha. Okay, so in when I was kind of setting up that last question, I used the term composable architecture, and I'll admit that I pulled that from your material. Help me understand what that means. So you know that I've also seen you use composable infrastructure sounds a bit abstract, like, what? What are we talking about here? What does that actually look like?

Phil Harris 19:20

When a consumer, or someone who's building a data centre buys their computer equipment, they usually will actually buy the computers, the GPUs, the storage and other things at the same time, and they will get delivered together, and that box now becomes a unit of compute capacity. But the thing about that is whether you're able to use that entire capacity, the length in which that's a useful there's a lot of innovation churn right now as new things are coming through very quickly. But that box is now solid. You know, it's statically built for the rest of its life. Pretty much, it's very expensive. IBM did a study to take a server out of a rack, these big, six foot racks or bigger, where. These servers are housed with lots of wires going into them, power and data and all sorts of things. It's about $1,000 a minute to take one of those servers out of the rack and either change something that's broken, update something so they just don't get taken out of the rack. Because the average time to take a server out of the rack is about an hour. The math on that's pretty simple. So if I'm spending $60,000 to upgrade a 20,030 $1,000 server, I'm just gonna leave it there and buy another one. So that creates more of these stranded assets. So composability says, Let's separate these things into, as I said, pools of resources, compute accelerators and other devices, and have a fabric between them that allows us to, in real time, assemble a compute system that I need. That's the composing part as I need it, because I can now take the resources anywhere in my data centre, if you've got the right fabric, which we've built that allows you then to real time build that compute system with exactly the same capabilities, exactly the same performance, and without having to change any of your software or the way the service work. Everything has to be off the shelf to make this work, and that's what we've built.

Trevor Freeman 21:05

Got you. So, two of the terms, and you'll forgive me, this is sort of a new sector for me. Two of the terms that are used as metrics to determine performance are power usage, effectiveness, and you've kind of talked about, you know, GPU usage. Is the industry moving more towards that GPU usage metric? Is that just something that you guys are kind of leading the curve on? Or where are we at on that?

Phil Harris 21:34

Oh no, this is very much the industry way of describing not just efficiency, but requirements. And we use very weird terms for this. Every industry has their weird term. Weird terminology, and we're now moving to the for example, in AI, the number of tokens per second when you and I put a request or a question into ChatGPT or CoPilot or chord, whatever we use, those words get translated into tokens, actually numbers. Every compute system is just a big calculator. At the end of the day, we do, we do massive processing on numbers. How many of those tokens can I put into the system? How long does it take to process those tokens and give me a response? And the tokens per second, per watt is now what we're asking. So how many tokens a second, and what power per token is it costing me to process information? And that's the interesting way of thinking about how AI, for example, and that's value started this conversation will be measured is the most amount of tokens per second, per watt. Now, right now, we're focusing on tokens per second. We're not looking at the last denominator, which is watts. So that's why these data centres are getting so ridiculous. Ridiculously large. And you know, we even heard it in the in the State of the Union address in the United States earlier in the week, where, you know, there's now the administration pushing cloud vendors and AI vendors to say, Hey, pretty soon you're gonna be on your own about delivering power. Because, quite frankly, the way you're going. It's going to become untenable to think about that from a national grid perspective. Now, I think that may be a little bit into the future, but I don't think it's a completely unreasonable sentiment at this point.

Trevor Freeman 23:12

Yeah, and I mean, you're talking about, and we talked earlier about the just the scale of energy usage here is reaching a new height, a new level. And if we break it down to the individual racks, you know, these racks of servers or processors that you've got in your data centre, we're now talking about anywhere from 50 kilowatts to 100 kilowatts of cooling need. And that's the big driver of energy usage, I think, is correct here is the cooling need per rack multiplied by, of course, big numbers to get those, you know, 5-10-20-30, megawatt data cetnre we're talking about when we talk about cooling and we talk about, you know, hot spots within a data centre, how does your approach differ from kind of the standard way of doing it.

Phil Harris 24:02

So that's a great question, and I think we should explain why the cooling part, it's a bit like buying really good, expensive wagyu steak every day and then having to spend a lot of money on a gym membership to then go and burn off those calories. So we put all this power into power these compute systems, but then we have to keep them cool, and the harder they that, the faster they run, the more powerful they run, the hotter they get. But we need to cool them. So there's this relationship between the more power we draw, the more cooling we need, and cooling is becoming, as I said, that sort of trade off for performance. Now there's lots of exotic ways of cooling computer systems. We can just blow air across them. We can have a liquid like the radiator in your car, or we can literally drop these compute systems into bars of solvents. Ferdinand Porsche, I like to use of other industry analogies. Ferdinand Porsche, the guy who obviously designed the first Porsches and the VW Beetle, realized if I could distribute the heat of the engine block with a horizontal block, I could blow air across it. It was much more efficient than trying to put a radiator to actually cool down the engine block the way that other cars who have the engine in the front, and it's because of surface area. Now, if I've got to put all my GPUs and CPUs and memory close together, either in the same box or the same rack, that concentration of heat needs to be addressed with cooling. One of the ways we can address this is not only to be very selected when I compose the GPU, it's the only time it's drawing power, but also I can spread them out through my data centre by having a fabric that allows me to connect them to the compute systems with the same performance, but now I can distribute my heat generation. That means I can cool more efficiently, just like that Fernand Porsche analogy of the of the Porsche 911 because now heat over over, spread of distance and surface area is a more efficient way, which means it won't mean that we won't ever get to liquid cooling. I don't think immersion cooling is a good idea for lots of other reasons. It's a necessity, more than an optimization, but we can defer the complexity, the cost of those exotic cooling systems if we're more efficient in a way we use and design our data centres.

Trevor Freeman 26:18

And I guess there's a similar description there of, if you're concentrating all that heat in a specific, you know, physical area within a bigger building room, whatever you want to call it, that that cooling system is having to work to that peak cooling need, so to that hot spot effectively. But it's not working just on that spot. It's working across the whole physical area. If you're spreading that cooling need out across the whole room, one the peak is a little bit lower, and you're just more effectively using your whole cooling system. Is that fair to say?

Phil Harris 26:52

And that's exactly the right way of looking at this. And think about it from this perspective as well. The reason we have to cool is because if we don't call sufficiently, those devices become very unreliable and reduce a useful lifespan without going into who, because they keep this information confidential. But one large cloud provider in the US, for example, a GPU that normally has a lifespan of at least three years, is going down to about nine months right now. And the reason for that reduction the lifespan of the use of that GPU, is because of the heating characteristics within these boxes that are getting even with all these cooling mechanisms are becoming now a reduction in the lifespan. So that means we have to create even, remember, I said what it costs to take a system out of a rack. That means we don't have to apply an efficient and effective cooling strategy, our power strategy and cooling trategy, then we start hitting problems very quickly.

Trevor Freeman 27:50

Got you okay. Okay, so there's a mantra that I admit I hadn't seen before until kind of reading some of your material. It's, it's friends. Don't let friends build data centres. And I think it's referring to, you know, this, this move. And there's so many industries that kind of do this cycle of centralization to decentralization, and the sort of data movement went towards that centralization, and you saw these big, massive data centres. But there's, there's kind of a move now back to, let's call it decentralization or repatriation of data. And so for various geopolitical reasons, organizations, companies, governments, are wanting to pull their data back home and have it kind of be more in their control, living in their own servers. So how are you or how is Cerio helping companies kind of get back into the data centre business or repatriate their data without, kind of, you know, getting into the troubles that led for to that centralization in the first place?

Phil Harris 28:55

Yeah, and by the way, I can't take real credit for that quote. Cole Crawford, who was one of the early guys at Facebook before it became META, and was one of the leading voices in the Open Compute platform movement, which is try and standardize how we do these things. Cole is now the CEO of a company called Vapor IO, and what he was really saying is, it's so complicated and difficult to run data centres, let alone building the capital expense. AI isn't just one thing. There's lots of stages in the workflow of AI. We train these big models. You have heard of large language models like ChatGPT or copilot, but what we use them for the results of those trained models is what we call inference. Now you'll now hear about agentic AI, where we turn those results into actions. Okay, that's the agency part of agentic. Well, the use of AI in the corporate world is now becoming, as you said, both regulated, but from an intellectual property perspective, it's about how I control my data and my information. Because if I put that all into somebody else's large language model, I basically put. Populated somebody else's large language model with what might be my proprietary information or information that's very sensitive, and it's one of the reasons why you'll hear in the press about anthropic for example, trying to put guardrails around the use of their AI, because they're very sensitive to this. Most enterprises, governments of all sorts, have realized, though, they need to have run this in their own data centres, because they need to have control over this in control over this information and the use of this information, that's the repatriation you're talking about, moving these workloads now into the organization that previously said, Hey, cloud computing can take this problem. We're going to now figure out how enterprises, which are far many more of them in far more diverse locations, can now build their own data centres and get the right power, the right efficiency, the right capabilities at the right cost.

Trevor Freeman 30:47

Does that open the door? I mean, earlier, you talked about, you know, if we're talking about a five megawatt data centre, it's almost not worth it. You know, that's just sort of renting space in someone else's. How does that track with an organization that won't have enough data or enough computing power, whatever the metric is to warrant a 30 megawatt data centre for their own data, but wants to get that that control, wants to bring it more in house, is our is your technology helping those smaller data centres exist? Is that the correlation there?

Phil Harris 31:18

We can now move it into one of the things that we another couple of terms that may be an maybe not your your listeners may not be familiar with in the compute world or the data centre world, we talk of brownfield and Greenfield. Brownfield is that which is already there. Greenfield is something I have to build new. A lot of the Brownfield world is what is the predominant sort of quantity of compute power on the planet is primarily brownfield The question is, can I take that existing infrastructure and put the capabilities we've been describing in this discussion into those brownfields? So I can reduce the cost of the expansion of that because I can reuse the compute equipments there, I can now add just the discrete GPU technology, for example, into an existing data centre that doesn't therefore blow the power budget or the cooling envelope within that environment, but I can still now start taking advantage as I figure out what my larger plans are, and at the same time, how do we have a tier of providers? I'll give you an example. There's a company in, again, in Canada, think on who are building a data centre in in Ottawa, it's going to have its own liquid natural LNG as its source of power for its own power requirements. Why? Because they can have the power they need as they need it in that location, and they can provide that secure infrastructure for both government and private enterprises, and think on is certainly in Canada, one of those companies that's really seen to be a trusted partner in this. So it will be a bit of what can I do myself? How do I have a trusted partner? We think of sovereign AI a lot. That means trust more than anything, and that's becoming the new mechanism of thinking about this.

Trevor Freeman 33:04

Thinking about the environmental impact of tech and of data. We've talked about the energy usage here, but there's also the physical aspect to it. Of the pace of improvement in technology means we see obsolescence, or we see kind of technology being outdated fairly quickly. We all, like on the personal level. We all see this with our cell phones, our smartphones, our whatever tech we have at home that seems to be out of date fairly soon. I think that the stat, or that the saying that's out there is, you know, tech is kind of obsolete or becomes trash within three years. Obviously, this is not sustainable. Is this part of the drive of what you're doing? Is it? Are you looking to sort of extend the life of the physical equipment you've touched on this a little bit, but maybe expand a little bit on that?

Phil Harris 33:52

Yeah, this goes a little bit back to that Brownfield-Greenfield discussion. But one way of looking at I guess, is when I put all of these components into what the classic model, the current model, I put my central processing unit, my memory, my storage, my GPUs, all in the same box. What is the thing in that box that I want to take advantage of as new innovation happens, versus that which is happening over a slower evolutionary cycle? Well, right now, if I put everything in the same compute unit. Go back to my cost of taking that box out of the rack. I'm pretty much limited by the slowest innovation curve within that platform. Now as what I can take advantage over time. Interestingly, GPUs are innovating currently at a clip of about once a year. Nvidia comes out the new generation of GPUs once a year, but now we're getting more GPUs into the market. We're getting much more diversity, and that diversity means I'll have more options more often. But if my compute system itself is only innovating once every three years to your point, then if I don't decouple these things, if I don't have the ability to separate these innovations. Curves. I'm always stuck with the slowest innovation curve. One of the things we've done at serial with the fabric we've built and the platform we've built is to allow you now to, if you like, dislocate those innovation curves and those options, so as new technology comes along, I can apply it to the things that are innovating slower and still get the outcomes I'm looking for. And that will significantly increase the existing lifespan of equipment that's in people's data centre.

Trevor Freeman 35:26

So, looking at a data centre of the future, and not, you know, not far into the future, let's say 5-10, years from now, are we seeing some of the same technology still exist within that data centre, or is it, you know, everything gets cycled out within like, what's the generation of a data centre, for example? Like, how often, or how soon will we see it all cycle out?

Phil Harris 35:48

I think you there's a there's a technical answer to that, and the financial answer to that. The depreciation model, so that the capital infrastructure can be written off people's books over a three or five year window is very typical. So we see that there's just a financial inhibition to changing more or faster than that three to five year window. The technical churn, as I said, is happening much more rapidly in the technologies that are drawing most power but providing most capability. So one of the things that we're looking at is how companies now start leasing infrastructure, because if they lease the infrastructure, they can now recycle that and bring new technology in faster into their organizations. But to do that, you've got to have the ability to bring new technology in and not be stuck with these static systems that we have today. So there's a set of financial instruments, and now with work that Cerio is doing, technical capabilities that allow customers to really continue to innovate. So there's no real, hey, it's going to be all churned out in three years. I'll continue to innovate over those three years, reciting the technology that can stay where it is and bringing new technologies as it becomes available at the right financial model.

Trevor Freeman 36:56

I'm curious about what that innovation is. So you talked about Nvidia, kind of essentially a new GPU every year. There's a new version every year. What is the innovation? Are they just is it getting faster and more compute power, and therefore it's pulling more energy? And is that just like a perpetual increase, or is it kind of same compute power, less energy, like, do we ever see, I guess what I'm what I'm getting at with this little bit of a ramble here is, do we ever see that that rate of change in energy usage start to flatten out and come down while we still can grow our computing power? Or does energy usage just continue to grow? Like, are we on a bit of a path with no end right now,

Phil Harris 37:44

History taught us a little bit about this. Gordon Moore, who was one of the founders of Intel actually, we had this term called Moore's Law, and Moore's Law was basically this idea that every 18 months we'll double the number of transistors on a piece of silicon. Now, for those in the computer science world, we understand what that means. For the rest of the world, the Trans World. The transistor is the smallest unit of technology within the computer. It's the basic building block of how we build computers. The central processing is all the GPUs. They all come down to taking literally silicon and in a foundry, we call them, figuring out how to make as many transistors interconnect with each other in a in a smaller area as possible, or the most amount of transistors we can. So a bit of a geeky answer to your question. But the way that we look at how each innovation improves is, are we increasing the number of transistors, which means we can do more math? Remember, all we're doing is processing numbers.

Trevor Freeman 38:41

Per unit, per physical unit, right?

Phil Harris 38:43

Per physical unit.

Trevor Freeman 38:44

Okay.

Phil Harris 38:45

And the way we do that is in these big foundries that process all this silicon into these components. They have, what are called process nodes and the and literally how we etch a transistor, it's called lithography onto a piece of silicon. Tells us the power of that piece of silicon and the more I can etch. So we get into what we call the nanometer scale, or what we call a process node. So every time, if you really look into the spec sheets of Nvidia, every generation, they'll talk about how many nanometers their silicon process is based on. Because the smaller I can get that number, the more transistors I can have on the same amount of silicon, the more processing I have, but every transistor takes power. So with more transistors, I require more power, even though in the same physical space, it looks like the same amount of silicon. Therefore, your question was a great one. Do we ever get to zero nanometers? Well, no, we're going to hit a wall here eventually. So then the question is, that's the scale up model. Try and make one thing as big as possible. How about if we make lots of things powerful, but we have more of them in China, the last year, we heard of deep seek. Deep seek was a Chinese government sponsored effort to try and come up with a. Much more cost effective way of doing the equivalent to ChatGPT. They didn't do that with bigger GPUs. They did it with much smaller GPUs, but many more of them. And that comes back to how efficient I am in deploying lots of things together. And that goes back to my earlier point about we start with scale up. Inevitably, in the industry, we go to scale out.

Trevor Freeman 40:22

And is it fair to say that the power usage per transistor, is that fairly static? Like, is there efficiencies to gain there? Or your GPU is going to use more power because you're packing more transistors into it, and once you hit that wall, that's going to be the power consumption level, is that, right?

Phil Harris 40:43

Well, this is the games that these silicon manufacturers, like Intel, AMD, Nvidia, they're all trying to figure out how to sort of figure out new and interesting ways of packaging all of the silicon in these processing units. And we've got a whole industry and science around the packaging mechanism to make those tiles, and that we now think of them as little tiles of processing power, and some that will be doing very specific jobs. Some will be doing very general jobs. It's now getting to the point where the science around the packaging of these dyes or these tiles is as much as the of the of the innovation, as the actual tiles and the processing on them. So it's an extremely complex technical problem, and we are hitting some walls here, which is why I go back to my earlier point. We're now reaching a point where is it just a technical problem we're solving, or a technical, operational and commercial problem we have to think about? And this is that wall that wall that you asked me about right at the beginning of this conversation. Are we about to hit a wall? And the answer is, yes.

Trevor Freeman 41:46

Interesting. I mean, I'm always fascinated by like, what are the what are the really smart people in the industry focusing their time on? And it's so that's why we're talking to you. Of you know, you're looking at, how do we operationalize this. How do we get the most efficient combination and structure of what we're doing here? There's folks that are looking at, how do we pack the most computing power efficiency into these specific units? I guess there's an aspect of, how do we cool this in the in the most effective way, like, what's, how do we, you know, drive down the cooling power needed? What else is out there, in terms of, like, we have smart people focused on this efficiency. What's the thing that's missing from that, that sort of list?

Phil Harris 42:36

Well, I think maybe what's going on right now. And if I could just add a, unfortunately, just one more layer of complexity. Remember said we were processing silicon? Well, the Earth's got lots of silicon, but we don't have lots of places to process that silicon. The companies that are formed to process silicon into these processing units, we call them foundries. The world's largest is TSMC, based in Taiwan. And then we have Intel, we have Samsung, we have a few others around the world. Global Foundry is another one. There is a limit, physical limit, because these foundries are huge and they take decades of development and optimization. So if we start breaking ground on a new foundry tomorrow, we'll see output in about five years. So we have a constrained supply. So if I'm if I'm Jensen at Nvidia or any of the big silicon manufacturers, I'm going to optimize that relatively constrained supply to where I'm going to get the best return on my investment. And that's why this scale up model is happening. So given that we know that we won't have any more foundry capacity of scale for another couple of years, at least, then the reality is we've got to think differently about how we're thinking about the processing of that silicon. Do I want just ever bigger processes that become more expensive, more limited in where I can deploy them. And quite frankly, the top 15 consumers in the world of silicon consume about 80% of that silicon, if not more. How do I democratize that? Again, it goes from scale up to a scale out model, where I can use that same processing capacity to produce more silicon.

Trevor Freeman 44:20

Fascinating. Yeah, I just, I took us down a little bit of a nerd out path. You had me really interested in that. Okay, so last question here, we hear this term for a bunch of different reasons. Around the world right now we're hearing this term democratizing, happening a lot, and I know you've talked about democratizing, AI, what does that mean? What does that mean to you, or describe that for us?

Phil Harris 44:48

Yeah, I think it really means. Going back to my last point about if 15 big consumers of silicon are going to consume the vast majority of verbal supply chain, that makes the. At a losing proposition for the rest of the organizations and the rest of the governments and the rest of the individuals on the planet. So how do we make sure that AI can be built both responsibly from a sustainability perspective, right? And I don't mean just the ecological side, but that's important here too, but also from the ability to I was on a panel yesterday between the UK Government and the Canadian government, where we're looking at how do countries around the world have the ability to control their own destiny? And there's this whole notion of sovereignty and AI sovereignty right now that isn't because people want to have closed walls around them, that you want to have choice. They don't want to be dictated to by very dominant players where they, quite frankly, don't have the buying power to compete. You know that the amount of capital going into some of the AI companies, we saw $30 billion going into anthropic last week. That's actually a small increase in their capitalization relative to the other big AI players on the planet. That's $30 billion so we've got to think to ourselves, is that a sustainable model commercially? And the answer is no. So we've got to have technology. We've got to have the right ability to deliver power. We've got to have the right designs of data centres that can keep them cooled in an effective and efficient and responsible way. And we've got to be able to give them enough power to make them viable, to make them useful. That's the democratization we all have to be focused on.

Trevor Freeman 46:25

And we need every, I guess, to sort of round of the point is we need everybody to be able, everybody being, you know, whatever, major industry, countries, whoever, to be able to access that equally, so that we don't have to rely on the major players out there in order to do those things you just said, gotcha.

Phil Harris 46:41

That's exactly right. And look, there'll always be a pyramid here. There always has been a technology. There's always still the big players, right? But the question is, have the big players the stifled out the ability for smaller players to come up, innovate, provide choice, provide alternative ways of looking at things, and that's what got to make sure that we keep the and this always relies on some new technology coming along that enables that. Sarah believes that we've created that next layer in the stack, if you like, of technologies that gives us that opportunity to rethink the innovation curve going forward.

Trevor Freeman 47:14

Very fascinating. Phil, thanks for your time. I really appreciate it. This has been super interesting. It's not an area that I often get to spend my time thinking about so is great to chat today. As as you know, we always kind of round out our interviews with the same series of questions to our guests. So what's a book that you've read that you think everybody should read?

Phil Harris 47:34

Well, I'm not sure I can recommend this for everybody. One of the people who basically, along the lines of some of the things I've been talking about today, who revolutionized the computer world was a gentleman by the name of Linus Torvald in Helsinki in Finland. At the time, he's now based in the States, he realized that there was a dominance around how the operating systems on computers, the things that run the software, was limiting, basically, innovation choice and forcing us down a very closed path. So he wrote something called Linux, which was a new operating system. So be on your phone, your TV, your microwave that's running Linux today. Interesting because there wasn't an operating system that we could then generally deploy. That meant there was more developers had the ability to write applications, more hardware vendors could now have software they could run on their on their platforms. He gave the world a new innovation curve. And every time this happens to my last point, good things happen. Very good things happen for the world, for every individual on the planet. And Linus was one of those individuals who saw that need. And so his book, just for fun, and he's a very quirky guy, as you can probably imagine, is a great book about his philosophical approach to what it takes to change really big problems. And I would encourage all of you just to even just read the first few chapters. It's a fascinating view of how an incredibly smart man, smart individual took on probably one of the biggest problems we had in the 20th and 21st Century of computing, and solved it by recognizing you take a different path.

Trevor Freeman 49:11

Yeah, very cool.

Phil Harris 49:12

As far as shows, um, I don't know. I'm one of these guys. I've got two 13 year old daughters. So my wife and I get to watch TV for a very limited amount of time where we can watch it, about the things we want to watch, so we tend to sort of cram things in. But I'm a huge Aaron Sorkin fan, so if I ever need something on a rainy day to go back just to think about how the world could be, I watch the West Wing. It's a show that's imaginary. It's got incredible script writing, it's got incredible character development, but it really talks about how to think about doing the right thing as well. Now, whether you agree with the politics or not, that's a different question, but just the thought that smart thinking solves big problems, again, sort of It's a bit like the Linus Torvald book. It just speaks to me about sometimes we can solve big problems. With individuals or people who just had the right way of thinking about things.

Trevor Freeman 50:00

Yeah, I think that's the kind of, you know, call it entertainment, because it is entertainment, but it's the entertainment that sticks with you, and that we go back to time and again, is the ones that we can also, like, see the the underlying philosophy, or, you know, theory of change that goes into that entertainment. And it's, it's fun to watch. It's, you know, either humorous or dramatic or whatever, but there's still that underlying message. And I think, yeah, West Wing is a great example of of that. There's a handful of those other sort of classic shows that are in that line too. A free round trip flight anywhere in the world. Where would you go?

Phil Harris 50:40

This is hard. My wife and I were talking about this the other day, and I've had the luxury of traveling just about everywhere. I think there's 15 countries on the planet I haven't been to, but if I ever want to go to one place is Bali. And there's two reasons. One, my wife and I went there for a honeymoon, and it was the beginning of the most important chapter of my life by far. And secondly, it's because it has that balance of everything. It's I love to scuba dive. I love the rainforests, the jungle, the architecture, the people, the food. It just brings everything into one package for me. And so it just again. It's those things that sort of speak to you emotionally and also intellectually. It's one of those things that I could always go back too.

Trevor Freeman 51:26

Fantastic. Who is someone that you admire?

Phil Harris 51:29

In history or today?

Trevor Freeman 51:32

You pick, anything.

Phil Harris 51:33

that's fascinating. I think historically it's under Brit it's hard not to go back to some of my forebears, or my country's forebears, Alan Turing, who, against all adversity, social, political, technical, came up with an inspirational way of thinking about solving what are deemed to be unsolvable. And again, it's a tragic story. I think we've all, if you see the movie that was made about his life, it's a very tragic story, but it's an inspirational story about how, again, if you just take a different approach to solving what seems to be an unsolvable problem, you can you get smart people together. Doesn't have to be a big army of people. I think so. Turing is one of those people that always comes back for me t think, wow, if I could have just some of his courage and some of his imagination and some of his intellect, I'd be a very happy person.

Trevor Freeman 52:29

Yeah, and it's almost, I mean, obviously, a brilliant man, but it's the willing to think in a different way, or willing to approach a problem in a different way that I mean, there's a long list in history of major turning points that are as a result of someone thinking in a different way or doing something in a different way. And I think that's a great example of it.

Phil Harris 52:49

Just about the entire course of human life are in the midpoint of the 20th century, change on that, that man's inspiration, that man's imagination.

Trevor Freeman 52:57

Yeah, and that's, that's not an understatement. That's fantastic. Okay, last question, what's something about, kind of the energy sector, or, you know, your sector that that you're really excited about, or something that you see in the future that you're really excited about?

Phil Harris 53:09

Actually, I see it now, to be honest, there are things in the future. Hey, I have two 13 year old kids. I want to have a sustainable ecology and world environment for them to live in and bring their own families up in. And I think about how we can use power more efficiently, but how we can make it look sustainability is important. I want to see renewable, sustainable energy for the general world as a thesis right now. It's how we can be much more efficient in the use of power and the right power delivery. And I think, as I said, I gave the think on example, that's incredibly exciting, because now, if we can do that at scale, that's an opportunity to do that democratization that I spoke about. So when I think about the things that really excited me about the data centre world, the world I live in, actually that power generation and power availability in a clean, effective, well managed fashion is exactly what we need right now, while the rest of us are solving these transistor problems.

Trevor Freeman 54:04

Yeah, it's, I mean, our listeners are probably going to roll their eyes, because I say this all the time, but one of the things that excites me the most is seeing like we're in a period of change, and that's a really exciting time to be working in this and I kind of hear that from you in your sector as well, and I see it in mine, in the energy sector of we're actually getting to see some of this innovation, some of these like leaps and bounds forward. That's not to say there aren't still problems. It's not to say there aren't steps backwards as well. But it's very cool to be working on this in a time when we're seeing that change, and that's kind of what I'm hearing from you as well. Indeed. Awesome. Phil, thanks so much for your time. I really appreciate it. This has been great. Chatting with you.

Phil Harris 54:42

Trevor, the pleasure is all mine. Thank you.

Trevor Freeman 54:44

Fantastic. Take care.

Phil Harris 54:46

Take care.

Trevor Freeman 54:47

Thanks for tuning in to another episode of the thinkenergy podcast. Don't forget to subscribe wherever you listen to podcasts, and it would be great if you could leave us a review. It really helps to spread the word. As always, we would love to hear from you whether. Feedback, comments or an idea for a show or a guest, you can always reach us at thinkenerg@hydroottawa.com.