On this week’s show, Matt and Nick meet Alexander Feick, Vice President of eSentire Labs, to discuss his book “On Trust and AI” and why organisations need fundamentally different approaches to govern AI systems. Alex explains that whilst traditional computers multiply human intent predictably, generative AI acts more like an autonomous agent capable of creating its own decisions—something between software and people that requires new thinking about trust and verification. Using the example of AI-generated legal briefs with fabricated citations, he demonstrates why hallucination isn’t just a model problem but an architectural failure: when systems lack transparency, explainability, and alignment, organisations cannot verify the outputs they’re trusting.
Alex introduces the concept of a control plane—deterministic software and logging that wraps around AI models to verify outputs work-by-work. Instead of allowing AI to directly cite cases (which it could fabricate), the system only permits references to IDs in a verified database, creating verification breakpoints between untrustworthy model output and validated information. This shifts human work from production to verification, applying critical thinking and domain expertise to judge whether AI reasoning is sound. Alex argues that “when creation became free, trust became the product”—as AI makes generation near-costless, real value concentrates in verification systems. Organisations must ensure verification capacity scales with generation capacity, otherwise they risk producing “AI slop”: unverified outputs that erode trust and create liabilities rather than value.
You can read Alex’s book here: https://www.esentire.com/on-trust-and-ai/
Show transcript – automatically generated by Descript – below.
[00:00:00] Matt: Hello and welcome to episode 343 of WB 40, the sort of fortnightly podcast with me, Matt Ballantine, Nick Drage, and Alex Feick.
[00:01:08] Welcome to the first. February edition of the show and the first show that I’ve actually been involved with. So the front of house I’ve obviously for all of them, I am doing my devious work in the background to be able to edit the things. But this is the first time I’ve been on the show, uh, in 2026.
[00:01:29] And, um, I joined by co-host and also soon to be co-author Nick Drage. Nick, how the devil are you?
[00:01:40] Nick: I’m alright actually. How are you?
[00:01:43] Matt: I’m, yeah, I’m good. Having said that, I kept waking up in the middle of the night last night. I’ve got a little bit of a cold and as I woke up in the middle of the night, I thought to myself, I know what I can do.
[00:01:54] I can, in my head, I can rehearse. The talk that I’m giving in a couple of nights time in Nottingham which is gonna be the first or big public talk about randomness, which is the thing you and I have written the book about. And so in my brain, in my half, half awake, half asleep brain, I started to do my presentation and by about, by about slide three, I’d fallen back asleep again.
[00:02:19] And I don’t think this is really necessarily a good advert for the quality of the presentation I’m gonna be giving on Thursday, but you know. Apart from that, the reason why I kept working up as well though, is ’cause the night before I had the experience of what can only be described as the most tortuous user interface that there is on this planet, which is the, what happens when the backup battery in your smoke alarm decides that it’s not got enough power in it.
[00:02:49] And what it does is it makes a cheer up, very loud noise, but it does it in such a way that there’s a kind of. Improbable gap between them. And if you’ve got as many of the things in, uh, your house as I have, it’s next to impossible to know which one it is. But it’s really annoying. So it keeps, ’cause it’s designed to be annoying and they always go off, there’s a physics thing behind this.
[00:03:10] They always go off in the middle of the night because at the coldest point is the point where the batteries are least efficient, so therefore they will always be triggered. So two in the morning, I’m on a small step trying to not fall down 14 flights of stairs to be able to get the thing off. So just so apart from that, I’m fine.
[00:03:30] Nick: I thought you were making that bit up, but that makes sense. It does, doesn’t it? About batteries? About And also, yes, the interface is, the interface to them is horrible. I think it’s fair to say friend of the podcast, Dave Gray, who we both know to varying degrees, uh, had one in his, I think his basement where he does his video calls, online meetings, and.
[00:03:54] It was just, he didn’t know where it was in that huge, you know, sort the typical huge American basement. So we just all got used to it. There was just a beep sporadically in the video calls. He runs every week for something like three months and instead of one week, we all noticed, I think halfway through that it had gone and either, and I don’t think it replaced, but I think it was finally it’s died.
[00:04:17] Now, you know, your basement might catch fire, but at least you can have decent video calls.
[00:04:21] Matt: Uh, yeah. I, I, I need to look up at some point how many people are injured or killed through the process of having to replace batteries in the middle of the night. And then it’s the kind of having to shove screwdriver into the side of it.
[00:04:34] And the one literally at the top 14 stairs, and the way the builders installed it was the way you have to push it. The direction of push is down the stairs, so on a tiny little stair Oh,
[00:04:45] Nick: is the same.
[00:04:46] Matt: Yeah. Hal. So anyway, I think what I’m completely, there’s, there’s
[00:04:50] Nick: a, there’s a podcast in that, but not this
[00:04:51] Matt: episode.
[00:04:52] No, no, no, no. Absolutely. But all I’m concluding is I’m lucky to be alive and you’re lucky to have me here. There we go. Um, have you been up to anything other than risking life and death in the last few weeks, Nick?
[00:05:02] Nick: Oh yeah. ’cause I just suddenly realized, oh yeah, it’s the bit where you asked what I’ve been doing.
[00:05:06] Matt: Yes.
[00:05:06] Nick: That’s great. So I grabbed my work calendar and kind of. And I hope this sounds cooler than it is kind of, I can’t tell you, but as much as I can say ’cause of like NDAs and so on, is fighting an a well-known online chat platform and losing, let’s just leave that there. Helping develop and. Plan and soon we’ll be running a multi-agency war game, multi-agency exercise, which has proven really challenging, but really interesting, especially for use of LLMs, which we might get onto later.
[00:05:43] And just planning another exercise where. An industry is looking forward to what might happen politically in the UK in the next few years and is looking to plan ahead, which is really good to hear about, really interesting to be involved with. And, um, just an an interesting sort of business and research projects.
[00:06:02] All the times I’ve popped into this podcast and see things about the way war, professional war games and exercises might go. It looks like it actually is going that way. Finally.
[00:06:13] Matt: That’s good to hear. I was talking to some people at government department today in the current state of British politics.
[00:06:17] They were wondering what would be happening in the next week, not necessarily the next, you know, couple of years. I think the short term thing has come back forward. It’s like 2018 all over again.
[00:06:28] Nick: Well, I mean, we can’t, I haven’t looked to the news for like six hours, so we can’t comment. Even though you are led it this quickly and get it out in a couple of days, we, you know, we could be completely out of date,
[00:06:38] Matt: who knows?
[00:06:38] Nick: But anyway, anyway, so that’s, that’s, that’s where we are.
[00:06:41] Matt: Excellent. Alex, welcome to the show. Thank you for, uh, for joining us. Uh, uh, how’s the political state of, uh, ’cause you’re in, in Waterloo in, um, Canada. Are you in a state of relative calm? Is it total turmoil?
[00:06:55] Alex: Uh, there’s, there’s been a lot of commentary since Kearney did his, uh, his big speech which everybody of course followed domestically at home.
[00:07:02] But, uh, other than that not too much, I would say
[00:07:05] Matt: relatively calm. So
[00:07:06] Alex: we’re all kind of wondering to see what the follow out of that is gonna be.
[00:07:09] Matt: Yeah, it’s, um, it’s terrible when you have a quite volatile neighbor. I think that’s the best way to be able to puss it. Um, have you been up to anything interesting in the last week or so?
[00:07:21] Alex: Last week or so I mean, I’ve been just getting final stuff, uh, for the print copy of, of my book to come out. And, uh, I’ve been working on, uh, AI driven malware, reversing a little bit, exploring that. A few other things just with, uh, some of the soc flows and, and research arrangements coming through for work.
[00:07:38] So it’s, it’s an interesting time at work right now, that’s for sure.
[00:07:42] Matt: Very good, but you haven’t been having to battle things in the middle of the night that might have caused me a certain death.
[00:07:48] Alex: No, not so much. We had, uh, a bunch of coyotes and foxes around our house, uh, the other night that woke us all up with, uh, they were playing out in our yard.
[00:07:56] So that was interesting. But, uh, no, no smoke alarms. I did actually have my life, uh, saved by one of those, uh, a couple years ago though, like our carbon monoxide detector went off ’cause our furnace was backing. Uh. Carbon monoxide into the house. So,
[00:08:10] Matt: see, this is the story I need to be able to motivate me to change the batteries in these things.
[00:08:14] So thank you. That’s good.
[00:08:15] Alex: Yeah, it, it, it absolutely did save my life just a couple of years ago. So they, they, they do work and, uh. I’m very thankful that we had them installed with batteries and we didn’t leave them run for three months and then die. So
[00:08:27] Matt: Well that idea of staying with that, but I guess you do just phase this stuff out after a while anyway.
[00:08:32] We, we used to live on the flight path to Heathrow and it’s incredible actually how much noise and disturbance you can just zone out. In your life. But, um, there we go. Anyway, you mentioned your book. We are going to be talking about that and we are going to be talking about AI and trust and governance.
[00:08:49] So I think we should probably crack on.
[00:09:48] Nick: Okay, Alex. And we’re both old enough that we’ve seen tech technological changes and they’ve, they’ve become relatively familiar in their lifetimes. But what is it about? Generative AI or LMS or whatever definition you want to use, what about it? What is it about them that makes them different and require the kind of thinking that your book contains?
[00:10:14] Alex: I mean, if, if, if I think about it getting back to first principles, I guess it’s, if we look at what computers have been able to do so far, it’s all been essentially multiplying the intent of a person in a predictable, deterministic way. When you look at what some of the new generative capabilities are doing, they’re actually capable of creating decisions, right?
[00:10:35] Like you can argue whether or not you know, they truly have intent from a philosophical perspective, but from an actual business and trust perspective, you have to consider them agents that can invent their own workflows, come up with their own decisions. And so, they allow you to do a lot of things that previously were impossible within software, um, that used to have to be done by people.
[00:10:55] And so if you think about them from a trust perspective, you have to think about them both in terms of the software being capable of running at speed and scale. And also in terms of them being somewhat like people and that you actually have to look at the decisions that they’re making and what they’re actually deciding to do as being something that you have to secure on each individual call, because you can’t predict what they’re going to do with perfect accuracy under all data conditions.
[00:11:22] So it’s, I think for those two reasons at the, at the very lowest level, I’d say that’s, that’s kind of why you need to think about them differently from a security perspective, because they’re not really. The old computers of before, and they’re not really the same as people. There’s something that’s a blending of both qualities and you kind of have to think about that differently if you’re gonna secure it properly.
[00:11:41] Nick: Yeah, that’s an excellent point. Like the old problem with computers was that they will do exactly what you tell them, you know, regardless of whether it’s what you meant, what you type into the command line or whatever is what they’ll do or what files they’ll delete and so on. Whereas you say I really like the way you put that in, that they’re not.
[00:12:00] What we’re used to from computers, but also, and I think that point, the second point is key, is that they’re also not people. They’re this different third thing that we, as you might tell by how slowly I’m talking, like we struggle to find the right words and language to describe them. Um, so your, you’ve got a book that’s proposing solutions.
[00:12:25] What makes your work. Unique or noteworthy among so many people getting into this area right now.
[00:12:34] Alex: So probably the. I, I was trying to write a book that fit the need for business leaders to sort of understand what they could do with ai and help them understand sort of like the conceptual reasons why it was different and what they had to think about from a security perspective.
[00:12:50] Sort of from an approach of, of really like enterprise design and business first, and, and not really delving into the weeds. I think there’s a lot of like, really in the weeds technical books about the how. I think that there’s a lot of existential books that are like philosophy or, or, or sort of thinking about where this might take us, but there’s not a lot about how to practically lead a business through adopting AI in a way that’s safe for today.
[00:13:14] From, from sort of like a people and process perspective, and that’s really the core of what the book is about. It’s not so much about, you know, the, the technical specifications of, of models and, how, how you actually implement nist. It sort of talks about business architectures and how you can actually think through those problems as somebody who’s not necessarily in the weeds as a deep technologist.
[00:13:35] Nick: Yeah, it seems really, and I, I shouldn’t sound as surprised as I am. From skimming through it in preparation for this, it seems really practical, which as you say, is what a lot of authors have avoided because I think it’s easier to be hand waving and philosophical. But it’s easy just to quote, I mean kind of stat blocks about the technology that’s out there, which is always out of date by the time the authors put that together, whereas your seems more like a guide.
[00:14:08] Aimed at people who know enough to understand the, the content of the book. But, um, what’s your plan with regard to keeping it up to date in such a fast moving field? Is it, uh, well, rather than guessing, I’ll let you explain what’s the plan? What, when, as we were saying about British politics earlier, you know, as soon as you turn your head and look at something else, potentially you’re out of date.
[00:14:32] Alex: Yeah, so I do a, a, a couple of things. One, I’ve got some companion resources that are published on my website. So like, if you want to like, explore how the book intersects with something like nist ai, RMF I’ve put together something that analyzes the text and sort of, connects it. Um, I’ve tried to do the text as an AI agent as well as as a book.
[00:14:50] So I have sort of like an evolving set of commentary that’s that’s kind of coming on around. The, the topic online. But I think the other thing is when I was writing the book, I was really trying to go down to what are sort of the fundamental things that you need to understand about building a secure system around the ai.
[00:15:09] And I think that’s gonna remain relevant regardless of how things go with ai. Because at the end of the day, I think we’re always gonna have the question of well, what do we do with models that are both incredibly useful? But also something that we don’t feel comfortable trusting directly.
[00:15:22] How do we actually guarantee that? And so I’m not talking about technical specifics, I’m talking about patterns that you can use in order to build that up. And I think they’ll remain practical for quite a while. I think that, that we might scale back their use, but I think that the patterns for how we actually achieve transparency, explainability, and alignment, they’re gonna remain the same whether we’re talking about auditing a small sample or whether we’re talking about having to apply them every single time to critical decisions and they’re worth thinking through.
[00:15:51] Matt: There’s an example that you give in the book of, a legal brief being produced as part of a lawyer team and generative AI tools are used and it creates a very impressive, very detailed, very referenced output, but the citations are basically made up. Um, and you describe that as being. And you talked a moment there about it being architectural problems.
[00:16:22] You know it’s gone wrong because AI hallucinates, but architecturally what, what’s going wrong in an example like that?
[00:16:29] Alex: So I think those are, are a couple of failures there, right? So, in that sort of situation, if you think about a lawyer who’s asking it to generate a brief, if you’re not architecting your system right, there’s no transparency into what the AI was actually pulling and, and referencing.
[00:16:45] There’s no explainability about how it’s actually like justifying each individual thing that it’s reasoning about and using. And so it’s really, really difficult to answer the question about whether or not the system’s actually aligned with what it is. That you wanted to produce. And I think there’s a new type of knowledge work that’s emerging, which is around figuring out how do you make the cost of verifying something much cheaper than the cost to produce it by hand the first time yourself.
[00:17:12] Because AI can produce almost anything that you’re after, but if you can’t verify it, you can’t trust it. And we keep seeing these examples hitting the media. Like I was actually surprised I wrote the first, the boardroom pilot state study that I opened the book with was something where, um, that hadn’t happened when I started writing the book.
[00:17:30] And then by the time that I got to the point where I was releasing the first versions of the book, it had happened several times and had made media headlines around. So I actually got to quote it and cited in there, and it was much stronger. But I see those sorts of things happening all sorts of times in the businesses because.
[00:17:45] A lot of businesses mistake what it is that you mean by transparency and explainability, and they think that means that the AI should explain itself or that you should have transparency into the thinking process of the ai, and that’s not really it. What you want is transparency into what the AI actually looked at and did you want an audit record of the artifacts that it’s actually interacting with?
[00:18:07] And when you’re talking about explainability, you actually want to be able to review the reasons that it’s attaching for each decision that it’s making. And then decide if they’re actually worth following along with or not. It doesn’t matter what the model’s thinking internally. It matters what it’s touching and how it’s justifying what it’s touching and if that’s actually in alignment with what you’re after.
[00:18:26] Matt: Uh, and that feels like it’s quite a distance from what our expectations. It’s come back to your point about how, systems used to be predictable and so. Interestingly, on the one hand, we’ve got a whole bunch of assumption that has built up in culture of organizations around management and technology, which is whatever the computer says will be, right?
[00:18:46] And the only reason it won’t be right is because of either bad data or bad coding, but not because it, there’s a mistake. The stuff that comes out of LLMs by Design doesn’t fit that model because it’s predictive. So therefore it just, it’s just a completely different sort of information that’s coming out.
[00:19:06] The other bit’s interesting that I think that and this is something I’ve been observing for years now because it brings outputs that are almost humanistic in their approach. They’re producing pros, they’re producing long form text, which is something we. Or, or images. And those are things that we just simply haven’t expected computers to be able to do for many years.
[00:19:28] We also anthropomorphize them as well. So on the one hand, a computer should provide right answers, and on the other hand it should be seen as being human because it looks like a human, the kind of deep cultural level. There’s a lot of stuff that we’ve gotta unpick in organizations to be able to dispel both of those.
[00:19:47] I think.
[00:19:48] Alex: A hundred percent. And it also has societal ramifications because it’s, uh, you know, when, when computers can generate those sorts of things. And we previously thought that they could only be produced by people. We attach a level of trust to the things that are that type of content, and we haven’t yet learned as a society how we’re going to be able to trust those types of content.
[00:20:10] Now that they can be cheaply and easily mass produced by computer systems.
[00:20:15] Matt: And those, those trust things keep getting pushed a bit as well. So, I’ve recently got a new phone, one of the new pixel the Google ones and it’s got a hundred times Zoom on it. Now, the a hundred times Zoom is mostly software.
[00:20:30] It’s not through. Physics. ’cause I mean, the phone would be enormous if it could have that sort of size of Zoom on it. And so it takes a photo and then it uses ai and it’s, it’s transparent in saying it’s using ai, but it’s using generative tools to be able to make a fairly blurry pixelated image into something that isn’t blurry and pixelated.
[00:20:50] And it’s not reality.
[00:20:52] Alex: Nope. It’s, it’s, it’s entirely generated. And how do you. How do you handle that when, you know all of the media that you are looking at could be generated that way? You don’t know. And I think this is something that businesses really need to adjust to. I’ve I’ve, I have a whole section on that because I, I really believe that the value in most of the things that we are looking at nowadays is what is actually verifiable.
[00:21:19] And what we’re asking enterprises and businesses to do for us is to take something that we could probably generate for ourselves off of chat, GPT or, or Claude or some other tool, and make sure that it’s something that we can actually trust. And so if the businesses don’t get the trust portion right, then why is it that I wouldn’t just generate whatever they’re producing and selling me off of chat, GPT or, or Claude or, or one of the others?
[00:21:46] Matt: So. Think of that from a, a business rather than a technological perspective. You’ve talked about these sort of three pillars of transparency, explainability, and alignment. How does how does a leader in an organization even big begin to approach this though? ’cause this isn’t about, and again, this sort of, the established models are, we invest in some machines.
[00:22:12] We, we still think of technology as being mechanistic, even if it’s now producing cultural artifacts, whatever else. So what, what are the things that need to be done to be able to start making a shift in an organization so it can even begin to start to be able to deal with these things in a different way?
[00:22:33] Alex: So if I go back to that example that you were raising earlier with the legal brief I mean, at the end of the day we can all go. Read the contracts and make, come to our own conclusions about whatever it is that that was in them. But we could not get it right. And we could also, uh, go to chat GPT and we could ask it for a case brief and it could not get it right.
[00:22:53] So when we’re going to the lawyer, what we’re really going for is we want the lawyer to be basically saying, yeah, that this is solid. I’ve, I’ve reviewed this. You can take this to a judge. They’re not gonna, throw you out of court with it. And so in order to do that, I think what you actually have to do is think about how you can.
[00:23:10] How you can tell somebody. That this is without using an appeal to authority, that this is something that they can actually trust. And to do that, I think what you actually have to do is you have to show the reasoning process. And because that’s a new type of work that we haven’t done before, most businesses haven’t actually thought through what the software to support that would be.
[00:23:28] But I actually think it for almost every knowledge work. Job type that you have out there, you actually have to think about building a new system of software that’s optimized around that work that is now unique that only humans can do, that multiplies the value of the AI outputs that your business can generate.
[00:23:45] So in the case of the legal brief, you might want a system that actually takes the AI generated case and, you know, compares it to, to good law and jeopardizes all of the notes so that the legal reviewer can actually take a look and see. Where the AI is getting its cases, where it’s getting its reasoning, and then because they’re lawyers and they understand all of that stuff, they in, in the right ui, they can very quickly see if, if the draft is actually something that would pass muster in front of a, a judge or not.
[00:24:13] But if you’re just going straight to the model and you’re asking it to. You’re not using pointers to a database, you’re not thinking about the system of work to actually verify the output. You just get three pages of text. And the fastest way that you would be able to actually verify that as as, as a lawyer is basically go repeat the whole process of generating the case brief.
[00:24:31] And that’s the trap that so many businesses are running into is they think that, you know, they’ve got an AI that generates the case brief, therefore they must be almost there. And really that’s the first 10% of the trip. The next 90% is figuring out how you’re actually gonna be able to stand behind that and do that in a way that isn’t gonna cost you more than just doing it by hand in the first place.
[00:24:51] Nick: Isn’t I dunno how else to put this. Isn’t that horrible? I’ve not been involved directly and it’s much more. Matt and Chris’s area and the other co-hosts and our audience. But so what you’re saying is we don’t need sort of more investment or arguments about resource allocation. This is like a change in thinking and a change in processes, which is traditionally the absolute worst kind of thing to have to do within an organization.
[00:25:19] But this technology makes that essential. I think from sort of my general analysis, and especially from what you’ve said is a, there’s just a fundamentally different meta approach if that’s not too pretentious, but meta approach to how to make this work. Am I understanding that correctly and should we be optimistic or pessimistic about the ability of organizations to do that?
[00:25:46] Alex: I think it’s, it’s gonna leave a lot of organizations flat-footed and unable to shift. It’s gonna be really, really challenging for a lot of the established players. I do really believe that it is a fundamental shift of that magnitude, though. I think it’s, it’s it’s actually fundamentally changing.
[00:26:01] The boundaries around what’s possible to do with software, right? Conventional software gets you so far and this new type of software takes you through things that previously you could only do with people and every enterprise that you can sort of point to today that works in knowledge work is predicated around the computers, do everything the computers can, and then the people do what the computers can’t.
[00:26:23] And the way that I would look at it is, it’s sort of like the transition that accounting houses faced when we developed. PCs and, and modern spreadsheets. Previously, people operated with ledgers and you would employ an entire floor of accountants to add numbers together in order to audit books.
[00:26:38] And if you can’t adapt to the emergence of, spreadsheets and, and, and computing technology, I don’t think that you can still get by with the old processes and in the same way. I think that that’s, that’s a similar degree of disruption that we’re actually facing when AI’s hit businesses. It’s very hard to think through how the human roles have to change.
[00:26:58] It’s very hard to get it right. But everybody out there is, is sort of experimenting with this and, in, in a free market sort of situation, somebody will get it right and when they do, they’re going to disrupt your business. And that’s something that I think everybody’s sort of struggling with right now.
[00:27:13] Matt: I wonder if it’s actually even further than the, you know, the shift from accountancy on paper to accountancy in Lotus 1, 2, 3, and, and Excel? Uh, I was chatting to somebody recently that the evolution of power in manufacturing initially there was no power and everything was hand built. And then we had things like horses and then people started to realize that if you found a source of power, you could attach things to it.
[00:27:41] So, uh, rivers providing water power, and then you’d have these very complicated power distribution systems, which were involving pulley and axles and wheels and cogs and steam engines worked on that same principle with rubber belts and, and all of that meant that actually where the work happened was where the tools were and.
[00:28:08] Yeah, that meant that you would have factories where the distribution of labor was split by the type of task. So there’d be a drilling room and a sanding room. I don’t know anything about these things, but you, you know, the, the, you’d have a room full of the same machine, and then you would bring the product to the machine room to be able to have whatever it needed to happen next.
[00:28:27] Happened to it, then electricity came along and still the distribution of power was working on the same principle until people started to realize that gave you much more flexibility. ’cause you didn’t have the constraints of whirring axles and stuff. And then barring the fact that the Venetians got there in the 14 hundreds first when they’re building ships.
[00:28:49] But you had the production line model and at that point then you’ve kind of brought the machines to. The the product rather than the product of the machines. And that fundamentally with what Ford and Taylor did in the early 19 hundreds, that fundamentally changed the nature of machine work in an auto factory or in a carriage factory or whatever else.
[00:29:12] And then further, you know, automation and, and, but the, where we are at the moment. Is that we’re not even beginning to think about how the, the nature of the work needs to change to be able to take advantage of these new technologies because we’re so early into it, realistically with, with LLMs.
[00:29:33] But the, the, the changes that may well come outta that will be deeply profound because it will be that actually entire, entire ways of working will change, but probably not now or in the next couple of years. ’cause that’s what a change takes. 20, 30 years perhaps. And if you look at how we use, business technology, I’ve been working for 30 something years.
[00:29:56] When I started work in, uh, the early nineties, we had network file stores. We didn’t have the internet, the network file store model of how we share information within organizations. Even though it’s now all on cloud-based systems, if people use ’em at the Microsoft platform, it hasn’t really changed very much.
[00:30:12] Since the days of NT servers and Novo NetWare it takes a long while for that kind of human behavior change and, and organizational behavior change to catch up with the opportunities that technologies have got.
[00:30:27] Alex: I think to some extent That’s right. I actually think that AI is a little bit different in that though, because the thing that it’s actually optimizing is the type of mental work for actually enacting that type of change, and so.
[00:30:41] If you know what you’re doing and you wanna plan out a project for how you’re gonna change something. So like one of the things that we’re doing in, in my own organization is, um, we’re trying to think about how do we adapt security operations center processes and previously that would involve like, you know, getting interviews, figuring out what the processes look like, doing cognitive task analysis work to split all that stuff apart, redesigning the processes.
[00:31:04] You would be talking about you know. Armies of people coming in to try and take a look at that. We’re actually able to use AI within that process and now we can literally just have an interview with somebody over the phone and at the end of the interview, AI will generate for us a complete.
[00:31:21] A flow diagram of what that process actually looks like and allow us to actually like, review it and discuss it. And I’ve led sort of like some of those enterprise change processes before. And I would say that it’s, it’s easily possible right now to compress, six weeks of work into a 30 minute interview with some of the, the tooling outputs that you can actually get with this because you literally, you just have to verify that you’re getting it right.
[00:31:46] And the AI can do everything from listening to your conversation, to transcribing it into well-written notes, to transforming it into a project roadmap plan, giving you a process description, recommending how the software could look, and you need to verify every stage of it. But once you’ve actually got that, you can accelerate your way through some of those changes so much faster than you ever could before.
[00:32:08] Matt: And I guess that also those kind of planning processes. There probably isn’t a right answer. And so if you can accelerate yourself to a point when you can start actually doing the work to see what works and what doesn’t, you cut out a whole load of not only time consuming work, but also work that actually is just polishing the problem rather than actually trying to work out what to do next.
[00:32:36] Alex: Yes. Very much so, and you can do rapid implementations of prototypes in 30 minutes. Whereas previously, if you wanted to implement a prototype for something, you might finish up your call, talk about it for a few days, get a plan together for what you were gonna do, then go off to a small dev team spend two weeks coming up with a pilot.
[00:32:54] Now it’s interview. AI coding assistance quick review of the production code, test it in the POC sandbox, and you can be looking at, a new way of doing things in the span of an afternoon.
[00:33:10] I don’t think most of society is ready for it. I think the primary thing that’s gonna slow us down is cultural inertia. But the startups especially have enormous advantages on their side when they’re approaching this. And I see a lot of. Like little companies that, you know, may entirely fail, but there’s a lot of them that are out there and they’re all trying new things.
[00:33:28] So I think there’s, there’s definitely going to be a lot of disruption in a lot of spaces as a result of this over the coming few years.
[00:33:37] Matt: You also talk in the book about the idea of, uh, the control plane. Um, and essentially, I guess to an extent, kind of observability about this stuff and what, what’s going on.
[00:33:51] Can you just unpack that a bit?
[00:33:54] Alex: Yeah, so essentially what I think the limit is around ai and I spend a couple of chapters in the book sort of discussing threats and break points. But ultimately what I get down to the core thesis of the book is, look, you just can’t trust a model. Um, we’re not at a point where we can trust a model yet.
[00:34:09] We might never be at a point where we can fully trust a model, but that doesn’t mean it can’t do useful work for you. And so the control plane is the idea that in order to actually get value out of a model, what you need is a system of deterministic software and logging around what the model is actually producing for you.
[00:34:27] So that work product by work product, you’re capable of actually understanding what is what is this AI decision gonna drive? What is that AI decision gonna drive? And, that’s essentially what the, what the control plane is for all of that. So if you think about in the legal circumstance, you constrain it so that when the model is generating its output, it has access to a database of case law.
[00:34:50] And it can’t actually directly cite cases. What it can do is it can look up that database of case law and it can cite goid. And when it cites a goid into the response the software that is gonna be used by your legal reviewers will substitute that goid for the actual looked up case. So they know at that point that there’s a break point.
[00:35:10] They’ve gone from something that is untrustworthy, that the model could have hallucinated and made up to something that is absolutely 100% verified by software to be real. And that system of injecting those break points into the out point, or sorry, into the output, is what I would consider to be the control plane for the model and everything is, is, is how do you build that in a way that that scales,
[00:35:35] Nick: that just sounds really clever. Like, I’d like to say something more. Uh, I see Matt smiling. I’d like to say something deeper and more insightful, but just that. That way of working and that new way of working. And especially in that particular example you cite, I assume that means that someone over a, an overworked human can’t just say, well, that’s probably right.
[00:36:01] These are usually right. I’ll just pass that along the process. It’s like, no, I have to look this up to, to make it make sense to other humans. So you are, you’re forcing people to do that kind of work and it sounds relatively. Interesting. Rather than you’re just checking like the output of a machine. It’s more analytical than that.
[00:36:21] So you, you still
[00:36:23] Alex: have, I think, I think it’s not even that the person has to do it. Right. Like the, the control plane could do it for the person and it could actually tell the ai. Okay. The, the case that you cited for me, right? You gave me a goid for a case, and I went to go look up that goid and it didn’t exist.
[00:36:39] Fail. That model output gets rejected and the model has to generate a new one until it gets one that actually passes all the original checks. What the human’s doing is actually more valuable. The human’s gonna look at it and they’re gonna go, I know that case is real because the deterministic software told me it was real.
[00:36:54] The model is saying that that case can be used in this part like this. And if they’ve got all the, the stuff that they trust beside them, they can look at it and they can actually reason about what the AI is saying there. And they’re providing that judgment as a lawyer, as somebody who actually has the technical expertise in the domain to understand that argument and just weigh in on it.
[00:37:16] Yeah. That’s plausible or no, that, that, that, that wouldn’t make sense. That case doesn’t support that argument that you’re making. But it gets back to that core critical thinking skill. And I think essentially if we think about what people will need to be doing in order to be valuable in the modern knowledge economy, it’s not gonna be just producing rote task outputs anymore.
[00:37:35] It’s gonna be applying critical thinking skills to something that they’ve actually, gone and gotten the critic credentials to be able to say that they’re an expert in that field and they can work with it.
[00:37:44] Matt: There is though a, a school of thought that. Proposes that. The problem with that though is that how do you get the next generation of lawyers or other knowledge workers trained?
[00:37:54] Because actually the, the drudgy grunt work that is done by junior people, whilst not very high value, gives people the exposure to just the material, the content, the ways of thinking that through repetition eventually gets them to be the sort of person who could make the judgment of calls that you’re talking about.
[00:38:15] How do you, well, how do we break against that argument because it’s quite compelling.
[00:38:21] Alex: I have thought about that from a systems perspective. And the way that I would look at that is that if you build your control plane properly the human’s job is to verify. What, like the AI is actually producing the cases that are unknown.
[00:38:35] How do you keep the human sharp? How do you train the human? You would present them with cases where you know what the right answer is, but the human doesn’t because they haven’t seen that particular example before. And you would see if they actually get the right answer when you know it. So say you’ve got 10 lawyers.
[00:38:50] Three of them are senior and seven of them are junior and they’re all reviewing cases. The cases that come in that, that get the highest scrutiny, get a senior reviewing all of the arguments, who has the highest rated accuracy rating across everything? But juniors could be contributing to some of that work and, you know, for less risky decisions.
[00:39:11] And you could classify that in your control plane as to which decisions would go where you could shunt some of those cases to juniors. And you could also use the same system that you’re using for verifying. To train people on the skills that they will need to verify the more sensitive edge cases. And essentially, if you, if you think about it as, as sort of like a dial, that you could flip it, it’s like, you know the highest risk decisions that need to get verified by the most skilled professionals.
[00:39:35] Go to your most expensive, most senior people, and then the ones which are, lower risk decisions. You can shunt to some juniors and maybe you can do two or three juniors instead of a senior in a certain circumstance if, if there’s an enormous pay gap between them. But I definitely think that there are systems of essentially throttling the severity of the work.
[00:39:54] To, or sort of r routing, it rather not throttling it to the right person so that the highest risk decisions end up with the most senior people. And the interesting thing about that is that the whole system builds up training examples that you can use. And maybe it, it becomes a situation where we send some of those off to universities and we say, yeah, you’re training to become a lawyer.
[00:40:14] Here’s the system of verification that the, that the lawyers are using. Here’s some of the, the sensitive, decisions that could have gone either way. And you can get yourself comfortable with that. With a system of working.
[00:40:26] Matt: Interesting. Other than obviously get a hold of your book, which we’ll put links to on the, uh, the show notes.
[00:40:33] If somebody wanted to be able to start to think about the better management of trust within ai, what would be the first thing that they should start to look at?
[00:40:50] Alex: It’s a tough question. If they’re not sure about the consequences, I would say go into the business processes. And whenever you’re looking at, if you’re a leader and somebody’s bringing an AI project in, the question that I would ask is not could AI get efficiency outta this thing?
[00:41:05] I would ask the question, what is the worst thing that could happen to you if the AI. Made the most egregiously wrong or bad decision at this stage of it. And build your system accordingly. Because if you can show your customers that when AI is acting in a bad way, when the model’s been poisoned, when it’s been prompt injected, or when it’s been data drifted away from where it should be, that even under all of those circumstances, you can still show them how it is that they’re getting something that’s worth paying for.
[00:41:37] Then you’ve got yourself something solid and I would, I would probably start there and everything else follows from that.
[00:41:47] Nick: That feels like.
[00:42:52] Matt: Fascinating. And as I said, we will put links to, there’s a free online version of the book as well as the physical version you’re having produced. Is that right?
[00:43:02] Alex: Yep. Uh, that’s, that’s right. So anybody can read it online if they want to or play with it as an AI agent.
[00:43:08] Matt: Brilliant. We will, um, stick the links to that on the show notes.
[00:43:11] Um, so that’s part of the show where we work out what is coming up in the future, but not too far into the future. ’cause who knows? The next week ahead, Alex, have you got anything exciting coming up for yourself?
[00:43:23] Alex: I guess I, I got a few things I wanna try and, uh, extend the number of crosswalks that I’ve got for the book before the physical copy comes out.
[00:43:30] And then I also have some stuff, uh, to do at work around, uh, pulling AI into some more processes and, and using it to build workflows faster. So looking forward to that as well.
[00:43:40] Matt: Excellent. Um, and how about you, Nick? What’s the weak head looking like for you?
[00:43:46] Nick: I’m intending to get some personal game development exercise development projects done to an extent where I can share them with people, but I have a worrying feeling.
[00:43:57] That’s what I said the last time I was co-hosting, which I think was last year. So, so, so let’s see how that goes.
[00:44:06] How about you, Matt?
[00:44:08] Matt: Uh, well, as I I think mentioned earlier in on Thursday I’m up in Nottingham at the University of Nottingham’s Innovation Center to give a talk about randomness, which is the thing that you, Nick and I have been working on for the last two years. So I have got a bag full of props.
[00:44:25] I’m taking a dice with me. The first part of the presentation will be like a normal presentation. The second part will be. Driven by what? Is, comes out with a throwing of a dye. And then the third part will be people being given parts of the book to be able to talk about in smaller groups. So it is, there’s lots of stuff that I have not done before, which makes it quite exciting.
[00:44:46] And then, uh, the week after, ’cause I won’t be done doing a show next week. I’m going to goway with some friends for our annual Christmas dinner, which is always slightly delayed because reasons. But, um, there’s six of us heading out to Claire Goway, which is a small town, about 10 miles outside of Goway City itself.
[00:45:07] And we will be enjoying, I mean, given that the UK has apparently had not a single day without rain since the beginning of the year. I thought I’d go and escape the weather by going to West Coast of Ireland. ’cause that’ll be dry madness.
[00:45:21] Nick: Best of luck.
[00:45:22] Matt: Thank you very much. Anyway Alex, thank you very much for joining us this week.
[00:45:26] Some really thought provoking stuff.
[00:45:28] Nick: Thank you,
[00:45:29] Matt: uh, Nick. Always a pleasure.
[00:45:31] Nick: Thank you.
[00:45:33] Matt: And we will be back in a couple of weeks time. We’re back on a better thing now, and I think we will be talking about partnerships with Shalene and Chris which is really interesting. Very different tack, much more about people than about technology, but that’s how we mix it up here on WB 40.
[00:45:51] So until then have a great fortnight and we’ll be back with you soon.
[00:47:04] Alex: Thank you for listening to WB 40. You can find us on the internet at wb40podcast.com and on all good podcasting platforms.