Synergy of Knowledge Graphs and LLMs

Fran Alexander When Fran Alexander looks at the current AI landscape she sees some interesting parallels between the Alien vs Predator science fiction franchise and the way RAG and other architectures are combining LLMs and knowledge graphs. We talked about: the analogy she draws between the Alien and Predator science fiction franchise with LLMs and knowledge graphs how the human-esque (if malevolent) cognitive and behavioral nature of Predators aligns more with knowledge graphs and how the unpredictable and stochastic nature of Aliens aligns more with LLMs how the eloquence of LLM outputs can deceive humans the lack of explainability and transparency in both Alien and LLM behavior, and the opposite in knowledge graphs the difficulty of dealing with baked-in biases in LLMs the lack of repeatability in LLMs and the opposite in KGs the current trend of architectures and practices like RAG that draw on the strengths of KGs and LLMs to get better results, just as the Alien and Predator media franchises combined forces how over the past year or so investment in LLMs has overshadowed all other investments, just as Aliens are out to wipe out anything that's not an Alien her approach to AI architectures that combine LLMs and knowledge graphs how different kinds of people consume LLM output how she helps enterprise decision makers choose whether to address a use case with a knowledge graph or an LLM how taxonomists and ontologists can use LLMs in their work the Alien Loves Predator UK Facebook group and Alien and Predator on a seesaw Alien and Predator cosplay actors on a seesaw Fran's bio Fran started her career as a writer and editor of dictionaries and thesauruses in the UK, and, as technology evolved, she specialised in information architecture, search systems, and digital archives, and more recently, the use of semantics in knowledge graphs and LLM applications. Having worked on reference publications including the Collins English Dictionary, and as Taxonomy Manager for the BBC Archive, she now lives in Montreal, Canada, and is the Senior Taxonomist for Expedia Group. She was Taxonomy Bootcamp London’s Taxonomy Practitioner of the Year 2023. Connect with Fran online LinkedIn Video Here’s the video version of our conversation: https://youtu.be/VWwDBIws6G8 Podcast intro transcript This is the Knowledge Graph Insights podcast, episode number 15. When two impressive domains converge, amazing things can happen. When the Alien and Predator science fiction franchises joined forces, both enjoyed new commercial success. Similarly, in the AI world right now, Fran Alexander sees knowledge graphs and large language models combining forces to create retrieval augmented generation and similar architectures that work together to create systems more useful and valuable than the sum of their individual capabilities. Interview transcript Larry: Hi everyone. Welcome to episode number 15 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show, Fran Alexander. Fran is an independent taxonomist and ontologist based in Montreal. And so, welcome Fran. Tell the folks a little bit more about what you're up to these days. Fran: Hi Larry. Well, it's nice to talk to you again. I really enjoyed talking to you on the previous podcast that we did a little while ago. And that one was kind of a bit of a general introduction to taxonomies, ontologies, thesauruses, knowledge modeling and semantics. But this time, I thought we could talk about knowledge graphs and LLMs. They're a big hot topic and I did a presentation earlier on in the year for Taxonomy Boot Camp London, Bite-sized Taxonomy Boot Camp London. That was a lot of fun and has been really popular. A lot of people have been asking me about it. I've revisited it a couple of times and that's LLMs versus knowledge graphs, Alien versus Predator. Larry: Okay. So why Alien versus Predator? Why not King Kong and Godzilla or...? Fran: I did think maybe King Kong versus Godzilla. Godzilla as LLMs and King Kong as knowledge graphs. Certainly, the idea is that it's a fun analogy. It's a fun way to start thinking about the differences between knowledge graphs and LLMs. It's not supposed to be a serious study of science fiction characters, but you certainly could pick your own pair of monsters and do the analogy there. I did consider, as I say, I did consider Godzilla versus King Kong. That would work, but personally, I happen to really like the Alien franchise. I'm a bit more familiar with the Alien franchise. I thought it was really, really fun to have the Alien versus Predator crossover and that's actually become quite a successful franchise in its own right. So yeah, so you could use many monsters, pick your own monsters and run your own analogy. But as a starting point for talking about LLMs, they can seem very Alien. They can seem very scary. So that was my starting point. Larry: Interesting. Yeah, and so I'm just trying to, I'm getting it as you talk about it. Well, tell me a little bit more about, because in ontology work, we figure out what the entities in the domain are and ascribe properties to them. What are the properties of an Alien that make you think they're like an LLM? Fran: So Aliens are very different from humans and one of the reasons why I like the Alien versus Predator analogy is in these characters, they're societies and the way they operate and what we know of them are very different. So Aliens, we don't really know much about Alien society. They're not at all like humans. They have acid for blood and what an Alien does is basically, just goes around killing everything in its path and making more Aliens. We can't really communicate with them, we don't know much about them. They're very, very different. Their approach to the universe is very, very different to ours. Fran: Whereas Predators, are still big, scary monsters, still very powerful, but they're much more humanoid. They kind of look humanoid. They move in a more humanoid way and Predators actually have a much more human-like society. So they have some kind of moral principles, maybe not that many. They're basically, they're mercenaries for hire, but they do have social structures, social hierarchies, complex societies. You can't go and hire an Alien to work for you in the way that you can hire a Predator. Fran: So Predators are starting to get into those kind of complexities, hierarchies. You talk about the way that we build knowledge graphs, they're usually very specific. So Predators will have a specific target. They're out to assassinate their designated target that they're doing for money. Knowledge graphs are very specific and targeted and precise. Whereas Aliens, they're not really involved in that at all. They're just going off to do their Alien thing in their own Alien way. So that was my starting point. You've got these two contrasting characters, one that's very, very strange and different. The acid for blood of Aliens, I compared to LLMs having maths for blood. Fran: The way that LLMs work in a kind of, and I'm not an expert and machine learning engineers and experts, I don't know whether they'd completely agree with my overview, but the way I look at how LLMs work is that, you take a big corpus of something - texts, documents, images - chop it up into lots of little pieces, and then you layer on lots and lots of algorithms and calculations and machine learning to figure out what the probability of one piece appearing next to another piece is. And that essentially is what LLMs are doing. They're very, very complicated probability engines. Whereas knowledge graphs are built up using structures and hierarchies and conceptual models that come from humans and come from people. Fran: Now I don't know anyone who learns by chopping things up and calculating probabilities of bits of text. Humans don't read books and learn from them like that and humans don't discuss and describe concepts and approach the world like that. But a human way of looking at the world is in thinking of things with structures and hierarchies that we're used to. Our taxonomies that are kind of like the backbone with knowledge graphs, that kind of parent-child broader narrower concept relationship is something very, very familiar to us. We talked last time about supermarkets being organized with the dairy section and milk within the dairy section and types of milk. That's very natural to us and we approach the world with a mental map or a mind map. Fran: It's very much like in ontology. So when you build a knowledge graph, you tend to start with subject matter specifics like the Predator having a specific target and a specific reason for going out after its targets, a specific motivation and you'll build up, you start to put your lists, your labels, your taxonomies, your ontologies. You're building them up from a very human perspective with your subject matter experts or your business drivers to come up with a knowledge base that answers specific questions in a focused and targeted way. So that's kind of where I started with the analogy. Larry: Yeah and especially the way you punctuated at the end there. But when you were talking earlier about the LLMs and instead of acid, they have math for blood, maths for blood. And that's so Alien and I think what's interesting to me, there's weird contrast there between, I agree with you that that's like a metaphor and an analogy that really works for me in feeling LLMs, but I think because of their conversational interface that they typically use, I think a lot of people perceive them as more, ascribe more humanity to them than they deserve. Does that make sense? Fran: Yeah, I think that makes sense. I think it's really interesting. I think it's one of the dangers actually and there are probably other sci-fi monsters.