Transforming Knowledge Management with Large Language Models

Juan Sequeda Knowledge graph technology has been around for decades. The benefits so far accruing to only a few big enterprises and tech companies. Juan Sequeda sees large language models as a critical enabler for the broader adoption of KGs. With their capacity to accelerate the acquisition and use of valuable business knowledge, LLMs offer a path to a better return on your enterprise's investment in semantics. We talked about: his work data.world as Principal scientist and the head of the AI lab at data.world the new discovery and knowledge-acquisition capabilities that LLMs give knowledge engineers a variety of business benefits that unfold from these new capabilities the payoff of investing in semantics and knowledge: "one plus one is greater than two" how semantic understanding and the move from a data-first world to a knowledge-first world helps businesses make better decisions and become more efficient the pendulum swings in the history of the development of AI and knowledge systems his research with Dean Allemang on how knowledge graphs can help LLMs improve the accuracy of answers of questions posed to enterprise relational databases the role of industry benchmarks in understanding the return on your invest in semantics the importance of treating semantics as a first-class citizen how business leaders can recognize and take advantage of the semantics and knowledge work that is already happening in their organizations Juan's bio Juan Sequeda is the Principal Scientist and Head of the AI Lab at data.world. He holds a PhD in Computer Science from The University of Texas at Austin. Juan’s research and industry work has been on the intersection of data and AI, with the goal to reliably create knowledge from inscrutable data, specifically designing and building Knowledge Graph for enterprise data and metadata management. Juan is the co-author of the book “Designing and Building Enterprise Knowledge Graph” and the co-host of Catalog and Cocktails, an honest, no-bs, non-salesy data podcast. Connect with Juan online LinkedIn Catalog & Cocktails podcast Video Here’s the video version of our conversation: https://youtu.be/xZq12K7GvB8 Podcast intro transcript This is the Knowledge Graph Insights podcast, episode number 19. The AI pendulum has been swinging back and forth for many decades. Juan Sequeda argues that we're now at a point in the advancement of AI technology where businesses can fully reap its long-promised benefits. The key is a semantic understanding of your business, captured in a knowledge graph. Juan sees large language models as a critical enabler of this capability, in particular the ability of LLMs to accelerate the acquisition and use of valuable business knowledge. Interview transcript Larry: Hi, everyone. Welcome to episode number 19 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Juan Sequeda. Juan is the principal scientist and the head of the AI lab at data.world. He's also the co-host of the really good popular podcast, Catalog & Cocktails. So welcome, Juan. Tell the folks a little bit more about what you're up to these days. Juan: Hey, very great. Thank you so much for having me. Great to chat with you. So what am I up to now these days? Obviously, knowledge graphs is something that is my entire life of what I've been doing. This was before it was called knowledge graphs. I would say that the last year, year-and-a-half, almost two years now, I would say, is been understanding the relationship between knowledge graphs and LLMs. If people have been following our work, what we've been doing a lot has been on understanding how to use knowledge graphs to increase the accuracy for your chat with your data system, so be able to do question answering over your structured SQL databases and how knowledge graphs increase the accuracy of that. So we can chat about that. Juan: But what I'm interested next now is I'm calling this how do we extract knowledge from people's heads? And this isn't new, this is not new, this is back from the '80s and early '90s, all the work on knowledge acquisition. People wanted to create rule-based systems and expert systems, and you need to have knowledge engineers, who go talk to the domain experts and be able to extract their knowledge and codify that. Well, that was hard, and it was brittle, and it was not scalable. And now we have a very fantastic tool called LLMs that I'm very excited to use them to be able to find ways to scale that back. So I think what's old is new, and I think the opportunities to use these new tools to do the work that we've wanted to go do, but we haven't been able to go do, which is just to improve knowledge management. Juan: And one of the reasons why I argue that, we still try to solve the same problems. We're solving the same problems today that we had 30 years ago, but the core problems continue to be there, and I think it's this lack of knowledge. Now, we have a way to scale this out so people can stop complaining, saying, "Oh, it's just too hard, too expensive." That's my long-winded answer of what I'm up to. Larry: Yeah, well, I'd love to jump right into that, because there's all these intersecting things, at least in my world. I run in a lot of different worlds, the content world and knowledge management folks. Anyhow, like you said, in the AI world, there's this longstanding, the expert systems and all that kind of stuff, and then there's the knowledge engineers and enterprise knowledge management stuff. It's always been really hard work, but parts of that are now more amenable to, if not automation, at least acceleration. Can you talk a little bit about how LLMs have kind of advanced the stuff you've always been doing and always wanted to do more of? Juan: Well, so LLMs, first of all, we look at the LLMs, and I like to look at things like as inputs and outputs. So the input is a prompt, and the output is going to be some text around this prompt. That's at a super high level. But what's been fantastic is that if you give it some context as an input, and part of that context, you also give it a task of what you want it to go do, the output that comes out actually aligns very nicely to it. Now, yes, there's hallucinations and stuff, but look, if it gets 80%, 90% done, that's better than nothing. It's better than actually doing things completely manual. And guess what, if a human does it, they are not always getting things 100% correct anyways. So that's at a super high level. Juan: So the example I give all the time is organizations always complain, "Oh, how many customers do we have? Oh, we have so many different customers. It's so complicated." Okay, so what? You're just going to cross your hands and just live like that? No. How about you figure out at least how many different ones are there, and what are the implications of those things? But how would you do that? Well, you need to go start talking to people, and then somebody needs to go talk to all these people and start cataloging and combine all this stuff. Well, what if I actually just posed that question to somebody, saying, "Can you describe what is a customer for you in a sentence or in 10 seconds, in 15 seconds, right? Give me a minute rant of what is a customer for you?" Just take that, and you got a transcript, you got text right there. Juan: So now, I could prompt the LLM to say, "Hey, here's this definition of a customer. Can you please give some definitions?" Here's where the prompting, this is where the interesting experiments are going to come in. And then you can now do that for so many different other people. And then you can do further analysis to figure out, saying, "Hey, where are the overlaps of this stuff? Actually, you know what? Can you codify this? Can you turn this into code? Can you turn this into already some sort of a knowledge graph on it?" And I can now start comparing these things and realize, "Oh, look. There's really X different definitions of this, where the alignment is on 80%, and the 20% of this." Now we know these things. Now I can actually go back. That's the type of stuff that we can actually go do very quickly in the initial experiments that I've been doing. It's like, this stuff is working so freaking well. Juan: I wish we had this back 20, 30 years ago, but now it's time to go look at the problems that we're trying to go solve there. But back then and now we can actually solve them with these new tools. So that's what I'm really excited about and this is something that I really hope, one, I want to show that this can be done, but second, but most importantly is the so what? Once we've done this, what is the value for an organization? So I think those are the ties that we need to go do still. It's like we all complain about we don't know the definition of the customer. So what is the problem for that? What's the money that we're leaving on the table? How much money are we losing by not having a complete definition of a customer? Now that we do have that, we need to be able to say, well, I do have that now. How is that improving? I think that's really important for the next step. Larry: So we're a couple of years into this now, where is the value emerging? You've talked a lot about the workflow in making knowledge graphs and capturing knowledge, but how does that percolate up to business value? Where are folks finding that payoff? Juan: You mean from knowledge graphs in general or...? Larry: Well, just from this whole scheme you're describing, the ability to accelerate understanding of knowledge and what you just described is what I do. I've been doing content modeling for many years and it's sort of very similar kind of work and a lot of that back and forth. I've already seen it accelerated in my content modeling practice as well. And so the way you're talking about,