Handwritten Text Recognition Experiment

Robert Sanderson Yale University manages huge collections of precious cultural heritage artifacts housed in multiple museums, libraries, and other collections. Using knowledge graph and ontology engineering design patterns that he has developed over his career, Robert Sanderson helps scholars, researchers, and the general public access information about — and make connections across — millions of unique items in Yale's collections We talked about: his work as Senior Director for Digital Cultural Heritage at Yale University the knowledge graph and ontology engineering design patterns that guide his work the scope of his work — improving discoverability of Yale's extensive collections of artifacts, facilitating the management of collection information, and even collecting data on physical artifact storage facilities how their linked data approach lets researchers easily connect information about artifacts and information housed in multiple museums, libraries, and collections how the growth of LLMs has affected their KG user interfaces how AI is accelerating their ability to add to their knowledge graph the millions of artifacts in their collections that aren't yet accounted for the compact nature of their three-billion-triple KG ontology, just 10 classes and 50 relationships the extensive vocabularies and taxonomies they use how they handle the need to reconcile the identity of lesser-known people who don't have a Wikipedia page or other authoritative references available how they balance the competing needs of comprehensiveness and usability as they build their knowledge graph how knowledge graphs facilitate discoveries that other search tools can't current opportunities for post-docs to join his team to work on leading-edge AI projects Robert's bio Dr. Robert Sanderson is the Senior Director for Digital Cultural Heritage at Yale University, where he works with the libraries, archives, and museums to ensure that data and other digital efforts are coherent and connected. He is the principal architect for Yale’s cross-collection discovery system, LUX, which is built on the Linked Art specifications, for which he is an editor. He is also an editor for the IIIF specifications, was the co-chair and editor for JSON-LD and the Web Annotation data model in the W3C. He has previously worked at the Getty in Los Angeles, Stanford University, Los Alamos National Laboratory, and the University of Liverpool. His current areas of work and research are at the intersections of cultural heritage, knowledge graphs, data usability, and generative AI. Connect with Rob online LinkedIn email: robert dot sanderson at yale dot edu Rob's LinkedIn post series on KG and ontology design patterns The 10 Design Principles to Live By Ontology Design Patterns Naming Things Avoiding Reification Foundational Ontologies Multiple Inheritance, Not Multiple Instantiation Predicate Reuse... Meh Document your ABCs Separate Query and Description Semantics Usable vs Complete acknowledgements Video Here’s the video version of our conversation: https://youtu.be/SMAVyrL3aSU Podcast intro transcript This is the Knowledge Graph Insights podcast, episode number 46. When your job is to help scholars and the public discover information about millions of cultural heritage artifacts that are housed in multiple museums, libraries, and other collections, you need a powerful — but also manageable — knowledge graph. That's Rob Sanderson's role at Yale University. He and his team apply time-tested ontology and knowledge engineering design patterns to help people discover — and see the connections between — these precious human artifacts. Interview transcript Larry: Hi everyone. Welcome to episode number 46 of the Knowledge Graph Insights Podcast. I am really delighted today to welcome to the show Robert Sanderson. Rob is a professor and the senior director of Digital Cultural Heritage at Yale University, the Ivy League School in Connecticut. Welcome to the show, Rob. Tell the folks a little bit more about what you're up to these days. Rob: Hi, Larry. Thank you so much for inviting me to be part of the illustrious lineup of guests on your podcast. So yeah, I'm Rob Sanderson, as you said, Senior Director for Digital Cultural Heritage at Yale. So I work with the libraries, the archives, and the museums and other collecting organizations at Yale to help them to be more connected with linked data organizationally and more coherent in the way that we do things digitally. So our projects really focus on discovery and access to the collections in service of the university mission, which of course is teaching and learning, research, and preparing our students to be the next generation of leaders in the world. Rob: So for that, the university invests very heavily in the collections, which is fantastic. We are super proud of the 300 years of collecting that we've done. But we want to make sure that if you can't come to New Haven, you still have as good access to those collections as possible. And the ability to find amongst the many millions of objects that we steward exactly what it is that you need. So a lot of our projects focus on describing the collections in a more computationally tractable way so that that discovery can be better. And also how to manage the information that's associated with the collection, but isn't a museum object or a archival object itself. For example, I have two postdocs that are openly available. So if you are a few years out of your PhD or just about to graduate, do get in touch to work on how to use AI to extract the ownership history or the provenance of particular museum objects from the archival content that we also manage. Equally, how can we align research data sets with the collections? So we also have a natural history museum as well as two art museums. How can we align the environmental datasets that are out there on the web with the natural history specimens that could have been impacted by those environments? Rob: Yeah. And then equally, we look at the environment of Yale. So we have a large project at the moment to set up environmental monitoring with sensors for light, for humidity, temperature, and so on, to be able to generate a large data warehouse aligned with linked data with the collections so that we can have evidence of what the effects of the environment are on the collection items themselves. Larry: Interesting. That is so fascinating. What a fascinating remit. One quick thing about what you just said. Is that about humidity and temperature and all the things that might affect the endurance of these physical artifacts? Rob: Yep. Yes. That's right. Larry: Yeah. Rob: We have about 200 sensors around the place monitoring every five minutes a new data point, which if you think about it, it's actually not that much data. Larry: Yeah. I have to say, I just love that you're doing data stuff along with it. That you're not just sitting in a dusty old room collecting things. You're doing cool modern stuff too. But hey, I want to quickly interject how we met, and I just want to put this in because we won't have time to talk about it today, but I want people to know about this fantastic series you did. That's how we met was somebody drew to my attention the series you've done on ontology design and on knowledge engineering design patterns. And I'll point to that in the show notes, but I just wanted to mention. And the more I think about what you just said, because I didn't know all of this background before we started recording, I'm like, "Oh, this is even better than I thought." So I'll point to that in the show notes. Larry: But the main thing I wanted to talk about today is what you were just talking about. This amazing cultural heritage operation that you're running there, especially the knowledge graph component of it and the AI, of course, because we're in the 21st century, and that's all anybody talks about. One of the things we talked about before we went on the air was how AI is accelerating the ability for you to build your knowledge graphs of these cultural heritage artifacts and data. Can you talk a little bit about that, how AI is helping in that? Rob: Yeah. Of course. Absolutely. So just a little bit of a background about the knowledge graph itself first before I get to the AI part. So over the past five years, we've built without AI, a very large scale knowledge graph, well, in cultural heritage terms of very large scale, which has about three billion triples in it. And it follows the principles and the design patterns that you mentioned in those posts on Linked Art. It then aligns the people, places, concepts, events, objects, works, collections that we manage here at Yale across the two art museums, Natural History Museum, the dozen or so libraries. There's also a collection of musical instruments, the Institute for the Preservation of Cultural Heritage, and we even have a little outpost in London, in England for art history research that we include. So that work uses the linked art ontology, which is based on the foundational site CRM ontology and is publicly available both in terms of the data, you can just download it. But also in terms of the graph queries, we don't force you to learn SPARQL. We have a user interface on top of it, which allows you to generate queries and find the objects that you are looking for. Rob: So one of the things that we noticed first about the user interface is that only about 5% of searches are actually using the graph affordances. Mostly, 95% of the time, people just put in keywords because that's what they're used to. You go to Google, you type in your five favorite keywords that you think might match and you scroll through the results. However, now in 2026,