(29 Oct 2026) “Machine learning gives us more flexibility in evaluating the relationship between records. Rather than requiring a certain amount of metadata to line up, the model can consider different signals together to determine whether two records are likely to describe the same resource. The ISBN, when present, is one particularly useful signal because it can remain consistent even as other parts of a record change. It can help the model connect a sparse pre-publication record with the fuller record created after publication.
Our machine learning model is also a more computationally efficient way to identify these duplicates than traditional de-duplication methods. With machine learning, we can conduct the analysis up front, identify the duplicates, and then process them for merging. What previously could take months can now be processed in a matter of days.
But here’s the thing: we’re not just creating a machine learning model and letting it loose on WorldCat. A vital step in our process has been keeping human expertise in the loop to continually analyze, train, refine, and adjust the model that’s running. When a cataloger tells us we’ve merged something incorrectly, we can roll back the merge, investigate what happened, and use what we learn to improve the model. This approach has helped us evolve the model with community input over time.”
OCLC explains how they combine machine learning and human stewardship for data cleaning. Read more here.
