I have two lists of names that are approximately similar. So I wanted to figure out the best way to find the matches. For example, some sample data, the lists might look like: list1 = ["Colby, James"; "Arnoldson, Robert J"; "Linkletter, Mary"] list2 = ["Colby, J"; "Arnoldson, R J", "Jefferson, Robe…

I’ve done a fair amount of this kind of thing and have tried several different ways…a few things I’ve learned: Invest in tidying up your strings as it pays off (e.g., remove punctuation, separate first/last names, convert everything to upper/lower case, etc.) Take the time to figure out which stri…

To help your search, this is called “record linkage”. There have been some packages over the years.

I do something similar. I made a “blended” comparison metric fuzzy_strcmp(a, b) = 1 / 10 * (5 * compare(a, b, Levenshtein()) + 3 * compare(a, b, RatcliffObershelp()) + 2 * compare(a, b, DamerauLevenshtein())) with absolutely no rigor put into those coefficients – I just picked from the metrics ava…

For this specific example or not very different situations perhaps we can do without bothering specific packages for textual analysis. It might be enough to do something like this and adapt it to the case with the use of other basic functions. using IterTools st=sort(union(list1, list2)) dt=partit…

I looked for record linkage packages, but I believe those things require a number of fields to measure over right? I did not seem to find something that would match just on names, but I might not have been looking in the right place. But I will keep looking. Some of the Tree stuff seems interesting,…

Ahh, this is helpful. Yeah, I can see what you mean. I imagine I would need to normalize over the scores too, since they might be different scales. but this gives me a sense of how to do it. Thanks for the info.

This is very helpful, thank you. Yeah, this is good practical info. I have been working on separating last names and first names, and removing extra punctuation and such. I have never used those Tree structures before, so I need to research that some more. That is great info on how to pick the metri…

Approximate string matching for two lists of names in Julia

General Usage

rafael.guerra April 3, 2024, 7:04pm 4

Linking related thread

Topic		Replies	Views
What's the best way to compare items in an array to every other item? General Usage performance , strings , arrays	11	3694	July 26, 2019
Similar strings New to Julia strings	5	669	January 13, 2022
Why do these two functions benchmark the same? Performance	2	1365	August 3, 2018
Entity resolution/duplicate data in Julia General Usage	10	1765	June 8, 2023
Jaccard distance of each combo of elements in 2 lists General Usage strings	2	446	November 2, 2020

Approximate string matching for two lists of names in Julia

Related topics