D-HAT: unsupervised matching of heterogeneous records
An entity matching method for records that do not line up neatly, with no labelled data needed.
- Setting
- Research at LIRIS, Université Claude Bernard Lyon 1
- Published
- ADMA 2022; EGC 2023 (French version)
- Results
- 30% higher F1 than the leading unsupervised method, in 1 minute instead of 6 hours; 10th on the Amazon-Google leaderboard, above GPT-4o and Llama-3.1-70B, without fine-tuning
The problem
Records from different sources rarely line up: fields are formatted differently, values are missing, and labelled examples to learn from are rarely available. D-HAT matches such records without labelled data.
Results
30% higher F1 than the leading unsupervised method, in 1 minute instead of 6 hours. On the Amazon-Google benchmark, D-HAT ranks 10th on the public leaderboard, above GPT-4o and Llama-3.1-70B, without fine-tuning.
The same benchmark, hands on
The matching demo uses records from the same benchmark to show the data quality problems and the comparison bottleneck any matcher faces.
Facing a similar problem? Email me.