Skip to content

An improved method of locality-sensitive hashing for scalable instance matching. In this study, we propose a scalable approach for automatically identifying similar candidate instance pairs in very large datasets utilizing minhash-lsh-algorithm in C#.

Notifications You must be signed in to change notification settings

mehmetaydar/LinstaMatch-Csharp

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LinstaMatch-Csharp

An improved method of locality-sensitive hashing for scalable instance matching. In this study, we propose a scalable approach for automatically identifying similar candidate instance pairs in very large datasets. Efficient candidate pair generation is an essential to many computational problems involving calculation of instance similarities. Calculating similarities of instances with a large number of properties and efficiently matching a large number of similar instances in a scalable way are two significant bottlenecks of candidate instance pair generation. In our approach, we utilize locality-sensitive hashing (LSH) technique to greatly improve the scalability of candidate instance pair generation. Based on the candidate similarity threshold, our algorithm automatically discovers the optimum number of hash functions in each band in LSH. Moreover, we evaluated the scalability of our approach and its effectiveness in instance matching task using real-world very large datasets.

This is a research project. Technical details can be found in the article: https://link.springer.com/article/10.1007/s10115-018-1199-5

If you use this work, please cite the following research paper: Aydar, M. & Ayvaz, S. Knowl Inf Syst (2019) 58: 275. https://doi.org/10.1007/s10115-018-1199-5

About

An improved method of locality-sensitive hashing for scalable instance matching. In this study, we propose a scalable approach for automatically identifying similar candidate instance pairs in very large datasets utilizing minhash-lsh-algorithm in C#.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages