On the database lookup problem of approximate matching

Details

Serval ID
serval:BIB_92345A1051A8
Type
Article: article from journal or magazin.
Collection
Publications
Title
On the database lookup problem of approximate matching
Journal
Digital Investigation
Author(s)
Breitinger Frank, Baier Harald, White Douglas
ISSN
1742-2876
Publication state
Published
Issued date
05/2014
Volume
11
Number
0
Pages
S1-S9
Language
english
Abstract
Abstract Investigating seized devices within digital forensics gets more and more difficult due to the increasing amount of data. Hence, a common procedure uses automated file identification which reduces the amount of data an investigator has to look at by hand. Besides identifying exact duplicates, which is mostly solved using cryptographic hash functions, it is also helpful to detect similar data by applying approximate matching. Let x denote the number of digests in a database, then the lookup for a single similarity digest has the complexity of O(x). In other words, the digest has to be compared against all digests in the database. In contrast, cryptographic hash values are stored within binary trees or hash tables and hence the lookup complexity of a single digest is O(log2(x)) or O(1), respectively. In this paper we present and evaluate a concept to extend existing approximate matching algorithms, which reduces the lookup complexity from O(x) to O(1). Therefore, instead of using multiple small Bloom filters (which is the common procedure), we demonstrate that a single, huge Bloom filter has a far better performance. Our evaluation demonstrates that current approximate matching algorithms are too slow (e.g., over 21 min to compare 4457 digests of a common file corpus against each other) while the improved version solves this challenge within seconds. Studying the precision and recall rates shows that our approach works as reliably as the original implementations. We obtain this benefit by accuracy–the comparison is now a file-against-set comparison and thus it is not possible to see which file in the database is matched.
Keywords
Law, Medical Laboratory Technology, Computer Science Applications
Web of science
Open Access
Yes
Create date
06/05/2021 12:01
Last modification date
06/05/2021 12:21
Usage data