A Probabilistic Address Parser Using Conditional Random Fields and Stochastic Regular Grammar

Tools

Wang, Minlue; Haberland, Valeriia; Yeo, Amos; Martin, Andrew; Howroyd, John and Bishop, Mark. 2016. 'A Probabilistic Address Parser Using Conditional Random Fields and Stochastic Regular Grammar'. In: IEEE 2016 International Conference on Data Mining. Barcelona, Spain Dec 12 - 15, 2016. [Conference or Workshop Item]

Preview

Text
Minlue_Wang_at al_camera_ready.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.
Download (182kB) | Preview

Official URL: http://ieeexplore.ieee.org/document/7836670/

Abstract or Description

Automatic semantic annotation of data from databases or the web is an important pre-process for data cleansing and record linkage. It can be used to resolve the problem of imperfect field alignment in a database or identify comparable fields for matching records from multiple sources. The annotation process is not trivial because data values may be noisy, such as abbreviations, variations or misspellings. In particular, overlapping features usually exist in a lexicon-based approach. In this work, we present a probabilistic address parser based on linear-chain conditional random fields (CRFs), which allow more expressive token-level features compared to hidden Markov models (HMMs). In additions, we also proposed two general enhancement techniques to improve the performance. One is taking original semi-structure of the data into account. Another is post-processing of the output sequences of the parser by combining its conditional probability and a score function, which is based on a learned stochastic regular grammar (SRG) that captures segment-level dependencies. Experiments were conducted by comparing the CRF parser to a HMM parser and a semi-Markov CRF parser in two real-world datasets. The CRF parser out-performed the HMM parser and the semiMarkov CRF in both datasets in terms of classification accuracy. Leveraging the structure of the data and combining the linear chain CRF with the SRG further improved the parser to achieve an accuracy of 97% on a postal dataset and 96% on a company dataset.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):

https://doi.org/10.1109/ICDMW.2016.0039