Publication Details
Box clustering segmentation: A new method for vision-based web page preprocessing
Burget Radek, doc. Ing., Ph.D. (DIFS)
Zendulka Jaroslav, doc. Ing., CSc. (UIFS)
box clustering, graph clustering, vision-based page segmentation, VIPS
This paper presents a novel approach to web page segmentation, which is one of
substantial preprocessing steps when mining data from web documents. Most of the
current segmentation methods are based on algorithms that work on a tree
representation of web pages (DOM tree or a hierarchical rendering model) and
produce another tree structure as an output. In contrast, our method uses
a rendering engine to get an image of the web page, takes the smallest rendered
elements of that image, performs clustering using a custom algorithm and produces
a flat set of segments of a given granularity. For the clustering metrics, we use
purely visual properties only: the distance of elements and their visual
similarity. We experimentally evaluate the properties of our algorithm by
processing 2400 web pages. On this set of web pages, we prove that our algorithm
is almost 90% faster than the reference algorithm. We also show that our
algorithm accuracy is between 47% and 133% of the reference algorithm accuracy
with indirect correlation of our algorithms accuracy to the depth of inspected
page structure. In our experiments, we also demonstrate the advantages of
producing a flat segmentation structure instead of an hierarchy.
@article{BUT133487,
author="Jan {Zelený} and Radek {Burget} and Jaroslav {Zendulka}",
title="Box clustering segmentation: A new method for vision-based web page preprocessing",
journal="INFORMATION PROCESSING & MANAGEMENT",
year="2017",
volume="53",
number="3",
pages="735--750",
doi="10.1016/j.ipm.2017.02.002",
issn="0306-4573",
url="http://www.sciencedirect.com/science/article/pii/S0306457316301169"
}