Publication Details
Scraping Data from Web Pages using SPARQL Queries
Web scraping, Page rendering, Data extraction, RDF, SPARQL
Despite the increasing use of semantic data, plain old HTML web pages often
provide a unique interface for accessing data from many domains. To use this data
in computer applications or to integrate it with other data sources, it must be
extracted from the HTML code. Currently, this is typically done by single-purpose
programs called scrapers. For each data source, specific scrapers must be
created, which requires a thorough analysis of the source page's implementation
in HTML. This makes writing and maintaining a set of scrapers a complex and
time-consuming task. In this paper, we present an alternative approach that
allows defining scrapers based on visual properties of the presented content
instead of the HTML code structure. First, we render the source page and create
an RDF graph that describes the visual properties of every piece of the displayed
content. Next, we use SPARQL to query the model and extract the data. As we
demonstrate with real-world examples, this approach allows us to easily define
more robust scrapers that can be used across multiple web sites and that that
better cope with changes in the source documents.
@inproceedings{BUT183806,
author="Radek {Burget}",
title="Scraping Data from Web Pages using SPARQL Queries",
booktitle="Web Engineering - 23rd International Conference, ICWE 2023",
year="2023",
series="Lecture Notes in Computer Science",
pages="293--300",
publisher="Springer Nature Switzerland AG",
address="Alicante",
doi="10.1007/978-3-031-34444-2\{_}21",
isbn="978-3-031-34443-5",
url="https://link.springer.com/chapter/10.1007/978-3-031-34444-2_21"
}