Publication Details

Scraping Data from Web Pages using SPARQL Queries

BURGET, R. Scraping Data from Web Pages using SPARQL Queries. Web Engineering - 23rd International Conference, ICWE 2023. Lecture Notes in Computer Science. Alicante: Springer Nature Switzerland AG, 2023. p. 293-300. ISBN: 978-3-031-34443-5.
Czech title
Získávání dat z webových stránek pomocí dotazů v jazyce SPARQL
Type
conference paper
Language
English
Authors
URL
Keywords

Web scraping, Page rendering, Data extraction, RDF, SPARQL

Abstract

Despite the increasing use of semantic data, plain old HTML web pages often provide a unique interface for accessing data from many domains. To use this data in computer applications or to integrate it with other data sources, it must be extracted from the HTML code. Currently, this is typically done by single-purpose programs called scrapers. For each data source, specific scrapers must be created, which requires a thorough analysis of the source page's implementation in HTML. This makes writing and maintaining a set of scrapers a complex and time-consuming task. In this paper, we present an alternative approach that allows defining scrapers based on visual properties of the presented content instead of the HTML code structure. First, we render the source page and create an RDF graph that describes the visual properties of every piece of the displayed content. Next, we use SPARQL to query the model and extract the data. As we demonstrate with real-world examples, this approach allows us to easily define more robust scrapers that can be used across multiple web sites and that that better cope with changes in the source documents.

Published
2023
Pages
293–300
Proceedings
Web Engineering - 23rd International Conference, ICWE 2023
Series
Lecture Notes in Computer Science
ISBN
978-3-031-34443-5
Publisher
Springer Nature Switzerland AG
Place
Alicante
DOI
BibTeX
@inproceedings{BUT183806,
  author="Radek {Burget}",
  title="Scraping Data from Web Pages using SPARQL Queries",
  booktitle="Web Engineering - 23rd International Conference, ICWE 2023",
  year="2023",
  series="Lecture Notes in Computer Science",
  pages="293--300",
  publisher="Springer Nature Switzerland AG",
  address="Alicante",
  doi="10.1007/978-3-031-34444-2\{_}21",
  isbn="978-3-031-34443-5",
  url="https://link.springer.com/chapter/10.1007/978-3-031-34444-2_21"
}
Back to top