Publication Details

Low Overhead Distributed IP Flow Records Collection and Analysis

WRONA, J.; ŽÁDNÍK, M. Low Overhead Distributed IP Flow Records Collection and Analysis. In 2019 IFIP/IEEE International Symposium on Integrated Network Management. Washington DC: 2019. p. 557-562. ISBN: 978-3-903176-15-7.
Czech title
Distribuovaný sběr a analýza záznamů o IP tocích s nízkou režíí
Type
conference paper
Language
English
Authors
Wrona Jan, Ing.
Žádník Martin, Ing., Ph.D. (DCSY)
URL
Keywords

NetFlow, IPFIX, IP flow collector, distributed system, parallel computing, Hadoop, big data

Abstract

Collection and analysis of IP flow records belong to a class of data-intensive tasks, the class for which big data analytics systems should be effective. Several Hadoop-based solutions for network traffic processing exist but are generally suitable only for truly big data, otherwise the disadvantages of Hadoop dominate. In this work, we present a distributed platform for IP flow records collection and analysis together with a reference implementation. It focuses on smaller clusters, has low overhead, allows interactive work, and exploits the prospects of distributed systems like high throughput and scalability. Experiments show low query latency and linear scalability with respect to the growth of both amount of work and computer cluster. Extensions for data mining and machine learning are easy to include and are already work in progress. Moreover, the whole software stack is open-source.

Published
2019
Pages
557–562
Proceedings
2019 IFIP/IEEE International Symposium on Integrated Network Management
ISBN
978-3-903176-15-7
Place
Washington DC
UT WoS
000469937200098
EID Scopus
BibTeX
@inproceedings{BUT161793,
  author="Jan {Wrona} and Martin {Žádník}",
  title="Low Overhead Distributed IP Flow Records Collection and Analysis",
  booktitle="2019 IFIP/IEEE International Symposium on Integrated Network Management",
  year="2019",
  pages="557--562",
  address="Washington DC",
  isbn="978-3-903176-15-7",
  url="https://ieeexplore.ieee.org/document/8717873"
}
Back to top