Publication Details
Heterogeneity-Aware Scheduler for Stream Processing Frameworks
scheduling; resource awareness; benchmarking; stream processing; Apache Storm;
heterogeneous clusters; heterogeneity awareness; resource allocation
This article discusses problems and decisions related to scheduling of stream
processing applications in heterogeneous clusters. An overview of the current
state of the art of the stream processing on heterogeneous clusters with a focus
on resource allocation and scheduling is presented first. Then, common scheduling
approaches of various stream processing frameworks are discussed and their
limited applicability in the heterogeneous environment is demonstrated on
a simple stream application. Finally, the article presents a novel
heterogeneity-aware scheduler for the stream processing frameworks based on
design-time knowledge as well as benchmarking techniques. It is shown that the
scheduler overcomes alternatives in resource-aware deployment over cluster nodes
and thus it leads to a better utilisation of the clusters.
@article{BUT119792,
author="Marek {Rychlý} and Petr {Škoda} and Pavel {Smrž}",
title="Heterogeneity-Aware Scheduler for Stream Processing Frameworks",
journal="International Journal of Big Data Intelligence",
year="2015",
volume="2",
number="2",
pages="70--80",
doi="10.1504/IJBDI.2015.069090",
issn="2053-1397",
url="http://www.inderscience.com/info/inarticle.php?artid=69090"
}