Machine Learning in Context of IoT/Edge Devices and LoLiPoP-IoT Project

Machine learning models are traditionally deployed in the cloud or on centralized
servers to leverage their computing resources. However, such a deployment may
reduce privacy, introduce extra latency, consume more power, etc., and
subsequently negatively impact properties of an application that typically runs
on a battery-operated device used to communicate via a wireless network. To
minimize the negative impact, it is necessary to deploy a model directly to such
a device to minimize data transfer energy and run the model closer to the data
source and, application and its environment. However, this kind of deployment is
a  challenging task due to the very limited resources available in such devices
and applications. Many people and companies have tackled this challenging problem
and proposed different ways and means to solve it. Having defined the problem and
our area of interest, the paper provides an overview of representative
applications, methods and means, including libraries, frameworks, datasets,
devices etc. It then presents a  typical deployment process workflow in the
context of resource-constrained devices. Finally, it sums representative results
for popular resource-constrained devices (e.g., Arduino, ARM Cortex-M, ESP32,
nRF5x, Nvidia Jetson, Raspberry Pi) to demonstrate how various phenomena (e.g.,
model type, setting, quantization) affect model performance (e.g., accuracy,
loss), metrics (e.g., ROC AUC, F1 scores) and device performance (e.g., feature
and inference processing time, memory usage).