LEDNeCo: Low Energy Deep Neurocomputing

deep neural networks;tranformers;energy complexity;hardware accelerator;approximation theory;robust learning;genetic programming

Modern artificial intelligence technologies based on deep neural networks (DNNs)
such as GPT are computationally extremely demanding. In addition to consuming an
enormous amount of energy, this limits their deployment in battery-powered
embedded (edge) devices (e.g. smart mobile apps). LEDNeCo is a project of basic
research whose ambition is to develop a low-energy deep neurocomputing paradigm
based on machine-independent energy complexity theory for DNNs, which issues from
practical experience in the design of diverse DNN hardware accelerators. Among
other things, universal lower bounds on energy complexity of DNNs and estimates
of inference error will be derived for identifying DNN components (e.g. weights,
neurons, layers) whose approximation is provably the most energy efficient. The
achieved theoretical knowledge will be used in new advanced approximation
techniques (e.g. weight compression, Boolean optimization, robust aproximation of
components) for low-power hardware implementations of DNN (incl. transformer),
which will be tested on benchmark datasets.