# Anomaly detection

The goal is to detect anomalies on the basis of given difference of quantity values measured in two different time points. One time point is current time and the second one is selected from data that has been recently measured.

The configuration parameters in the examples below can be changed in the configuration file `config.ini` that is described in section Configuration file.

## CO2

Detection of anomalies is based on difference in carbon dioxide concentration values measured in two various time points.

    """Anomaly detection - CO2.
    """
    from os.path import dirname, abspath, join
    import sys
    sys.path.append(abspath(join(dirname(__file__), '../..', '')))

    from dm.ConnectionUtil import ConnectionUtil as cu
    from dm.models.ModelsUtil import ModelsUtil
    from dm.WundergroundCom import WundergroundCom
    from dm.DateTimeUtil import DateTimeUtil

    if __name__ == '__main__':
        cu.setup_logging()
        cls = cu.setup_clients()

        actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
        lat = 49.1649894
        lon = 16.562262499999974

        w = WundergroundCom()
        w.api_key = cu.wunderground_api_key()

        devs = [
            {
                'db_column_name': 'co2_in_ppm',
                'name': '',
                'gateway_id': '1816820318180747',
                'device_id': '0xa900811026800001',
                'module_id': 2,
                'server_name': 'ant-work',
            }
        ]
        notification = ModelsUtil.anomaly_diff(devs, cls, lat, lon, actual_time, w, 2, 7200, 'co2')
        ModelsUtil.json_to_file(notification, 'co2_notification.doc.json', log_notification=True)

It is necessary to set clients that can communicate with a remote server using the function `setup_clients`. Current time is stored in the variable `actual_time`. Latitude and longitude that enable to get weather data according to a sensor position are stored in the variable `lat` and `lon`. The API key allows to get weather data from server and it is obtained by the function `wunderground_api_key`. The sensors used to gather data are stored in the variable `devs`. Anomaly detection is performed by the function `anomaly_diff` and the result is stored in the variable `notification`.

An output of the detection is a notification that contains basic information described in the section Notifications and the following information:

- actual_diff - current difference in carbon dioxide concentration values,

- anomaly - if anomaly is detected,

- min_anomaly_diff - minimal difference in carbon dioxide concentration values to detect anomaly,

- min_anomaly_time - time interval between two time points,

- type - what quantity is used for prediction.

The example of a notification follows.

    {
        "data": {
            "actual_diff": 0.0,
            "anomaly": false,
            "min_anomaly_diff": 2,
            "min_anomaly_time": 120,
            "type": "anomaly_co2"
        },
        "device_id": "0xa900811026800001",
        "event": "env-notification-pre",
        "gateway_id": "1816820318180747",
        "raise": false,
        "raise_catch": false,
        "readable": "2019-02-20 03:00:00",
        "timestamp": 1550628000
    }

The example of a command that can be used to detect anomalies on the basis of carbon dioxide concentration difference.

    python examples2/0305_anomaly_detection/co2_run.py

## Temperature

Detection of anomalies is based on difference in temperature values measured in two various time points.

    """Anomaly detection - temperature.
    """
    from os.path import dirname, abspath, join
    import sys
    sys.path.append(abspath(join(dirname(__file__), '../..', '')))

    from dm.ConnectionUtil import ConnectionUtil as cu
    from dm.models.ModelsUtil import ModelsUtil
    from dm.WundergroundCom import WundergroundCom
    from dm.DateTimeUtil import DateTimeUtil

    if __name__ == '__main__':
        cu.setup_logging()
        cls = cu.setup_clients()

        actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
        lat = 49.1649894
        lon = 16.562262499999974

        w = WundergroundCom()
        w.api_key = cu.wunderground_api_key()

        devs = [
            {
                'db_column_name': 'temperature_in_celsius',
                'name': 'BeeeOn sensor',
                'gateway_id': '1816820318180747',
                'device_id': '0xa900811026800001',
                'module_id': 0,
                'server_name': 'ant-work',
            }
        ]
        notification = ModelsUtil.anomaly_diff(devs, cls, lat, lon, actual_time, w, 2, 7200, 'temperature')
        ModelsUtil.json_to_file(notification, 't_notification.doc.json', log_notification=True)

It is necessary to set clients that can communicate with a remote server using the function `setup_clients`. Current time is stored in the variable `actual_time`. Latitude and longitude that enable to get weather data according to a sensor position are stored in the variable `lat` and `lon`. The API key allows to get weather data from server and it is obtained by the function `wunderground_api_key`. The sensors used to gather data are stored in the variable `devs`. Anomaly detection is performed by the function `anomaly_diff` and the result is stored in the variable `notification`.

An output of the detection is a notification that contains basic information described in the section Notifications and the following information:

- actual_diff - current difference in temperature values,

- anomaly - if anomaly is detected,

- min_anomaly_diff - minimal difference in temperature values to detect anomaly,

- min_anomaly_time - time interval between two time points,

- type - what quantity is used for prediction.

The example of a notification follows.

    {
        "data": {
            "actual_diff": 0.22,
            "anomaly": false,
            "min_anomaly_diff": 2,
            "min_anomaly_time": 120,
            "type": "anomaly_temperature"
        },
        "device_id": "0xa900811026800001",
        "event": "env-notification-pre",
        "gateway_id": "1816820318180747",
        "raise": false,
        "raise_catch": false,
        "readable": "2019-02-20 03:00:00",
        "timestamp": 1550628000
    }

The example of a command that can be used to detect anomalies on the basis of temperature difference.

    python examples2/0305_anomaly_detection/t_run.py

## Humidity

Detection of anomalies is based on difference in humidity values measured in two various time points.

    """Anomaly detection - humidity.
    """
    from os.path import dirname, abspath, join
    import sys
    sys.path.append(abspath(join(dirname(__file__), '../..', '')))

    from dm.ConnectionUtil import ConnectionUtil as cu
    from dm.models.ModelsUtil import ModelsUtil
    from dm.WundergroundCom import WundergroundCom
    from dm.DateTimeUtil import DateTimeUtil

    if __name__ == '__main__':
        cu.setup_logging()
        cls = cu.setup_clients()

        actual_time = int(DateTimeUtil.local_time_str_to_utc('2019/02/20 03:00:00').timestamp())
        lat = 49.1649894
        lon = 16.562262499999974

        w = WundergroundCom()
        w.api_key = cu.wunderground_api_key()

        devs = [
            {
                'db_column_name': 'rh_in_percentage',
                'name': 'BeeeOn sensor',
                'gateway_id': '1816820318180747',
                'device_id': '0xa900811026800001',
                'module_id': 1,
                'server_name': 'ant-work',
            }
        ]
        notification = ModelsUtil.anomaly_diff(devs, cls, lat, lon, actual_time, w, 2, 7200, 'humidity')
        ModelsUtil.json_to_file(notification, 'h_notification.doc.json', log_notification=True)

It is necessary to set clients that can communicate with a remote server using the function `setup_clients`. Current time is stored in the variable `actual_time`. Latitude and longitude that enable to get weather data according to a sensor position are stored in the variable `lat` and `lon`. The API key allows to get weather data from server and it is obtained by the function `wunderground_api_key`. The sensors used to gather data are stored in the variable `devs`. Anomaly detection is performed by the function `anomaly_diff` and the result is stored in the variable `notification`.

An output of the detection is a notification that contains basic information described in the section Notifications and the following information:

- actual_diff - current difference in humidity values,

- anomaly - if anomaly is detected,

- min_anomaly_diff - minimal difference in humidity values to detect anomaly,

- min_anomaly_time - time interval between two time points,

- type - what quantity is used for prediction.

The example of a notification follows.

    {
        "data": {
            "actual_diff": 1.5,
            "anomaly": false,
            "min_anomaly_diff": 2,
            "min_anomaly_time": 120,
            "type": "anomaly_humidity"
        },
        "device_id": "0xa900811026800001",
        "event": "env-notification-pre",
        "gateway_id": "1816820318180747",
        "raise": false,
        "raise_catch": false,
        "readable": "2019-02-20 03:00:00",
        "timestamp": 1550628000
    }

The example of a command that can be used to detect anomalies on the basis of humidity difference.

    python examples2/0305_anomaly_detection/h_run.py