Greenbyte Documentation

Data cleaning

Time series data used in Greenbyte goes through a data cleaning process to automatically filter out invalid data. There are a few different filtering mechanics.

Note

  • All the raw data is stored in Greenbyte. The filtering is only for the purpose of presentation and calculations.

  • Configured alerts will always analyze raw data without any cleaning applied.

Threshold filtering

Time series data that falls outside of specific thresholds is filtered out and discarded.

The thresholds are set on a by-signal basis, and only a few signals have thresholds set. These thresholds are quite conservative and will only filter out clearly invalid data.

To see which thresholds apply, go to Edit Device > Communication.

data_cleaning_device_ui.png

Here is a summary of thresholds currently applied.

Data Signal

Min

Max

Irradiance

0.1

1500

Wind speed

0

120

Rotor speed

0

50

Ambient temperature

-120

120

Power

-50 for inverters

-100 for turbines

100000

Module temperature

-50

50

Nighttime filtering

For irradiance there should never be any data a night, but it is common that we get noisy data or spikes during night time. To solve this issue we filter out all values during nighttime. Nighttime is determined using the device coordinates to determine sunrise and sunset time.

Flatline filtering

Flatline filtering is applied to the resource data in both wind and solar.

Implement flatline filtering of values => 0 and threshold 0.001 for 6 consecutive values for all DataSignalTypeId = 1Calculated signals should not be filtered

  • For wind speed data we filter out values greater than 0 if 6 or more consecutive values stay within 0.001 m/s of each other.

  • For irradiance data we filter out values if 4 or more consecutive values stay within 0.01 W/m2 of each other.

Raw data

To see the raw data without data cleaning, go to Data Studio and select Options > Filters > Show raw data.

data_cleaning_ui.png