Greenbyte Documentation

Data cleaning

The time-series data used in Greenbyte goes through a data cleaning process to automatically filter out invalid data. There are a few different filtering mechanisms:

Note

  • All the raw data is stored in Greenbyte. The filtering is only for presentation and calculations.

  • Configured alerts will always analyze raw data without any cleaning applied.

Threshold filtering

Time series data that falls outside of specific thresholds is filtered out and discarded.

The thresholds are set on a by-signal basis. Only a few signals have thresholds set. These thresholds are quite conservative and will only filter out the invalid data that is obvious.

To see which thresholds apply, go to Edit Device > Communication.

data_cleaning_device_ui.png

Here is a summary of the thresholds currently applied.

Data Signal

Min

Max

Irradiance

0.1

1500

Wind speed

0

120

Rotor speed

0

50

Ambient temperature

-120

120

Power

-50 for inverters

-100 for turbines

100000

Module temperature

-50

50

Nighttime filtering

For irradiance, there should never be any data at night. But commonly, we get noisy data or spikes during nighttime. To solve this issue, we filter out all values during nighttime. Nighttime is determined using the device coordinates to determine sunrise and sunset time.

Flatline filtering

Flatline filtering is applied to the resource data in both wind and solar.

Implement flatline filtering of values => 0 and threshold 0.001 for 6 consecutive values for all DataSignalTypeId = 1Calculated signals should not be filtered.

  • For wind speed data, if 6 or more consecutive values stay within 0.001 m/s of each other, we filter out values greater than 0.

  • For irradiance data, if 4 or more consecutive values stay within 0.01 W/m2 of each other, we filter out values.

Raw data

To see the raw data without data cleaning, go to Data Studio and select Options > Filters > Show raw data.

data_cleaning_ui.png

Counter signal rollover values

We can add rollover values for counter signals. For example, for one of our customers, we have set 10000 MWh as a rollover value for a Grid Meter Energy Export signal.

Note

Currently, this can only be set up in the database through support's help.