Calibration and Data Quality Assessment

The sensors have been field tested to evaluate reliability and calibration over a variety of atmospheric conditions. Before field testing, the sensors were calibrated in a controlled environment. The temperature, pressure, and relative humidity (RH) sensor calibration test results were all within the manufacturer’s specifications in the laboratory. The tipping bucket rain gauge had less than 5% error for simulated rainfall rates of 0.1 to 30 mm/h. The 3D-PAWS 3-cup wind speed anemometer and wind vane sensors were tested and calibrated in the NOAA wind tunnel at the NOAA Testbed facility. Stress testing was conducted to maximum speeds of 70 m/s. The 3-cup anemometer performed very well at “hurricane-like” wind speeds. The 3D-PAWS wind vane was also tested over the same range of wind speeds. The tests indicated the wind direction measurements were consistent through the entire range of wind speeds.

Field evaluation of sensors was conducted at the NCAR Marshall Research Facility in Boulder, CO and at the NOAA Testbed Center in Sterling, VA. Sensor observations were compared with calibrated commercial reference sensors. Table 2 shows the reference sensors that were used in the evaluation. For the analysis, reference observations were matched at 1-min resolution to compute error estimates of the 3D-PAWS sensors. Observations were evaluated for the period: June 2016 – March 2017. Evaluation periods varied depending on data availability of each sensor. Comparisons were made to determine any environmental dependencies on the results. For example, the sensors were stratified by day/night and season (e.g., warm season/cold season) to evaluate possible dependencies on measure error.

Table 2: Reference Sensors used in the evaluation.

NCAR Testbed	NOAA Testbed
Temperature: Campbell Scientific 500 series sensor Pressure: Vaisala PTB101B Humidity: Campbell Scientific 500 series sensor Wind speed: RM Young 05108 anemometer Wind Direction: RM Young 05108 anemometer Precipitation: Geonor T-200 weighing precipitation gauge	Temperature: Technical Services Laboratory 1088 hygrothermometer Pressure: Coastal Environmental Systems precision digital barometer PDB-1 Humidity: Technical Services Laboratory 1088 hygrothermometer Wind speed: Vaisala, Inc. 425NWS ice free wind sensor Wind Direction: Vaisala, Inc. 425NWS ice free wind sensor Precipitation: OTT AWPAG weighing precipitation gauge

Temperature observations were evaluated for both the NCAR Marshall Research Facility and the NOAA Testbed Site. Both sites showed similar errors in measurements. The results from the NCAR Marshall Research Facility for all observations, daytime observations and nighttime observations are shown (Fig. 3). The period of study was from June 2016 to March 2017. There are three temperature sensors (BMP180, HTU21D, and MCP9808) integrated on the 3D-PAWS system. All three sensors agree well with the calibrated reference sensor. The error for all measurements is ±0.57° C. The error is slightly larger for daytime observations and lower during nighttime. This is expected due to solar heating on the radiation shield during the day. Overall, the 3D-PAWS temperature sensors have good performance over the range of values (-25° to 37° C) observed in Colorado.

Figure 3. Temperature sensor evaluation. From left to right: All data, daytime observations, nighttime observations.

The barometric station pressure was evaluated at both sites with similar performance being observed. The observations from the NCAR Marshall Research Facility are shown as reference (Fig. 4). The 3D-PAWS pressure sensor (BMP180) is well-behaved. The observed error for all measurements during the June 2016 to March 2017 period was ±0.49 hPa. Observations stratified by time of day show slightly greater error (±0.53 hPa) during the day and less during the night (±0.37 hPa). This is likely due to the heating and larger temperature variations observed during daytime hours.

Figure 4. Atmospheric pressure sensor evaluation. From left to right: All data, daytime observations, nighttime observations.

The relative humidity sensors were evaluated at both the NCAR Marshall Research Facility and at the NOAA Testbed Site (Fig. 5). Both sites observed similar behavior in sensor performance (the Marshall site results are shown below). The comparison for all measurements matched with the reference sensor for the period June 2016 – March 2017 is shown in the left panel. The mean error from the analysis ±5.7%. However, there is large scatter in the mid-humidity ranges (20-80%) and an observed low-bias at low (< 10%) and high (> 90%) relative humidities. Nighttime observations showed slightly less error (±4.87%) compared to daytime measurements (±5.98%). However, the shape of the scatterplot and errors are nearly the same. The plot shows an occasional odd behavior with either the 3D-PAWS and/or reference sensor, likely due to condensation on the sensor.

Figure 5. Relative humidity sensor evaluation. From left to right: All data, daytime observations, nighttime observations.

Wind speed and wind direction observations performance were evaluated (not shown). For wind speed, the measurement error was computed to be ±0.87 m/s. The nighttime observations had observation errors (±0.79 m/s) slightly less than daytime errors (±0.94 m/s). The results show that the 3D-PAWS sensor was in agreement with the reference sensor with the predominant wind observed. The measured error was less than ± 5°.

Rainfall from the 3D-PAWS tipping bucket rain gauge was compared to the NOAA Testbed weighing precipitation gauge (Fig. 6). Precipitation was compared for the period of October 2016 to March 2017. A total of ~200 mm of rainfall was recorded at the site. There is good agreement between the gauges in total accumulation. There are small differences observed for individual precipitation events. These differences are likely attributed to wind and precipitation rate errors.

Figure 6. Rainfall accumulation observed by the 3D-PAWS tipping bucket rain gauge and the NOAA weighing gauge reference sensor.

Table 4 provides a summary of sensor measurement characteristics and performance during the evaluation. Overall, the sensors compared well with calibrated reference sensors. The only exception is the performance of the relative humidity sensor which has a bias at high and low humidities and larger then expected errors in the mid-range. A new relative humidity sensor is being evaluated.

Table 4: Summary of sensor evaluation.

Parameter	Resolution	Uncertainty
Temperature (°C)	0.1 °C	±0.4 °C
Pressure (hPa)	0.1 hPa	±0.4 hPa
Relative Humidity (%)	1 %	±5.7 %
Wind Speed (m/s)	0.1 m/s	±0.8 m/s
Wind Direction (deg)	1 deg	±5 deg
Rainfall (mm)	0.2 mm	10%