Bug report: Null values

[cyounkins]

Hello and thank you for providing this API. I have an issue - I'm intermittently seeing null in temperature, pressure, and windspeed fields for my local station.

My script polls https://api.weather.gov/stations/KBWI/observations/latest and logs the current time and values from the JSON response: id, properties.temperature.value, properties.barometricPressure.value, properties.windSpeed.value. Here is a log that shows the issue: https://gist.github.com/cyounkins/63e985be2525eb3f8c8cafd527464170

Take the first response with nulls:

Tue Oct 26 2021 02:38:37 GMT+0000 (Coordinated Universal Time)
https://api.weather.gov/stations/KBWI/observations/2021-10-26T02:16:00+00:00
{
  temperature_c: null,
  barometricPressure_pa: null,
  'windspeed_km_h-1': null
}

At 2:38 GMT a request was made to /stations/KBWI/observations/latest, and the response had an id of https://api.weather.gov/stations/KBWI/observations/2021-10-26T02:16:00+00:00 and the null values.

I have confirmed this issue can be seen outside my script in Firefox at both /latest and /<date> endpoints. That is, accessing the /<date> endpoint also shows nulls. Here is a curl of a different date showing the issue: https://gist.github.com/cyounkins/50eebdfbfefd98c8e70277ff5a00995a

At some point later (possibly with the addition of new data), /latest and /<date> endpoints will be fixed. So now if you go to the id URL given in the sample above, it will have data that didn't appear in the log.

As a user, I would much rather have no update provided than have an update with these critical fields be null.

Knowing nothing about your architecture, some very shot-in-the-dark ideas on what could be causing this:

• NULL values getting into the database -> filter data from stations
• Race condition where a new ID is inserted before the temperature, a bad response is generated and then stuck in cache

Again, thanks for your work on this free service.

[cyounkins]

As expected, the curl example endpoint (second gist) now returns data with temperature, wind speed, etc. I note that the rawMessage component did not change.

[researchgrant]

I'm also getting null values when pulling from https://api.weather.gov/stations/KBOS/observations/. I've tried using a time range, or pulling just a single value and I get the same nulls. I'm mostly looking for precipitation data, which is primarily nulls. The html site where the 3 day history can be found is working properly: https://forecast.weather.gov/data/obhistory/KBOS.html. So, it appears that the data were collected but they are just not accessible by the api.

[AndrewBarfield]

Came here to report the same issue for :

https://api.weather.gov/stations/KEYF/observations/latest

https://api.weather.gov/stations/KILM/observations/latest

My request rate is once per hour.

The API is very unstable/unreliable.

[Sudo-Rob]

I'm using KBWI data, and although current conditions seem to be reliable, it is not reliably providing 5-day forecast information as part of the transaction. Is this expected behavior?

[rr0ss0rr]

Just started using the api to create a humidity set point on my ecobee. I was surprised when I looked at the log and saw the outside temp (F) was 32 (It's way colder). Looks like it occurred multiple times last night and again this morning. I did add error checking on the observance of ".status" in the json response, but that wasn't the case so I saved the json response when this occurs.

Using KROC/observations/latest and sample below

json.properties.temperature = {}; json.properties.temperature.qualityControl = "Z"; json.properties.temperature.unitCode = "wmoUnit:degC"; json.properties.temperature.value = null;

for me, I can just ignore the response, but it would be nice if errors were standardized so there is one place to check if the response is valid.

[cyounkins]

I suspect you figured this out, but just in case you didn't - Your log likely indicated 32F because the null was treated as 0 and was converted to Fahrenheit.

For example, in nodejs:

> (null * 1.8) + 32
32

I suggest explicitly verifying that temperature.value !== null before using the value.

The errors in an HTTP API should usually use HTTP status codes. For this case, I'd really prefer to continue receiving old data instead of new data with null temperature.

[rr0ss0rr]

Thanks .. I added a validity check before any processing occurs. Prior to this I was only checking the json ".status" key.