How can we help?


Weather Underground public station actual rainfall data --- Did we really need the API?

Comments

12 comments

  • Avatar
    RDLinKY

    Also, the log of my HD-16, I see the following request going to wunderground;

    https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID= KKYCREST23&day=16&month=6&year=2019&graphspan=week&format=0&units=metric

    Which show the following error: error: HTTP Error 400: Bad Request


    However when I enter that same url directly into a browser, I get the following which clearly show observed data for the past 3 days;

    Date,TemperatureHighC,TemperatureAvgC,TemperatureLowC,DewpointHighC,DewpointAvgC,DewpointLowC,HumidityHigh,HumidityAvg,HumidityLow,PressureMaxhPa,PressureMinhPa,WindSpeedMaxKMH,WindSpeedAvgKMH,GustSpeedMaxKMH,PrecipitationSumCM
    2019-6-16,28.6,24.0,19.4,20.0,14.6,6.1,89,63,29,1007,1002,20,1,28,3.76
    2019-6-17,28.0,24.3,20.6,20.9,18.0,13.6,96,75,43,1008,1004,13,0,20,1.35
    2019-6-18,22.3,21.9,21.6,20.2,19.9,19.7,89,88,88,1007,1006,6,0,8,0.53

    Probably worth noting that the "KKYCREST2" weather station is not mine. It's just one nearby.

    0
    Comment actions Permalink
  • Avatar
    Justin Rizel

    Maybe the Rainmachine is trying to do what I'm envisioning, but there is a bug or something such that its not getting any data back? Maybe something only the developers can explain? I'll have to look at the logs on my unit when I'm home to see if it has a similar message.

    0
    Comment actions Permalink
  • Avatar
    RDLinKY

    I'm not sure if this is the correct way to ask, but can we get an official response from the support team at RainMachine as to why the observed data seems to be available from WUnderground but it is not working on our RainMachines? Is the only way that I can fix this is by spending a bunch of money on a Weather Station?

    Users need some way of getting observed weather data into their RainMachine device so that the proper calculations are made. After all, that is why we all bought the device. The NOAA forecasting is just not accurate enough. For instance, the other day NOAA forecasted 1.5" of rain for my area... it didn't rain 1 drop.

    RainMachine... Please Help!!

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    The WUnderground service uses the same url that you find out when not configured with a API Key.  It still works on our internal test machines although it's not very reliable. It does seem to get errors from time to time, probably related to WUnderground servers. 

    You can try to REFRESH the weather services by hand and/or select another station to check if you can get data.

    We are also working on getting this observed data from NOAA (which has a good part of stations that WUnderground has) and integrate this in the NOAA weather service.

    0
    Comment actions Permalink
  • Avatar
    Justin Rizel

    That is important into to know, I removed the old API key and saw the same entry as RDLinKY, I added a few options for stations around me, and it seems to be pulling data from one station more consistently. For some reason the station closest to me never returns data even though it all appears there when loading the page in a browser, but the second closest station returns rainfall data so its better than nothing.

    I had also found some beta wu parser code on here but it seems to work the same way as the officially integrated code, from what I can tell at least.

    Keep up the good work, if you want any specifics or more details let me know.

    Thanks
    JR

    0
    Comment actions Permalink
  • Avatar
    RDLinKY

    Any word on the effort to get observed data from NOAA (which has a good part of stations that WUnderground has) and integrate this in the NOAA weather service as mentioned by Nicholas above?

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    Still in the works, there is atm no reliable way to get NOAA observed data from both eastern/western side of US.

    0
    Comment actions Permalink
  • Avatar
    Marc Kessler

    Can you elaborate on what the problem is with getting NOAA observed data?

    From https://w1.weather.gov/xml/current_obs/index.xml there is a like for the weather stations to html_url which has the observation history.  The html links include 1/3/6 hr precipitation data.  Is there a problem with that field?

    IE:  https://w1.weather.gov/data/obhistory/KMYF.html

    0
    Comment actions Permalink
  • Avatar
    Marc Kessler

    I found some spare time and made a quick parser for NOAA observed... may/maynot work for others, but seems to have good results from today's rain.

    https://github.com/mycon/rainmachine-developer-resources/blob/master/sdk-parsers/RMParserFramework/parsers/noaa-observed-parser.py

     

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    Great addition thank you ! We are trying to stay away from HTML parsing and use CDC API data which at this moment doesn't send precipitation data.

    I haven't run the parser yet but I would recommend saving just 1 value per day with timestamp set to day timestamp at 00:00. This would prevent issues if the hourly intervals change during successive runs.

    0
    Comment actions Permalink
  • Avatar
    Marc Kessler

    Understandable you want to avoid scraping HTML :)

    I noticed in the "Observed Rain Amount" in the android app after "Show Logs" on the Dashboard Weather pane seems to pull the value for only the last value added for that day rather than an accumulation over the full day.

    Can you elaborate on the expectation for how observed data should be entered into the RM database?  Will the actual value for observed rain that gets used be an accumulation over the entire day?  From a few other posts it sounds like it should be, but most parser only add a single daily value for "yesterday". 

    Because of how NOAA updates the .html, I don't necessarily have all the data for yesterday when the parser runs at midnight today...  Is it safe to update yesterday's daily accumulation multiple times?

     

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    Yes, it's safe to update yesterday data multiple times if you just use day timestamp (at 00:00 hours). Same it's applicable to today data too. That single value will represent total day rain. 

    0
    Comment actions Permalink

Please sign in to leave a comment.