How can we help?


Ecowiit GW1000 Parser

Pinned Featured

Comments

35 comments

  • Avatar
    RainMachine Nicholas

    That's great work ! Thanks for sharing your work with us. I will have to try with my personal Fine Offset station , if it works it can greatly simplify a weewx setup.

    0
    Comment actions Permalink
  • Avatar
    GHammer (Edited )

    Thanks for this! I'm hoping that this will not interfere with the GW1000 custom server functionality.

    I have installed this parser, very nicely done. I'm assuming that nothing will be in the RainMachine logs until it writes the data at it's one per day interval.

    I do see that it found my GW1000 and that success is reported as status, so I will wait until tomorrow to verify via the log that data is flowing and reported properly for my system.

     

     

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira

    Hi GHammer,

    Thanks for your feedback. I am happy that someone else's is using it.

    I do confirm that this parser does not interfere with the GW1000 custom service functionality.

    The parser only writes one historical data point per day. The parser updates the data point every 60s.

    Please, let me know if that data looks ok and if you found any issue.

    Regards,
    Pedro

    0
    Comment actions Permalink
  • Avatar
    GHammer

    Pedro,

    I have had the parser running for 3 days and see no errors, or unexpected values.

    So, let me make sure I understand correctly.

    The parser gets data from the GW1000 once per minute.
    The data is stored, then various values are calculated and reported to the RainMachine weather system once per day.

    First, the data is being sent/written at 0500 daily in my location. Not a problem for me as I will not start watering until 0545 at the earliest.

    Next Max Temp is greatly different than the NOAA value, but I believe the NOAA value is the forecast Max Temp, not the actual Max Temp.
    Min Temp is 7F different than the local weather push for the same time. 29.4F for the local weather push (from WeeWX) and 22.6F for the parser. 
    Temp is within 0.2F of the local weather push.
    Wind is fairly different, local weather shows 0.54 m/s vs 3.12 m/s for the parser value.
    Pressure is within 0.05 kPa

    I'm fairly certain this is due to different measurements/accumulations being reported for the given values.

    Curious though, why not report once per hour vs once per day?

     

    0
    Comment actions Permalink
  • Avatar
    GHammer

    So, I looked today and the parser has found a console instead of the GW1000.

    I have a 2902A, a WS2000, and the GW1000. It found first the 2902A then when I reset and refreshed it, found the WS2000.

     

    So, it looks like this either needs to parse and find the GW1000 or allow manual config for those who have other consoles on their network.

     

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira (Edited )

    GHammer, 

    Good to know that the parser was working without errors or unexpected values.

    The parser:

    1) It gets data from the GW1000 once per minute.

    2) It stores the sensor data locally.

    3) It computes the following daily values:

    • MEAN, MAX and MIN TEMPERATURE
    • MEAN, MAX and MIN RH
    • MEAN PRESSURE
    • MEAN WIND
    • TOTAL RAIN
    • MEAN SOLAR RADIATION

    4) It updates a daily historical data point, at midnight UTC of the current day, with the computed values. Again, it updates the same data point every minute.

    5) Rainmachine uses that data point, as a personal weather station data, to adjust the next day Eto, based on the observation.

     

    Regarding other questions:

    • Data wrote at 05:00 at your location: The parser writes the data at midnight UTC, your local timezone should be 5 hours away from UTC. According to Rainmachine Nicholas, data should be written in UTC timezone.
    • Why not report once per hour vs once per day? If I wrote data every hour, even better every minute, Rain machine would transform that in a single data point at the end of the day. That single data point generated by rain machine does not match the arithmetic calculations for max, min, avg. For some variables, the rain machine uses the last value of the day. ( I think is because this is a historical parser and not a forecast parser).
    • The parser has found a console instead of the GW1000: The discovery mechanism of other FineOffset consoles is the same that GW1000. As a workaround, for now, I have implemented a quick fix, that allows setting the device IP address manually. 

    Thank you for your input,

    Pedro

    0
    Comment actions Permalink
  • Avatar
    GHammer

    Hi Pedro,

    I edited the file and changed the IP value as I have a static IP assigned to all IoT devices on the LAN. Looking at the various forums, I'd guess that many have a GW1000 and a console of some sort so the manual setting will be good for those uncomfortable with editing directly.

    If you look at the WeeWX extension that I believe was written by someone with RainMachine, it sends data once per hour. So, if I understand you, there is no point as RainMachine will transform the data and do it incorrectly at that.

    If so, then I should disable the WeeWX extension.

    Thanks for the parser and for the info!

     

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira

    The concept of just having one data point is also present on the WeatherFlow Smart Weather Station data parser, made available by Rainmachine. (https://github.com/sprinkler/rainmachine-developer-resources/blob/master/sdk-parsers/RMParserFramework/parsers/wf-parser.py)

     

    0
    Comment actions Permalink
  • Avatar
    GHammer

    Oh, I'm not concerned if you say it is immaterial, good enough for me.

    My question is why the RainMachine devs have frequent data entry across all they provide. The inbuilt NOAA/WU/NETAMO and the only one I use, WeeWX extension.

    Personally, I have questioned the data from those in the past and not gotten a satisfactory answer.

    Would you like me to install the new version, or is the IP address entry the only change?

    0
    Comment actions Permalink
  • Avatar
    GHammer

    I've noticed that, after installing this parser, my Mini-8 looks like it restarts at around 0200 daily.

    Here's the log around the event.

    2020-03-11 01:50:18,014 - INFO  - rmHTTPResourceApi4Parser:416 - Local Weather API push: 1583905800: {'maxtemp': 15.222222222222221, 'temperature': 8.6666666666666639, 'dewpoint': 3.95790891045331, 'maxrh': 72.277777777777779, 'rain': 0.0, 'pressure': 101.43541115191671, 'timestamp': 1583905800, 'mintemp': 8.6666666666666639, 'et': 0.0023845454400549757, 'wind': 2.1058067549659297}
    2020-03-11 02:44:04,401 - INFO  - main:50 - RainMachine v2 Copyright (c) 2015 Green Electronics LLC
    2020-03-11 02:44:04,421 - INFO  - main:51 - Running on platform OpenWrt OS
    2020-03-11 02:44:04,611 - INFO  - main:73 - WIRELESS Interface wlan0, Mode: managed, IP: 10.10.100.120, MAC: a8:80:38:25:0b:94, hasClientLink: 1, Mode: managed
    2020-03-11 02:44:05,422 - INFO  - rmRestrictions:394 - *** Populating week weather data from database.
    2020-03-11 02:44:05,635 - INFO  - rmTimeZoneOpenWRT:46 - Setting timezone to: America/New_York Olson: EST5EDT,M3.2.0,M11.1.0
    2020-03-11 02:44:05,774 - INFO  - rmStationDownloader:27 - Downloading station info: http://54.186.186.3/api/et_stats/nearby_stations?latitude=42.220391&longitude=-72.638608&elevation=136
    2020-03-11 02:44:06,595 - INFO  - rmDoyDownloader:22 - Downloading DOY database: http://54.186.186.3/api/et_stats/eto?station_id=9051&source=station
    2020-03-11 02:44:07,852 - INFO  - rmDoyDownloader:48 - Got 365 values for et0 and 5.00 as et0Avg
    2020-03-11 02:44:08,394 - INFO  - rmUserSettings:139 - ---------------------------------------------------------------------------------------------
    2020-03-11 02:44:08,408 - INFO  - rmUserSettings:161 - Version ('4.0.999') Running for:
    - name: 'REMOVED"
    - timezone: 'America/New_York'
    - latitude: **.******
    - longitude: -**.******
    - elevation: 136
    - et0Average: 5.5
    - krs: 0.16
    - rainSensitivity: 0.73
    - windSensitivity: 0.4
    - wsDays: 2
    - database path: '/rainmachine-app/DB/Default'
    - httpsServerPort: 8080 (ssl)
    - httpServerPort: 18080
    - wizardHasRun: 1
    2020-03-11 02:44:08,420 - INFO  - rmUserSettings:162 - ---------------------------------------------------------------------------------------------
    2020-03-11 02:44:08,439 - INFO  - rmThreadWatcher:136 - ThreadWatcher running True
    2020-03-11 02:44:08,523 - INFO  - rmLocalNetworkAdvertise:111 - Local Network Advertiser starting
    2020-03-11 02:44:08,649 - INFO  - rmParserManager:228 - *** BEGIN Loading parsers from './RMParserFramework/parsers'
    2020-03-11 02:44:08,708 - INFO  - rmParser:71 - *** Registering parser OpenWeatherMap Parser with interval 21600
    2020-03-11 02:44:08,902 - INFO  - rmParser:71 - *** Registering parser Netatmo Parser with interval 21600
    2020-03-11 02:44:09,106 - INFO  - rmParser:71 - *** Registering parser Ecowitt Parser with interval 60
    2020-03-11 02:44:09,293 - INFO  - rmParser:71 - *** Registering parser WUnderground Parser with interval 21600
    2020-03-11 02:44:09,497 - INFO  - rmParser:71 - *** Registering parser CIMIS Parser with interval 21600
    2020-03-11 02:44:09,651 - INFO  - rmParser:71 - *** Registering parser ForecastIO Parser with interval 21600
    2020-03-11 02:44:09,905 - INFO  - rmParser:71 - *** Registering parser Weather Rules Parser with interval 3600
    2020-03-11 02:44:10,224 - INFO  - rmParser:71 - *** Registering parser NOAA Parser with interval 21600
    2020-03-11 02:44:10,325 - INFO  - rmParser:71 - *** Registering parser FAWN Parser with interval 21600
    2020-03-11 02:44:10,484 - INFO  - rmParser:71 - *** Registering parser WeatherDisplay Parser with interval 21600
    2020-03-11 02:44:10,656 - INFO  - rmParser:71 - *** Registering parser METNO Parser with interval 21600
    2020-03-11 02:44:10,413 - INFO  - rmParserManager:313 - *** END Loading parsers
    2020-03-11 02:44:10,502 - INFO  - rmSimulator:1086 - Loading last weather simulation data for today programs
    2020-03-11 02:44:10,947 - INFO  - rmTouchManager:544 - Closing boot animation
    2020-03-11 02:44:10,975 - INFO  - rmTouchManager:260 - Touch Manager is starting up ...
    2020-03-11 02:44:10,987 - INFO  - rmMainManager:139 - Stopping bonjour service

     

    0
    Comment actions Permalink
  • Avatar
    GHammer

    I had to disable this parser.
    The CPU usage value on my Mini-8 pegs at 100% and everything (populating logs, weather, etc) is extremely slow.
    Disable this parser all returns to normal.

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira (Edited )

    Hi GHammer,

    I have published a new version: https://github.com/pjpeartree/rainmachine-gw1000

    Changelog:

    20200902:

    • Check device name to avoid detection of unsupported local consoles.
    • Try to find the device within 5 retries.
    • Update discover socket timeout to 2 seconds.
    • Reduce the arithmetic effort by using cumulative numerical total divided by the number of observations.

    20200903:

    • Adding Battery Temperature sensor ids.

    20200904:

    • Increase the default min and max temperatures.
    • Fix day max wind sensor byte size.
    • New helper function to report observations into rainmachine.
    • Move new day check into the perform function, for better code readability.
    • Performance improvement, keep observations in memory only, do not save them into a data file. The downside, in case of an unlikely power outage, the current day observations are lost. This also takes care of flash lifespan avoiding I/O operations.

    Thank you, for your valuable feedback.

    Pedro

    0
    Comment actions Permalink
  • Avatar
    BB

    Hey Pedro,  

    I have an Ecowiit GW1000 Gateway that is plugged in indoors with a linked WH40 Rain Gauge outside.  

    I only want the rain total to come into the RainMachine and not the indoor temperature from the gateway.

    Does the parser import the temperature reading from the gateway into the RainMachine?

     

    Thanks,

    Ben

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira

    Hi Ben,

    No, the parser does not import the Indoor Temperature. The Indoor Temperature sensor is ignored 

    Regards,

    Pedro

    0
    Comment actions Permalink
  • Avatar
    BB

    Cool, Thanks.  I look forward to trying this out.

     

    0
    Comment actions Permalink
  • Avatar
    BB

    There is some weird behavior for me.  When I load the parser the first time, a Custom:METNO parser shows up in the User Uploaded tab. 

    it shows the following in the RainMachine log:

    rmParser:73 - *** Registering parser %s failed
    rmParserManager:384 -   * Parser user-gw1000-parser.py successful loaded from file '/rainmachine-app/RMParserFramework/parsers/user-gw1000-parser.py'

     

    Then, if I delete the invalid parser and load the Ecowitt parser again, it shows up correctly as Custom:GW1000.  I can then enable and save it and it successfully runs and finds the gateway.

    rmParser:71 - *** Registering parser GW1000 Parser with interval 60
    rmParserManager:384 -   * Parser user-gw1000-parser.py successful loaded from file '/rainmachine-app/RMParserFramework/parsers/user-gw1000-parser.py'
    2

    However, if I reboot the RainMachine, it disappears from the User Uploaded tab. 

    It is also missing from the RainMachine log during the parser registration at bootup or shows the failed to register message:

    INFO  - rmParser:73 - *** Registering parser %s failed

     

     

    Here are the versions from my RainMachine About menu:

    Firmware version
    4.0.1148 (No updates)

    Web UI Version
    1.14

    Device model
    5

    API
    4.6.1

     

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira

    Hi Ben,
    Thank you for report this issue.

    I have successfully reproduced it.
    I found an error when rainmachine tries to create a GW1000 parser instance.

    I have fixed the issue, and a new version is now available. https://github.com/pjpeartree/rainmachine-gw1000

    Regards,
    Pedro

    0
    Comment actions Permalink
  • Avatar
    BB (Edited )

    Thanks, that fixed the issue.  It now installs correctly and stays installed after a reboot.

     

    One thing I noticed is that I get the following errors every minute in the log now.  Is this to be expected?

    2020-09-10 10:06:31,227 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
    2020-09-10 10:06:31,236 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidated
    2020-09-10 10:07:56,472 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
    2020-09-10 10:07:56,489 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidated
    2020-09-10 10:09:00,056 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
    2020-09-10 10:09:00,067 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidated
    2020-09-10 10:10:23,877 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
    2020-09-10 10:10:23,887 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidated

     

    Also, the weather services comparison is showing the GW1000 is reporting 32 degrees for the min and max temp. See the screenshot:

     

    Same for Wind and Humidity.  

    0
    Comment actions Permalink
  • Avatar
    BB

    Just FYI, for now, I've commented out the code for the sensors I do not have and re-uploaded the parser to my RainMachine.  It looks to be working now without reporting errors or invalid data in the weather graphs.

     

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira

    Hi Ben,

    Good to know that the initial issue is now solved.

    Regarding the Min-Max Temperature, the parser was designed with all sensors in mind (WH3000SE sensor array), but your use case, only have a rain gauge sensor.

    That is good to know, I need to add some extra lines of code to avoid reporting ignored sensors, and the parser will be ready for other use cases.

    The current version is reporting the default values of the ignored sensors.  You are seeing 32º Fahrenheit because the parser default value for temperatures is 0º Celsius.

     

    0
    Comment actions Permalink
  • Avatar
    BB

    Yep, that totally makes sense.  Thanks for looking into it. 

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira (Edited )

    Hi,

    The new version is now available. https://github.com/pjpeartree/rainmachine-gw1000

    Please, can you give it a try?

    For your specific use case, and because of gw1000 report rain as a daily amount, you can increase the polling interval to 3600, or more.

    0
    Comment actions Permalink
  • Avatar
    BB (Edited )

    Yeah, it's working without errors now.  Thanks!

     

    By the way, do you know of any way to clear out the invalid temperatures reported into my RainMachine from the parser for the last two days?

     

     

    Just FYI, the Defaults button doesn't work and the log shows this:

    ERROR - rmHTTPResourceApi4Parser:193 - 'GW1000' object has no attribute 'defaultParams'

     

     

    0
    Comment actions Permalink
  • Avatar
    PedroJorgePereira

    Hi,

    The new version is now available. https://github.com/pjpeartree/rainmachine-gw1000

     - Update isEnabledForLocation to match the default expected behaviour.
     - Remove unnecessary params.
     - Fix default params error.

    BB, Changing location settings will delete all weather data. I do that to remove old entries, and then I revert to the original location.

    0
    Comment actions Permalink
  • Avatar
    BB

    I've loaded the new version and it is working. I also successfully tested the Defaults button.

    However, I did like the old version where it listed the Gateway Name and IP address that it found during the auto discover process. But I guess I can see how it's not necessary for the average user.

    Thanks again for developing this parser.  I really like how I can avoid linking to WUnderground.

     

    0
    Comment actions Permalink
  • Avatar
    BB

    It's finally rained a couple more times and I can report back that the parser still looks to be detecting rain correctly.

     

    One thing I noticed is that it inserts the data at 5:00 am CST for me.   Is this the correct behavior?  I would think you would want it at 12:00 midnight.   

     

    The attached screenshot is a composite showing the GW1000 time of 5:00 am as compared to the RainMachine Mixer at 00:00.

     

     

    0
    Comment actions Permalink
  • Avatar
    Alessandro Del Prete

    As Pedro wrote here: https://support.rainmachine.com/hc/en-us/community/posts/360038349133/comments/360010665574

    "Data wrote at 05:00 at your location: The parser writes the data at midnight UTC, your local timezone should be 5 hours away from UTC. According to Rainmachine Nicholas, data should be written in UTC timezone."

    0
    Comment actions Permalink
  • Avatar
    Alessandro Del Prete (Edited )

    One question for Pedro: in RM's Observed Weather Station Rain tab I see that your parser always gives the accumulated rain total each day, while others only provide the daily value. How is that supposed to work? Is your parser correct and the others get it wrong?

    Here Nicholas says here: https://support.rainmachine.com/hc/en-us/community/posts/360016701114/comments/360003923094

    "Day will always default to 00:00 the only information taken by RainMachine and UI is the day date not hour. Mixer will only work with aggregated day values. Parsers might have hourly values but mixer only day aggregated from all parsers and all hours."

    So I guess it should report the daily aggregated, not the monthly total, right?

    0
    Comment actions Permalink
  • Avatar
    BB

    Thanks for the response.  It looks like there still may be a bug in the Web UI graphs:

    https://support.rainmachine.com/hc/en-us/community/posts/360016701114-Timzone-issues-in-custom-parser

     

     

    0
    Comment actions Permalink
  • Avatar
    BB (Edited )

    Do you have the latest parser version released on Oct 5, 2020?   

     

    He made some changes to reset the accumulation value each day:

    # 20201005
    # - Update the reset observations function.

     

    Mine is only giving the total value for each day and reverts back to 0 the following days as expected.

     

    Update:  I checked and I am running version 20200915 without the changes mentioned above.

     

    0
    Comment actions Permalink

Please sign in to leave a comment.