Ecowiit GW1000 Parser
Pinned Featured
I have created a parser for Ecowiit devices.
This parser was created to avoid the use of cloud solutions, like WUnderground.com or Ecowitt.net (API not available yet). The parser establishes a direct connection between the Ecowitt and the RainMachine devices.
https://github.com/pjpeartree/rainmachine-gw1000
It was created and tested using data from an Ecowiit GW1000 with WH3000SE sensor array.
Feel free to use it.
-
Thanks for this! I'm hoping that this will not interfere with the GW1000 custom server functionality.
I have installed this parser, very nicely done. I'm assuming that nothing will be in the RainMachine logs until it writes the data at it's one per day interval.
I do see that it found my GW1000 and that success is reported as status, so I will wait until tomorrow to verify via the log that data is flowing and reported properly for my system.
-
Hi GHammer,
Thanks for your feedback. I am happy that someone else's is using it.
I do confirm that this parser does not interfere with the GW1000 custom service functionality.
The parser only writes one historical data point per day. The parser updates the data point every 60s.
Please, let me know if that data looks ok and if you found any issue.
Regards,
Pedro -
Pedro,
I have had the parser running for 3 days and see no errors, or unexpected values.
So, let me make sure I understand correctly.
The parser gets data from the GW1000 once per minute.
The data is stored, then various values are calculated and reported to the RainMachine weather system once per day.First, the data is being sent/written at 0500 daily in my location. Not a problem for me as I will not start watering until 0545 at the earliest.
Next Max Temp is greatly different than the NOAA value, but I believe the NOAA value is the forecast Max Temp, not the actual Max Temp.
Min Temp is 7F different than the local weather push for the same time. 29.4F for the local weather push (from WeeWX) and 22.6F for the parser.
Temp is within 0.2F of the local weather push.
Wind is fairly different, local weather shows 0.54 m/s vs 3.12 m/s for the parser value.
Pressure is within 0.05 kPaI'm fairly certain this is due to different measurements/accumulations being reported for the given values.
Curious though, why not report once per hour vs once per day?
-
So, I looked today and the parser has found a console instead of the GW1000.
I have a 2902A, a WS2000, and the GW1000. It found first the 2902A then when I reset and refreshed it, found the WS2000.
So, it looks like this either needs to parse and find the GW1000 or allow manual config for those who have other consoles on their network.
-
GHammer,
Good to know that the parser was working without errors or unexpected values.
The parser:
1) It gets data from the GW1000 once per minute.
2) It stores the sensor data locally.
3) It computes the following daily values:
- MEAN, MAX and MIN TEMPERATURE
- MEAN, MAX and MIN RH
- MEAN PRESSURE
- MEAN WIND
- TOTAL RAIN
- MEAN SOLAR RADIATION
4) It updates a daily historical data point, at midnight UTC of the current day, with the computed values. Again, it updates the same data point every minute.
5) Rainmachine uses that data point, as a personal weather station data, to adjust the next day Eto, based on the observation.
Regarding other questions:
- Data wrote at 05:00 at your location: The parser writes the data at midnight UTC, your local timezone should be 5 hours away from UTC. According to Rainmachine Nicholas, data should be written in UTC timezone.
- Why not report once per hour vs once per day? If I wrote data every hour, even better every minute, Rain machine would transform that in a single data point at the end of the day. That single data point generated by rain machine does not match the arithmetic calculations for max, min, avg. For some variables, the rain machine uses the last value of the day. ( I think is because this is a historical parser and not a forecast parser).
- The parser has found a console instead of the GW1000: The discovery mechanism of other FineOffset consoles is the same that GW1000. As a workaround, for now, I have implemented a quick fix, that allows setting the device IP address manually.
Thank you for your input,
Pedro
-
Hi Pedro,
I edited the file and changed the IP value as I have a static IP assigned to all IoT devices on the LAN. Looking at the various forums, I'd guess that many have a GW1000 and a console of some sort so the manual setting will be good for those uncomfortable with editing directly.
If you look at the WeeWX extension that I believe was written by someone with RainMachine, it sends data once per hour. So, if I understand you, there is no point as RainMachine will transform the data and do it incorrectly at that.
If so, then I should disable the WeeWX extension.
Thanks for the parser and for the info!
-
The concept of just having one data point is also present on the WeatherFlow Smart Weather Station data parser, made available by Rainmachine. (https://github.com/sprinkler/rainmachine-developer-resources/blob/master/sdk-parsers/RMParserFramework/parsers/wf-parser.py)
-
Oh, I'm not concerned if you say it is immaterial, good enough for me.
My question is why the RainMachine devs have frequent data entry across all they provide. The inbuilt NOAA/WU/NETAMO and the only one I use, WeeWX extension.
Personally, I have questioned the data from those in the past and not gotten a satisfactory answer.
Would you like me to install the new version, or is the IP address entry the only change?
-
I've noticed that, after installing this parser, my Mini-8 looks like it restarts at around 0200 daily.
Here's the log around the event.
2020-03-11 01:50:18,014 - INFO - rmHTTPResourceApi4Parser:416 - Local Weather API push: 1583905800: {'maxtemp': 15.222222222222221, 'temperature': 8.6666666666666639, 'dewpoint': 3.95790891045331, 'maxrh': 72.277777777777779, 'rain': 0.0, 'pressure': 101.43541115191671, 'timestamp': 1583905800, 'mintemp': 8.6666666666666639, 'et': 0.0023845454400549757, 'wind': 2.1058067549659297} 2020-03-11 02:44:04,401 - INFO - main:50 - RainMachine v2 Copyright (c) 2015 Green Electronics LLC 2020-03-11 02:44:04,421 - INFO - main:51 - Running on platform OpenWrt OS 2020-03-11 02:44:04,611 - INFO - main:73 - WIRELESS Interface wlan0, Mode: managed, IP: 10.10.100.120, MAC: a8:80:38:25:0b:94, hasClientLink: 1, Mode: managed 2020-03-11 02:44:05,422 - INFO - rmRestrictions:394 - *** Populating week weather data from database. 2020-03-11 02:44:05,635 - INFO - rmTimeZoneOpenWRT:46 - Setting timezone to: America/New_York Olson: EST5EDT,M3.2.0,M11.1.0 2020-03-11 02:44:05,774 - INFO - rmStationDownloader:27 - Downloading station info: http://54.186.186.3/api/et_stats/nearby_stations?latitude=42.220391&longitude=-72.638608&elevation=136 2020-03-11 02:44:06,595 - INFO - rmDoyDownloader:22 - Downloading DOY database: http://54.186.186.3/api/et_stats/eto?station_id=9051&source=station 2020-03-11 02:44:07,852 - INFO - rmDoyDownloader:48 - Got 365 values for et0 and 5.00 as et0Avg 2020-03-11 02:44:08,394 - INFO - rmUserSettings:139 - --------------------------------------------------------------------------------------------- 2020-03-11 02:44:08,408 - INFO - rmUserSettings:161 - Version ('4.0.999') Running for: - name: 'REMOVED" - timezone: 'America/New_York' - latitude: **.****** - longitude: -**.****** - elevation: 136 - et0Average: 5.5 - krs: 0.16 - rainSensitivity: 0.73 - windSensitivity: 0.4 - wsDays: 2 - database path: '/rainmachine-app/DB/Default' - httpsServerPort: 8080 (ssl) - httpServerPort: 18080 - wizardHasRun: 1 2020-03-11 02:44:08,420 - INFO - rmUserSettings:162 - --------------------------------------------------------------------------------------------- 2020-03-11 02:44:08,439 - INFO - rmThreadWatcher:136 - ThreadWatcher running True 2020-03-11 02:44:08,523 - INFO - rmLocalNetworkAdvertise:111 - Local Network Advertiser starting 2020-03-11 02:44:08,649 - INFO - rmParserManager:228 - *** BEGIN Loading parsers from './RMParserFramework/parsers' 2020-03-11 02:44:08,708 - INFO - rmParser:71 - *** Registering parser OpenWeatherMap Parser with interval 21600 2020-03-11 02:44:08,902 - INFO - rmParser:71 - *** Registering parser Netatmo Parser with interval 21600 2020-03-11 02:44:09,106 - INFO - rmParser:71 - *** Registering parser Ecowitt Parser with interval 60 2020-03-11 02:44:09,293 - INFO - rmParser:71 - *** Registering parser WUnderground Parser with interval 21600 2020-03-11 02:44:09,497 - INFO - rmParser:71 - *** Registering parser CIMIS Parser with interval 21600 2020-03-11 02:44:09,651 - INFO - rmParser:71 - *** Registering parser ForecastIO Parser with interval 21600 2020-03-11 02:44:09,905 - INFO - rmParser:71 - *** Registering parser Weather Rules Parser with interval 3600 2020-03-11 02:44:10,224 - INFO - rmParser:71 - *** Registering parser NOAA Parser with interval 21600 2020-03-11 02:44:10,325 - INFO - rmParser:71 - *** Registering parser FAWN Parser with interval 21600 2020-03-11 02:44:10,484 - INFO - rmParser:71 - *** Registering parser WeatherDisplay Parser with interval 21600 2020-03-11 02:44:10,656 - INFO - rmParser:71 - *** Registering parser METNO Parser with interval 21600 2020-03-11 02:44:10,413 - INFO - rmParserManager:313 - *** END Loading parsers 2020-03-11 02:44:10,502 - INFO - rmSimulator:1086 - Loading last weather simulation data for today programs 2020-03-11 02:44:10,947 - INFO - rmTouchManager:544 - Closing boot animation 2020-03-11 02:44:10,975 - INFO - rmTouchManager:260 - Touch Manager is starting up ... 2020-03-11 02:44:10,987 - INFO - rmMainManager:139 - Stopping bonjour service -
Hi GHammer,
I have published a new version: https://github.com/pjpeartree/rainmachine-gw1000
Changelog:
20200902:
- Check device name to avoid detection of unsupported local consoles.
- Try to find the device within 5 retries.
- Update discover socket timeout to 2 seconds.
- Reduce the arithmetic effort by using cumulative numerical total divided by the number of observations.
20200903:
- Adding Battery Temperature sensor ids.
20200904:
- Increase the default min and max temperatures.
- Fix day max wind sensor byte size.
- New helper function to report observations into rainmachine.
- Move new day check into the perform function, for better code readability.
- Performance improvement, keep observations in memory only, do not save them into a data file. The downside, in case of an unlikely power outage, the current day observations are lost. This also takes care of flash lifespan avoiding I/O operations.
Thank you, for your valuable feedback.
Pedro
-
Hey Pedro,
I have an Ecowiit GW1000 Gateway that is plugged in indoors with a linked WH40 Rain Gauge outside.
I only want the rain total to come into the RainMachine and not the indoor temperature from the gateway.
Does the parser import the temperature reading from the gateway into the RainMachine?
Thanks,
Ben
-
There is some weird behavior for me. When I load the parser the first time, a Custom:METNO parser shows up in the User Uploaded tab.
it shows the following in the RainMachine log:
rmParser:73 - *** Registering parser %s failed rmParserManager:384 - * Parser user-gw1000-parser.py successful loaded from file '/rainmachine-app/RMParserFramework/parsers/user-gw1000-parser.py'
Then, if I delete the invalid parser and load the Ecowitt parser again, it shows up correctly as Custom:GW1000. I can then enable and save it and it successfully runs and finds the gateway.
rmParser:71 - *** Registering parser GW1000 Parser with interval 60 rmParserManager:384 - * Parser user-gw1000-parser.py successful loaded from file '/rainmachine-app/RMParserFramework/parsers/user-gw1000-parser.py' 2
However, if I reboot the RainMachine, it disappears from the User Uploaded tab.
It is also missing from the RainMachine log during the parser registration at bootup or shows the failed to register message:
INFO - rmParser:73 - *** Registering parser %s failed
Here are the versions from my RainMachine About menu:
Firmware version
4.0.1148 (No updates)Web UI Version
1.14Device model
5API
4.6.1 -
Hi Ben,
Thank you for report this issue.I have successfully reproduced it.
I found an error when rainmachine tries to create a GW1000 parser instance.I have fixed the issue, and a new version is now available. https://github.com/pjpeartree/rainmachine-gw1000
Regards,
Pedro -
Thanks, that fixed the issue. It now installs correctly and stays installed after a reboot.
One thing I noticed is that I get the following errors every minute in the log now. Is this to be expected?
2020-09-10 10:06:31,227 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
2020-09-10 10:06:31,236 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidated
2020-09-10 10:07:56,472 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
2020-09-10 10:07:56,489 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidated
2020-09-10 10:09:00,056 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
2020-09-10 10:09:00,067 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidated
2020-09-10 10:10:23,877 - ERROR - rmLimits:43 - MAXTEMP value -100 less than limits minimum of -60, invalidated.
2020-09-10 10:10:23,887 - ERROR - rmLimits:47 - MINTEMP value 100 more than limits maximum of 60, invalidatedAlso, the weather services comparison is showing the GW1000 is reporting 32 degrees for the min and max temp. See the screenshot:
Same for Wind and Humidity.
-
Hi Ben,
Good to know that the initial issue is now solved.
Regarding the Min-Max Temperature, the parser was designed with all sensors in mind (WH3000SE sensor array), but your use case, only have a rain gauge sensor.
That is good to know, I need to add some extra lines of code to avoid reporting ignored sensors, and the parser will be ready for other use cases.
The current version is reporting the default values of the ignored sensors. You are seeing 32º Fahrenheit because the parser default value for temperatures is 0º Celsius.
-
Hi,
The new version is now available. https://github.com/pjpeartree/rainmachine-gw1000
Please, can you give it a try?
For your specific use case, and because of gw1000 report rain as a daily amount, you can increase the polling interval to 3600, or more.
-
Yeah, it's working without errors now. Thanks!
By the way, do you know of any way to clear out the invalid temperatures reported into my RainMachine from the parser for the last two days?
Just FYI, the Defaults button doesn't work and the log shows this:
ERROR - rmHTTPResourceApi4Parser:193 - 'GW1000' object has no attribute 'defaultParams'
-
Hi,
The new version is now available. https://github.com/pjpeartree/rainmachine-gw1000
- Update isEnabledForLocation to match the default expected behaviour.
- Remove unnecessary params.
- Fix default params error.BB, Changing location settings will delete all weather data. I do that to remove old entries, and then I revert to the original location.
-
I've loaded the new version and it is working. I also successfully tested the Defaults button.
However, I did like the old version where it listed the Gateway Name and IP address that it found during the auto discover process. But I guess I can see how it's not necessary for the average user.
Thanks again for developing this parser. I really like how I can avoid linking to WUnderground.
-
It's finally rained a couple more times and I can report back that the parser still looks to be detecting rain correctly.
One thing I noticed is that it inserts the data at 5:00 am CST for me. Is this the correct behavior? I would think you would want it at 12:00 midnight.
The attached screenshot is a composite showing the GW1000 time of 5:00 am as compared to the RainMachine Mixer at 00:00.
-
As Pedro wrote here: https://support.rainmachine.com/hc/en-us/community/posts/360038349133/comments/360010665574
"Data wrote at 05:00 at your location: The parser writes the data at midnight UTC, your local timezone should be 5 hours away from UTC. According to Rainmachine Nicholas, data should be written in UTC timezone."
-
One question for Pedro: in RM's Observed Weather Station Rain tab I see that your parser always gives the accumulated rain total each day, while others only provide the daily value. How is that supposed to work? Is your parser correct and the others get it wrong?
Here Nicholas says here: https://support.rainmachine.com/hc/en-us/community/posts/360016701114/comments/360003923094
"Day will always default to 00:00 the only information taken by RainMachine and UI is the day date not hour. Mixer will only work with aggregated day values. Parsers might have hourly values but mixer only day aggregated from all parsers and all hours."
So I guess it should report the daily aggregated, not the monthly total, right?
-
Thanks for the response. It looks like there still may be a bug in the Web UI graphs:
-
Do you have the latest parser version released on Oct 5, 2020?
He made some changes to reset the accumulation value each day:
# 20201005
# - Update the reset observations function.Mine is only giving the total value for each day and reverts back to 0 the following days as expected.
Update: I checked and I am running version 20200915 without the changes mentioned above.
Please sign in to leave a comment.
Comments
52 comments