How can we help?

  1. RainMachine
  2. Community
  3. Developers

Parser for german DWD (Deutscher Wetterdienst)

Pinned Featured
Follow
Avatar
Sebastian

Hello everybody, 

 

as discussed - among others - here, WeatherUnderground is not giving out any more free API keys. The future of already existing API keys and the associated services is unknown. 

Therefore I'm looking for alternatives that do not have parsers yet. Since I'm from germany, I had a look at the OpenData services from the German Meteorological Office ("Deutscher Wetterdienst")/DWD.

I forked the developer ressources repository on GitHub and added the parser to my fork. A pull request is already raised.

If you want to try it, you can just grab the file here. An then upload it to your rainmachine.

You have to enter a stationID. The list of available stations can be found in the DWDs opendata station catalogue:

https://www.dwd.de/DE/leistungen/opendata/help/stationen/mosmix_stationskatalog.cfg?view=nasPublication&nn=16102

It's my first python project. So maybe someone more experienced can review the code and add some improvements.

1

Comments

44 comments

Sort by Date Votes
  • Avatar
    RainMachine Nicholas

    Great work, code looks clean. I'm adding this to test queue and maybe we can include in the next builds with your permission to modify if needed.

    0
    Comment actions Permalink
  • Avatar
    Sebastian

    Hi Nicholas, 

    since your source repo is GPLv3, the repo fork is of course the same license.

    This means that you are of course allowed to modify if needed and include it into Rainmachine FW deliveries.

    To highlight this, I added the GPL header to the file in my repo.

    0
    Comment actions Permalink
  • Avatar
    Tomas

    Many thanks! Vielen Dank!

    0
    Comment actions Permalink
  • Avatar
    Tomas

    Hi Sebastian,

    please check my post regarding a bus with the DWD parser

    https://support.rainmachine.com/hc/en-us/community/posts/360012576333/comments/360001312674

    after adding the DWD parser my RM calculates a very high EvapoTranspiration and wants to extend the watering massively

    0
    Comment actions Permalink
  • Avatar
    Sebastian

    Hi Tomas, 

    thanks for the hint. I found the problem an I will add an updated version later.

    Greetings

     

    Sebastian

    0
    Comment actions Permalink
  • Avatar
    Sebastian

    And fix is here:

    https://github.com/SgtSeppel/rainmachine-developer-resources/blob/master/sdk-parsers/RMParserFramework/parsers/dwd-parser.py

    Solar radiation was not converted from MJ/m² to kJ/m². My bad ...

    With this, it looks a lot better now.

    0
    Comment actions Permalink
  • Avatar
    Tomas

    cant upload the new file.

    i replaced the code of my existing file and created a new one.

    but RM says : error uploading dwd-parser.py

    0
    Comment actions Permalink
  • Avatar
    Sebastian

    Its too late to be doing such things...

    Can you please move the line

    # coding=utf-8

    to the first line of the file, before the disclaimer?

    Otherwise I'll reupload tomorrow.

    0
    Comment actions Permalink
  • Avatar
    Tomas

    Thanks. Will try tomorrow. Just shut down the pc

    0
    Comment actions Permalink
  • Avatar
    Tomas

    OK, uploaded successfully. BUT: the Evapo Transpiration goes up again from 3.8 to avg 25.

    Is this realistic?

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    No. I looked at the source and I have 2 comments:

    1. I would disable SOLAR RADIATION as it might come wrongly from the station/service. We had a similar issue with SOLAR RADIATION from Weather Underground, coming with either wrong values or different unit of measurement than the one documented.

    2. The min/max temperatures seems to be for yesterday (in CSV) but the parser inserts them for today. This has a small impact (but nothing of that 25 ET0 magnitude). I would add them with something like:

    yesterdayTimestamp = rmGetStartOfDay(timestamp - 12 * 60 * 60)
    0
    Comment actions Permalink
  • Avatar
    Sebastian

    Hi Nicholas,

    Thanks for the hints.
    Im currently installing my irrigation.
    As soon as I have closed all the trenches and installed the RM at the final install location, I will have more time to toy around with the RM and take another look at parser.

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    I pushed some small changes to DWD parser into our git  and I also disabled SOLAR RADIATION. Still needs more testing before inclusion.

    0
    Comment actions Permalink
  • Avatar
    Tomas

    hi nicholas,

    I tested ur version and I still have evapotranspiration of 170-300

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    Thanks ! Can you give me your station id ?

    0
    Comment actions Permalink
  • Avatar
    Tomas

    10736

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    I updated the parser again on git. I missed a second Solar Radiation being fetched. Running on my machine seems fine. 

    To update parser do (in Web UI):

    1. DELETE parser from User Uploaded tab

    2. Press REFRESH ALL 

    3. Upload the parser again, configure it and SAVE

    4. Press REFRESH ALL

    0
    Comment actions Permalink
  • Avatar
    Tomas

    I have copied the code from here:

    https://github.com/sprinkler/rainmachine-developer-resources/blob/a6eacc43f12eeb46d60c49e2d700b9badb82cb23/sdk-parsers/RMParserFramework/parsers/dwd-parser.py 

     

    and put it into my dwd-parser.py file.

    but I still have 160-300 evapotrans

    did I take the wrong file?

    could you show me the link of the latest file?

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    https://github.com/sprinkler/rainmachine-developer-resources/blob/master/sdk-parsers/RMParserFramework/parsers/dwd-parser.py

    0
    Comment actions Permalink
  • Avatar
    Tomas

    hey all,

    After my last dwd test I cant bring the RM-Mixer to recalculate the values.

    The ET ist still 160-300 and RM wants to extend the watering time up to 200%.

    Can I reset the RM-Mixer?I turned off my 3 weather services or used only the dwd. nothing made the mixer recalculate ET

    0
    Comment actions Permalink
  • Avatar
    Tomas

    I have also disabled all weather correction and rebooted the RM twice... no change

    0
    Comment actions Permalink
  • Avatar
    Tomas

    ok, it works again

    0
    Comment actions Permalink
  • Avatar
    Tomas

    Its me again. I see some strange behavoir of my RM. After 2 days of significant precipitation my watering forecast looks like this:

    loan is watered every day and the bushes not at all...

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    What's the field capacity of Zone 1 ? Can you compare the graphs ("show graphs" button) next to field capacity for Zone  1 and Zone 2 ?

    0
    Comment actions Permalink
  • Avatar
    Tomas

    here we go

    0
    Comment actions Permalink
  • Avatar
    RainMachine Nicholas

    We should move the discussion outside this thread but it's pretty strange why zone 1 didn't register that large amount of rain. Guess the program wasn't created after the rain event ? Maybe some zone soil setting won't let the water accumulate that much (I see much lower field capacity on Zone 1). Let me know if it doesn't auto-correct itself at next run.

    0
    Comment actions Permalink
  • Avatar
    Sebastian

    Hello together, 

    the changes discussed above, double entry of solar radiation, wrong conversion of some units, are included in the current commit on my git.

    If I just checked again. Without the DWD parser the ET from the mixer is 6.60mm for June 31st. 

    Thats the same value that I get with the parser?

    0
    Comment actions Permalink
  • Avatar
    Tomas

    I cant check the dwd parser. Made a factory rest becaue of the strange zone2 calculation. now my  field capacity ist still 21 and 91 but the watering time is correct. zone 2 is beeing watered again every day. I will try the dwd parser when its running without any errors. otherwise I risk again to reset everything

    0
    Comment actions Permalink
  • Avatar
    Sebastian

    I just added the current version of my parser to github and created a pull request. 

    I have been running that now for several weeks without issues. ET is now totally within expected ranges.

    0
    Comment actions Permalink
  • Avatar
    Sebastian

    So, after a while I finally had the time to move the DWD parser to the new XML interface from the DWD. 

    I can already run it locally on my PC, but I have a problem uploading it to my RM.

    I have to use the lxml package since the data is provided as an (zipped) XML. However, RM does not seem to know it. 

    Is there a way to install lxml automatically?

    0
    Comment actions Permalink

Please sign in to leave a comment.

Didn't find what you were looking for?

New post