×

INDI Library v2.0.6 is Released (02 Feb 2024)

Bi-monthly release with minor bug fixes and improvements

meteostationWEB no longer working due to python 3.8 on Ubuntu 20.04

  • Posts: 180
  • Thank you received: 30
if attrs.has_key('message'):

In Python 3, dictionaries changed a lot. Now is easier, change the line above to:

if 'message' in attrs:
3 years 6 months ago #59917

Please Log in or Create an account to join the conversation.

  • Posts: 180
  • Thank you received: 30
I got your code and had a look at it.
The code is not excellent and has some design and gramatical problems.
Anyway, the "Fail to retrieve internet data" comes from sounding.py.
First, urllib library has changed a lot. You had to modify the imports to this:

<code>#import tidy
import datetime
import time
import locale
from string import *
from urllib.error import HTTPError
from urllib.parse import urlencode
from urllib.request import urlopen, urlretrieve, Request </code>

and then substitute in the code correspondingly.

There are also some problems when opening the URLs. For example the uwyoClass class method get_sounding_skewt should read:

<code>def get_sounding_skewt(self):
data = {"region": "europe", "TYPE": "GIF:SKEWT", "YEAR": self.YEAR,
"MONTH": self.MONTH, "FROM": self.FROM, "TO": self.FROM, "STNM": SOUNDINGSTATION}

params = urlencode(data)
req = Request(url='weather.uwyo.edu/cgi-bin/sounding?%s' % params)
s = urlopen(req)
o = s.read()
s.close()
document = o
urlretrieve("weather.uwyo.edu/upperair/images/" + self.YEAR +
self.MONTH + self.FROM + ".08221.skewt.gif", CHARTPATH + "skewt.gif")</code>


Then, here there is another problem I has stopped on because I have to go for launch :)
The problem is that the URL build to retrieve the weather picture (for example "weather.uwyo.edu/upperair/images/2020091412.08221.skewt.gif") points to a document that simply doesn't exists...
So the URL must be changed, but now I am not sure what the program is looking for. I need some time to learn about the objective of this section of the code.
Due to other duties (like going to work) I can not continue reviewing the code right now but I could do it in two days if you can wait for so long.
I like to review python code, so I do it willingly.
The following user(s) said Thank You: Gonzothegreat
Last edit: 3 years 6 months ago by Joaquin. Reason: Typos
3 years 6 months ago #59934

Please Log in or Create an account to join the conversation.

  • Posts: 180
  • Thank you received: 30
I forgot:
I commented the tidy import because I do not have the tidy lib in my system, so I also nodified for example:

document = tidy.parseString(o)

to

document = o

This should be reverted in the code
3 years 6 months ago #59935

Please Log in or Create an account to join the conversation.

  • Posts: 2247
  • Thank you received: 223
I've found out that the following python scripts do die after a while... and I do not know why, so it's not quite yet fully solved.

26718 ? S 0:01 /usr/bin/python ./meteoRRD_updater.py &>/dev/null
26720 ? S 0:05 /usr/bin/python ./meteoRRD_graph.py &>/dev/null
26721 ? S 0:00 /usr/bin/python ./sounding.py &>/dev/null
26722 ? S 0:00 /usr/bin/python ./meteoRRD_MaxMinAvg.py &>/dev/null
26725 ? S 0:00 indi_weather -f /tmp/indi_weatherFIFO -p 7625
26726 ? S 0:00 \_ indi_duino
Last edit: 3 years 6 months ago by Gonzothegreat.
3 years 6 months ago #59947

Please Log in or Create an account to join the conversation.

  • Posts: 2247
  • Thank you received: 223

I was able to get that working with:
def get_sounding_skewt(self):
        data = urllib.parse.urlencode({"region": "europe", "TYPE": "GIF:SKEWT", "YEAR": self.YEAR,
                                       "MONTH": self.MONTH, "FROM": self.FROM, "TO": self.FROM, "STNM": SOUNDINGSTATION}).encode("utf-8")
 
        # print data
        print("data A=", data)
        s = urllib.request.urlopen(
            "http://weather.uwyo.edu/cgi-bin/sounding?", data)
        o = s.read()
        s.close()
        document = tidy.parseString(o)
        print("document A=", document)
        urllib.request.urlretrieve("http://weather.uwyo.edu/upperair/images/"+self.YEAR +
                                   self.MONTH+self.FROM+".08221.skewt.gif", CHARTPATH+"skewt.gif")
        # print document


check out the two extra "print()" to show the following:

data A= b'region=europe&TYPE=GIF%3ASKEWT&YEAR=2020&MONTH=09&FROM=1500&TO=1500&STNM=10548'


and:
document A= <!DOCTYPE html>
<html>
<head>
<meta name="generator" content=
"HTML Tidy for HTML5 for Linux version 5.6.0">
<title>10548 Meiningen Sounding</title>
</head>
<body bgcolor="white">
<center><img src=
"/upperair/images/2020091500.10548.skewt.parc.gif">
<p>Description of the <a href="/upperair/indices.html">sounding
indices</a>.</p>
<form><input class="button" type="button" value=
" Close this window " onclick="window.close();"> <input class=
"button" type="button" value=" Select another map " onclick=
"window.blur();"></form>
<hr size="1">
<i>Interested in graduate studies in atmospheric science? Check out
our program at the <a href="http://www.uwyo.edu/atsc/howtoapply/"
target="_top">University of Wyoming</a></i>
<hr size="1">
<font size="-1">Questions about the weather data provided by this
site can be addressed to <a href="mailto:ldoolman@uwyo.edu">Larry
Oolman (ldoolman@uwyo.edu)</a></font>
<hr size="1">
<script type="text/javascript">
<!--
window.focus();
// -->
</script></center>
</body>
</html>
 
3 years 6 months ago #59987

Please Log in or Create an account to join the conversation.

  • Posts: 2247
  • Thank you received: 223
  • Posts: 2247
  • Thank you received: 223
I think I just realised that the 08221 is the Madrid sounding station and it was hard coded.

weather.uwyo.edu/cgi-bin/sounding?region...0&TO=1500&STNM=08221
"Sorry, unable to generate 08221 LEMD Madrid Observations at 00Z 15 Sep 2020"

And at weather.uwyo.edu/upperair/sounding.html (europe) there is nothing available in Spain...
3 years 6 months ago #59989

Please Log in or Create an account to join the conversation.

  • Posts: 2247
  • Thank you received: 223
I have now fixed the sounding.py script and I pushed my updates to my repo github.com/Trigger-broom-289/indi-meteos...a7949ad5ec2c87720b30
Last edit: 3 years 6 months ago by Gonzothegreat.
3 years 6 months ago #59990

Please Log in or Create an account to join the conversation.

  • Posts: 2247
  • Thank you received: 223
Going to see what's going on when the scripts dies after a while, using the following command to start it:
nohup ./startMETEO.sh >> /home/heidenrod/meteostationWEB/startMETEO_out.log 2>>/home/heidenrod/meteostationWEB/startMETEO_err.log &

It never died by itself under Ubuntu 18.04 and Python 2.7
3 years 6 months ago #59994

Please Log in or Create an account to join the conversation.

  • Posts: 2247
  • Thank you received: 223
so far the scripts are holding, nothing has stopped. Maybe there is something different in Ubuntu 20 :huh:
3 years 6 months ago #59998

Please Log in or Create an account to join the conversation.

  • Posts: 2247
  • Thank you received: 223
Finally one of them has stopped and I see in the log file the following:
2020-09-15T14:22:58: Client 5: read: Connection reset by peer
[b]free(): invalid pointer
[/b]2020-09-15T14:23:09: Client 5: read: Connection reset by peer

script name is meteoRRD_MaxMinAvg.py
3 years 6 months ago #60001

Please Log in or Create an account to join the conversation.

  • Posts: 180
  • Thank you received: 30
As promised, a refurbished, shiny sounding.py (in a zip. The system doesnt allow me to attach a .py file)
Now everything works as I understood was expected.
File has been documented up to a level and code reformatted for robustness and efficiency.
I also ran a static code analysis (pylint):
The original file produced a global score of -8.93/10 :ohmy:
The refurbished file scored 9.85/10 :woohoo:
The following user(s) said Thank You: Gonzothegreat
3 years 6 months ago #60081
Attachments:

Please Log in or Create an account to join the conversation.

Time to create page: 0.611 seconds