744 Forum

Apron => Hangar 7 => Topic started by: RRVyper on Fri, 12 Apr 2019 16:29

Title: METAR Download Issues
Post by: RRVyper on Fri, 12 Apr 2019 16:29
Hi!
I was wondering if anyone else is having issues getting the weather to download. The last couple of days the METAR would begin to download but hang at a certain point (26% yesterday, 9% today). Yesterday I restarted PSX (10.76) and it still would finish getting the weather. Even, un-ticking the download box and re-ticking it did not help. I ended up shutting down PSX, deleting the METAR files and restarting PSX. It worked fine after that. I closed PSX at some point and resumed the flight today. Now I am having further issues. I updated the Sim time to current time but noticed METAR download stalled at 9%. After about 1/2 hr the Instructor/Weather page shows Disk: 1.9m/1.3m 11 Apr 2019, 2312z (next STBY). The site (https://tgftp.nws.noaa.gov/data/observations/metar/cycles/) shows good data for the nearest stations for 12 Apr (it is currently 1529z) but the latest METAR file in the Weather folder (2019041213.metar) is only 146KB in size. It seems like PSX can't get the complete file even though I can download it myself in the browser.
I am running FSX on the same laptop as a scenery generator via VisualPSX so Server and Boost are active. Could this be interfering?

Any ideas/suggestions?

Randy
Title: Re: METAR Download Issues
Post by: asboyd on Fri, 12 Apr 2019 23:21
I am running PSX on one machine with boost and main server on and it takes a few minutes to get the full download but it does happen...
So a little slow but OK...

Regards,
Alex B
Title: Re: METAR Download Issues
Post by: RRVyper on Sat, 13 Apr 2019 16:11
Thanks, Alex. I just fired up PSX by itself and it downloaded fine. Strange. I'll keep an eye on it in the future. I was also having an issue keeping FSX connected to VisualPSX yesterday after I posted (all running on the same laptop). Maybe the issues were related or I was overtaxing the machine.

Randy
Title: Re: METAR Download Issues
Post by: asboyd on Sun, 14 Apr 2019 00:07
Are you using windows 10 if so they pushed out some heavy security updates this week and that slowed my machine (and of course internet) down... I normally get 50Mb/s on the net connection but during this I found my CPU and disk were getting hammered so everything (including Outlook) were very slow to the point of seeming like the PC locked up.

Regards,
Alex B
Title: Re: METAR Download Issues
Post by: RRVyper on Sun, 14 Apr 2019 18:51
Interesting. Yes, I am using Windows 10. I didn't notice any other activity. I'm getting issues today as well. Download is stuck at 15%. I'm just sitting at the gate (CYWG) with power off. I'm on Wi-Fi with a strong signal and see no issues when surfing the Net. Again, Main and Boost servers are running (with VisualPSX and FSX-SE). Last METAR file in the Weather folder is 2019041416.metar but even after 15 minutes, it only shows 331kB as the file size. So it appears there is still an issue here. Task Manager Wi-Fi shows minimal activity and CPU and Graphics are below 50%. 😒
Title: Re: METAR Download Issues
Post by: Holger Wende on Sun, 14 Apr 2019 20:31
Hi

Just in case you have old weather URLs in your preferences files.
Check the thread under http://aerowinx.com/board/index.php?topic=4116.

Regards, Holger
Title: Re: METAR Download Issues
Post by: asboyd on Sun, 14 Apr 2019 21:59
Randy,
I am using the "new" url and I just tested again from YSSY (Sydney Aus) it took almost 1 minute on an 80Mb/s cable connection to download the full file, so maybe the Metar server has slow response times and may appear "hung".

Regards,
Alex B
Title: Re: METAR Download Issues
Post by: RRVyper on Mon, 15 Apr 2019 00:11
Hi

Just in case you have old weather URLs in your preferences files.
Check the thread under http://aerowinx.com/board/index.php?topic=4116.

Regards, Holger
Thanks, Holger. I mentioned the URL in my first post. 👍

I had to shut down PSX, delete the files in the Weather folder and restart PSX. It downloaded fine after that, but that's the second time I've had to do that. I'm on 300 Mbps cable so it should come down pretty fast. 😉

Randy
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Mon, 15 Apr 2019 03:56
The files in the Aerowinx/Weather folder have no influence on your download speed.

You wrote you had no issues when running PSX solo. So the problem seems to lie in your network system.

I don't recommend to use Wi-Fi to connect PSX network components. Your signal may be strong, but it may break nevertheless.


Regards,

|-|ardy
Title: Re: METAR Download Issues
Post by: RRVyper on Mon, 15 Apr 2019 16:54
The files in the Aerowinx/Weather folder have no influence on your download speed.
Agreed, Hardy. It's not the download speed that is the issue. It's that PSX sometimes hangs while downloading and sometimes won't recover. I reset the cable modem/router and will see how things go today. Also, I am not running this instance of PSX in a network config between computers; the network is only for running FSX as an external scenery generator, all on the same laptop.

Randy

P.S. Initial test today (running PSX solo) look promising - the 1st download completed in about 5 mins and the next one 2 minutes afterwards. It seems slow for a 2 MB file, but perhaps WX downloading is running at a lower priority than the rest of the PSX simulation. I will test with VisualPSX once I am satisfied PSX is OK by itself.
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Mon, 15 Apr 2019 17:58
Indeed, the download thread within the sim's multitasking system has a lower priority than the threads of the aircraft systems. This is intentional. The aircraft systems always have the highest priority to make the sim as smooth and fast as possible. The performance difference, however, is computer dependent. On my Mac I never see a difference higher than 5%, if at all. Maybe you have less multicore resources when you run FSX on the same CPU.


|-|ardy
Title: Re: METAR Download Issues
Post by: RRVyper on Mon, 15 Apr 2019 19:44
I allowed a few weather updates to happen on a flight from CYWG to CYVR then started FSX and VisualPSX, activated Main and Boost servers and launched FSWX, a weather program for FSX/P3D. There seems to be some 'interference' when FSWX is running as I will get occasional loss of connection between VisualPSX and FSX. And I noticed the weather files would also stop downloading in PSX (stuck at a Downloading... xx%). I shut down FSWX and had no more issues with the comms between PSX and FSX, but the weather never reinitialized - remained at xx% even after landing. Also tried toggling 'Set zones by flight track and downloaded METARs' on Weather tab and 'Allow METAR files download from Internet' in Preferences/Basics - no joy. It appears once the download hangs, it doesn't reset itself. Download speed on WiFi is 50 Mbps so the bandwidth is there. I'll check resource utilization next but will have to restart PSX as it is still stuck. I suspect I may be pushing this laptop a bit too much (Win 10/i7-8550U 1.8 GHz/Intel UHD 620 Graphics/12 GB DDR4/256 GB SSD).

Here is the error.txt log. It does seem to point to some network issue, I think.

[Aerowinx Precision Simulator - Exceptions]

java.net.SocketException: Socket closed
   at java.net.SocketInputStream.socketRead0(Native Method)
   at java.net.SocketInputStream.socketRead(Unknown Source)
   at java.net.SocketInputStream.read(Unknown Source)
   at java.net.SocketInputStream.read(Unknown Source)
   at sun.nio.cs.StreamDecoder.readBytes(Unknown Source)
   at sun.nio.cs.StreamDecoder.implRead(Unknown Source)
   at sun.nio.cs.StreamDecoder.read(Unknown Source)
   at java.io.InputStreamReader.read(Unknown Source)
   at java.io.BufferedReader.fill(Unknown Source)
   at java.io.BufferedReader.readLine(Unknown Source)
   at java.io.BufferedReader.readLine(Unknown Source)
   at NetServerThread.run(NetServerThread.java:210)

Randy
Title: Re: METAR Download Issues
Post by: Jeroen Hoppenbrouwers on Tue, 16 Apr 2019 01:18
I'm going off on a wild guess here. The download speed, or progress, also depends on the weather server. If for whatever reason that server decides you're too hungry it may temporarily block you, to give others a fair shot at the resources. If, say, FSX weather agressively downloads stuff, it may cause the server to temporarly block your IP and then PSX suffers.

It may be interesting to use a web browser to simulate weather download using the same URL as PSX and see what that does.

Hoppie
Title: Re: METAR Download Issues
Post by: asboyd on Tue, 16 Apr 2019 01:24
Accoring to PSX (my METAR settings are default for "live" weather) the download takes place every 23 minutes, FSX by default is every 10, so you may be onto something Hoppie... I use AS for P3D and I have it set for every 25 minutes so almost the same as PSX.... and have no problems, but my scenery is on a separate machine but they all use a single gateway so appear as though they come from one public IP....

Cheers,
Title: Re: METAR Download Issues
Post by: RRVyper on Tue, 16 Apr 2019 01:28
So, I began another round of testing this evening. Running PSX solo seems to work fine. I let it run for a few weather downloads and all was good. I was using the Basic 004 - Cleared for takeoff.situ for testing. Here are the steps and results:

Weather Folder Emptied
PSX Start Time 2034Z
WX download completed (2 files - 2019041519.metar, 2019041520.metar) at 2036Z (2.2m/1.8m Next 2115Z).
   Climbing through FL250: java.exe - Average CPU 27%
WX download completed (1 file - 2019041521.metar) at 2118Z (1.8m/1.4m Next 2143Z).
   java.exe - Average CPU 24%
   Total CPU around 50%
WX download completed (1 file - 2019041521.metar) at 2142Z (1.8m/2.2m Next 2210Z).
   java.exe - Average CPU 27%
Close PSX

Delete files in Weather folder
Restart PSX at 152149Z Apr (same .situ)
WX download completed (2 files - 2019041520.metar, 2019041521.metar) at 2036Z (2.2m/2.3m Next 2208Z).
Load FSX (B747-400 at TNCC Rwy 11)
Start VisualPSX at 2159Z
Start Main and Boost Servers
WX download completed (1 file - 2019041522.metar) at 2207Z (2.3m/1.1m Next 2243Z).
Climbing through FL250
   java.exe - Average CPU 38%
   FSX.exe - Average CPU 27%
   VisualPSX.exe - Average CPU 0.3%
   Total CPU around 82%
   Avg Network I/O 206 kbps
WX download initiated at 2243Z but not completed (paused at 86%, then stopped at 92%). No further downloads. Checked METAR site and updated files were there for 23Z and 00Z. Errors.txt was empty.

These files come down in about 2 seconds in a web browser so I'm not sure what the issue could be. Perhaps running the server is interfering. There's quite alot of data transferring from PSX to FSX (steady 200 kbps).

Something's not right.

Randy
Title: Re: METAR Download Issues
Post by: RRVyper on Tue, 16 Apr 2019 01:32
I'm going off on a wild guess here. The download speed, or progress, also depends on the weather server. If for whatever reason that server decides you're too hungry it may temporarily block you, to give others a fair shot at the resources. If, say, FSX weather agressively downloads stuff, it may cause the server to temporarly block your IP and then PSX suffers.

It may be interesting to use a web browser to simulate weather download using the same URL as PSX and see what that does.

Hoppie
Hi Hoppie.
For the last test I disabled FSX weather downloading (default used a different server which no longer works, hence the FSWX program). At any time during the PSX/FSX flight I am able to access the URL and download the weather files without issue.

Randy
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Tue, 16 Apr 2019 01:44
Quote
There's quite alot of data transferring from PSX to FSX (steady 200 kbps)

200 kB per sec? That's not normal. Way too much. What's your frame rate in PSX?


|-|ardy
Title: Re: METAR Download Issues
Post by: RRVyper on Tue, 16 Apr 2019 02:42
That's kilobits/sec, Hardy. PSX is showing 75 FPS.

Randy
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Tue, 16 Apr 2019 04:30
Still unusually high, I think.

The boost server sends about 64 bytes per frame.
The main server sends about 420 bytes per second -- plus some occassional peaks above 30 kByte.

64 bytes x 75 = 4800 bytes

So approx. 6000 bytes, i.e. ca. 48000 Bit per second.

Just for a test: Can you go to Instructor > Preferences > Basics, and set the frame rate limit to 48/3, and see if your network monitor still indicates 200 kBits?


|-|ardy
Title: Re: METAR Download Issues
Post by: RRVyper on Tue, 16 Apr 2019 14:10
My bad, Hardy. The Avg Network I/O of 206 kilobits/sec was for all network activity. When I said there was a steady 200 kbps between PSX and FSX, that was incorrect. It should have said ~5.8 kilobytes/sec, which is more in line with what you mentioned. Sorry for that. At 48/3 fps, I'm showing ~2.2 kilobytes/sec.

Randy
Title: Re: METAR Download Issues
Post by: RRVyper on Tue, 16 Apr 2019 18:23
Today I fired up the same flight along with VisualPSX, FSX and FSXWX and... no issues. I didn't do anything computer-wise since yesterday's flight. Weather download occurred on schedule without any hangs over 5 hrs, both in PSX and FSX.

Strange, but all is right again in the (sim) world. 🤞

Randy
Title: Re: METAR Download Issues
Post by: jcomm on Tue, 16 Apr 2019 19:07
Can incremental Win 10 updates be ruled out ?

There have been a few last week.

I've seen my networks going crazy due to it....
Title: Re: METAR Download Issues
Post by: RRVyper on Tue, 16 Apr 2019 23:06
Can incremental Win 10 updates be ruled out ?

There have been a few last week.

I've seen my networks going crazy due to it....
I didn't notice any heavy network traffic when I was having the issues. Even rebooting the computer and cable modem/router didn't help. My last Win 10 update was on 09 Apr. I'll keep an eye on it but it worked OK today.

Randy
Title: Re: METAR Download Issues
Post by: James H on Sun, 26 May 2019 15:02
Hi all,

I’ve noticed lately that I’ve been having similar trouble on my setup, namely the METAR download gets to some random percentage and then hangs. There doesn’t seem to be any consistent reason that I’ve noticed for it: sometimes it happens, sometimes it doesn’t, sometimes the download is VERY slow, and sometimes it’s almost instant. I assume this is because the connection between me and the METAR server gets interrupted somehow. I only tend to notice the METAR download has failed when the PSX ATIS differs from the VATSIM ATIS.

Is there a way to force the METAR download to restart while PSX is still running? Closing PSX, then deleting the weather files, then restarting PSX works, but this can be very inconvenient part way through a flight.

James
Title: Re: METAR Download Issues
Post by: Jeroen Hoppenbrouwers on Sun, 26 May 2019 18:37
Another technical approach would be to add a bit of monitoring code to the download thread, which notices that the data flow has stopped mid-flow. If there is no more progress after, say, 15 seconds, you can safely assume that something on the way from the server to the PSX thread has "forgotten" about the TCP link but did not report the problem towards PSX. This kind of network trouble is unavoidable anyway, so a monitor is always useful.


Hoppie
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Sun, 26 May 2019 19:52
On Instructor > Preferences > Basics, you can switch the checkbox "Download METARs ..." off and on at any time, even during a download. This will cancel the current download, and the next attempt will restart automatically several minutes later as usual.

However, this cancel function won't work if there is no progress at all. It needs to read at least one line feed from the downloading text to get into the cancel check.


|-|ardy
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Sun, 26 May 2019 23:49
I just added a timeout function provided by Java itself. This will be available in PSX 10.80.

I tested it with the timeout set to 1 millisecond, and this is the exception it throws (as expected):


Code: [Select]
[Aerowinx Precision Simulator - Exceptions]

javax.net.ssl.SSLException: Read timed out
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:137)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1180)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1091)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:187)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1581)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:246)
at WeatherLoader$DownloadThread.downloadOneFile(WeatherLoader.java:68)
at WeatherLoader$DownloadThread.run(WeatherLoader.java:157)


As this timeout test is OK, I'll set the value to 15000 milliseconds. Is that a reasonable time limit?

When the timeout occurs, the download will be canceled and the next attempt will start in a couple of minutes.


|-|ardy


What's a good setting for the restart time delay? Perhaps 3 minutes - plus some random seconds?
Title: Re: METAR Download Issues
Post by: Jeroen Hoppenbrouwers on Mon, 27 May 2019 01:05
15,000 ms sounds good to me for a non-critical far internet link. Let's see what the users find out. Making it configurable only will lead to confusion, I bet.

Restart delay, 3 +/- random is fine. I would not bother if it were 10 minutes. Even METARs are not accurate to the minute, that squall may be just ahead of you, that microburst just behind you.


Hoppie
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Mon, 27 May 2019 01:20
When a normal download starts, 30 minutes plus random time have already passed since the last METAR update, so the usual update interval plus random is already taken into account.

Compromise: 5 minutes :-)


|-|ardy
Title: Re: METAR Download Issues
Post by: James H on Mon, 27 May 2019 02:29
Outstanding once again Hardy!

Thanks,

James
Title: Re: METAR Download Issues
Post by: Hardy Heinlin on Mon, 3 Jun 2019 14:34
The timeout feature is now available in PSX 10.80:

http://aerowinx.com/board/index.php?topic=4191.0


Regards,

|-|ardy