News:

Precision Simulator update 10.173 (24 February 2024) is now available.
Navburo update 13 (23 November 2022) is now available.
NG FMC and More is released.

Main Menu

NEW: PSX_Wheramium, a moving map

Started by martin, Sat, 21 Mar 2020 16:32

martin

Excellent!
Thanks for testing and reporting!

Cheers,
Martin

David Palmer

#101
Hi Martin,

Could this CPU and heating issue just be a consequence of using 'Cesium'?

I remember using an app on my last Android cell phone that used Cesium and the phone generated a ton of heat and the battery took a big hit after only a few minutes of app usage.

Regards,
David
Regards,
David.
a.k.a. 'The Commodore'

martin

Hello David,

it's quite possible, Cesium is a bit of a monster (to this dragon non-slayer at least). But when Google let the Google Earth browser plug-in die, Cesium was the "nearest" (free) alternative I could find. By now, anything else (if it exists) would certainly require a re-write from scratch, and that seems too high a price (esp. as we have full-fledged scenery generators like X-Plane and FSX as alternatives).

Cheers,
Martin

Avi

Hi,

i didn't re-read everything but, Martin, do you know what this is:



Thanks,
Avi Adin
LLBG

martin

Hello Avi,

this is what I always fear most: an error deep within the Cesium code, and probably not reproducible (if it were a "reliable" bug, it would occur several times every second or so...)

The only related thing I can see in the Wheramium code is that "tick" function (last line in the error message), a sort of timer which calls the next animation frame for the Cesium map. But it's impossible to say why in this specific case it runs into an error of this kind. (I have never seen it myself in all the development and testing work with Cesium.)

How often does this happen? Is it a singular glitch or a regular occurrence?
If you get that error a lot, I would have to make and send you a special version which puts out some more info about the variables at that point, which may or may not help diagnosing the issue. 

And which Java version are you running? This tick() function uses the Date functions built into Java, but I can't imagine that has changed drastically between versions. (And again: if that were the reason, the error would happen many times per second.)

That's all I can currently think of, unfortunately...

Cheers,
Martin


Avi

Hi,

The good news is that it works now but here is the full story.

First of all, in too many cases after I load the http://localhost... into Chrome it takes long time until Chrome reads the file (it can take good several minutes in the worst case), in other cases it is faster (all pre-conditions are met of course).

I did a flight yesterday and it was ok (except it kills PSX frame rate but this is a known thing) all the way until I parked and then few seconds latter it crashed and I got this error message.

Today it happened to me (both problems) anytime I tried to start it and at the end I gave up.
At some point I wish I had restarted my computer but it was too late at the time.
I did restart it now and checked it again and it works so I don't know what is going on.

Let wait and see

Cheers,

P.S. Just for the record I use Java 8 update 251 and run PSX_WH v02.

Avi Adin
LLBG

martin

Hello Avi,

Quote from: Avi...after I load the http://localhost... into Chrome it takes long time until Chrome reads the file
The Wheramium HTML page itself is of course on the local machine (and very simple), so that should load almost at once. The problem is probably fetching the map, which comes from the Cesium servers, so it's more of an Internet connection issue (I think).

Quote from: Aviit kills PSX frame rate but this is a known thing
But the Wheramium PSX client only reads the data sent out by the Boost server; that should not slow down the frame rate? Of course, an indirect effect is possible: fetching the animated Cesium map in the web browser can put quite some load on the computer (if it runs on the same machine as PSX).

Quote from: AviToday it happened to me (both problems) anytime I tried to start it
Now that is very strange, but I have no idea what the reason could be; I have so far never seen (or heard of) anything like this. And apparently it does not even hit you all the time...

Quote from: AviLet wait and see
Good idea! :D
Perhaps I should update my Java (I'm on 8, but an older version) and see, just in case, if I can then replicate the problem. It's if course also always possible that something has changed on the Cesium side; but if that is the reason, we should be hearing complaints from other Wheramium users, too, I suppose.

Cheers,
Martin



Avi

Quote from: martin on Tue, 29 Sep 2020 12:01
Quote from: Avi...after I load the http://localhost... into Chrome it takes long time until Chrome reads the file
The Wheramium HTML page itself is of course on the local machine (and very simple), so that should load almost at once. The problem is probably fetching the map, which comes from the Cesium servers, so it's more of an Internet connection issue (I think).

Maybe. I had Internet problems in the past few days.


Quote
Quote from: Aviit kills PSX frame rate but this is a known thing
But the Wheramium PSX client only reads the data sent out by the Boost server; that should not slow down the frame rate? Of course, an indirect effect is possible: fetching the animated Cesium map in the web browser can put quite some load on the computer (if it runs on the same machine as PSX).

I do run everythings on one machine.

Cheers,
Avi Adin
LLBG

Jeroen Hoppenbrouwers

Quote from: martin on Tue, 29 Sep 2020 12:01
But the Wheramium PSX client only reads the data sent out by the Boost server; that should not slow down the frame rate?

It can. If it connects to PSX and reads a bit and then stops reading, it may cause PSX to temporarily halt because it cannot get rid of its data. There are buffers in the PSX transmitter but it remains TCP, there are only so many packets that it will buffer up before Whereamium needs to fetch and ack the first of the queue. Depending on how PSX has been written, such a blocked pipe may or may not affect the tight inner loops.

TCP can only be used if pipes do not block on the reading end (for more than a millisecond). UDP, a different kind of beast, would potentially be more appropriate for a sensitive realtime data stream like that of the Boost Server. Missing a boost packet here or there is not so important, as the receiving simulator can merely skip them and sync up when ready again. This trick saves the day for pretty much all streaming services out there.

However the basic problem isn't the sender, but the receiver, so there is something that locks it up. Possibly you can add a gimmick that senses that "something else" is stopping your receiver and then temporarily unblock the pipe and throw away data until it can be used again? Or implement a separate thread that only receives and fills a buffer, with an overwrite function that throws away data older than a second?

All low-level sorcery that you likely wanted to avoid, I know. The blessings of network programming.


Hoppie

martin

#109
Quote from: HoppieIt can. If it connects to PSX and reads a bit and then stops reading, it may cause PSX to temporarily halt because it cannot get rid of its data.

Yes, some time ago we have discussed the issue, and I learnt that communication pipes can indeed be clogged up.
The practical result was Wheramium "version 02" (announcement here, documentation here),  now reading the PSX data as fast as it can, without pausing for any artificial "poll interval" (all of which may or may not be sufficient).

But could a pile-up in the comm.s pipe increase the load on the whole application to the point that even the frame rate suffers visibly?

Quote from: HoppieAll low-level sorcery that you likely wanted to avoid, I know.

I am still hoping I can: What's happening now is simply that the data are read and put into a variable "as is" (i.e. the whole string) and then read again and put into the variable and read... No brakes on that loop. End of story on the Java side/PSX client.

The Javascript side (web page) then fetches the value of that variable from the Java side/web server, and parses/processes it at its leisure, but the Java side/PSX client does not take any notice of that and is not affected by how fast or slow this happens.

Cheers,
Martin

PS (or possibly XP):
Quote from: HoppieUDP, a different kind of beast, would potentially be more appropriate for a sensitive realtime data stream like that of the Boost Server.
Fun fact: X-Plane 11 does use UDP to broadcast its data.


Hardy Heinlin

#110
Background info: PSX runs five independent looped threads; each at a different frequency: 0.3 Hz, 1 Hz, 5 Hz, 20 Hz, and the highspeed loop which is limited at 74 Hz.

The frame rate indicator in PSX refers to the highspeed loop. The highspeed loop executes aerodynamics, autopilots, graphics, and the booster output. So, yes, a clogged booster output buffer can slow down the highspeed loop.

When PSX copies data to the output buffer 74 times per second, PSX doesn't start 74 new threads per second. Instead, the data from the simulation is directly written to the output buffer. PSX will close the output when a network error is detected. But a clogged buffer is not considered an error. So it just waits until the buffer is free again.


Cheers,

|-|ardy


I guess the Wheramium code should run a secondary, independent, looped thread that contains its own variables which it copies to Cesium (the copy is performed via TCP/IP probably, so the copy-confirm process may be paused sometimes). The primary loop reads data from PSX and copies its values to the variables of the secondary loop thread. When the secondary thread slows down (due to delayed Cesium confirmations) it won't affect the primary thread; the secondary thread will just miss some data updates.

martin

Separation of threads was the other innovation in "version 02".
("2. Copying the PSX data over from the PSX client part to the web server part of Wheramium is now thread-safe (I hope).").
The communication to Cesium happens only on the Javascript side.

I just did a (rough and amateurish) test to time the speed at which Wheramium now reads the PSX data lines, and on  my box (i7-8700, 3.2GHz, 16GB RAM) the frequency is 73.8 lines per second which I think means practically as fast as they are sent from PSX,  with no risk of backlog or pile-up.

But I am still not seeing the Cesium problem reported by Avi, so no progress there.

Cheers,
Martin

Avi

If a reboot solve it, maybe it was somthing on my side.
Avi Adin
LLBG

Hardy Heinlin

Quote from: martin on Thu,  1 Oct 2020 17:17
The communication to Cesium happens only on the Javascript side.

Does the Javascript code include a TCP/IP data transfer to Cesium?


Cheers,

|-|ardy

martin

#114
Quote from: HardyDoes the Javascript code include a TCP/IP data transfer to Cesium?
Wheramium "just" uses Javascript functions from the Cesium library, but I don't know what protocol these then use behind the scenes.
I just had a look-around at doc.s, API, blogs, and so on, but nothing found so far, or they are not telling; it's all about the higher levels, i.e. Javascript and other web technologies.

However, it's Open Source, so I can and will dig around in the source code (or alternatively try and intercept the network traffic), but that will take more time. I'll update this post if I find anything relevant.

UPDATE: Now I've had a quick look at the main Cesium Javascript file (the ancient version I am using has only 156,600 lines) in the hope of finding some low-level communication stuff. No such luck, but what I saw looks as if HTTP requests are being used. Which naturally then begs the question what the HTTP protocol uses as the underlying layer.
To which the Hive Mind (a.k.a. Wikipedia) says: "Transmission Control Protocol (TCP) is commonly used. However, HTTP can be adapted to use unreliable protocols such as the User Datagram Protocol (UDP)."

Cheers,
Martin "Never look under the hood" E.

Hardy Heinlin

Quote from: martin on Thu,  1 Oct 2020 17:17
... the speed at which Wheramium now reads the PSX data lines, and on  my box (i7-8700, 3.2GHz, 16GB RAM) the frequency is 73.8 lines per second ...

That's actually a fine result; your primary loop thread seems to be unblocked now. Just to be sure it's not coincidence: Do you get 48 Hz when you set the PSX limit to 48? (Instructor > Preferences > Basics.)


Cheers,

|-|ardy

martin

#116
Quote from: HardyJust to be sure it's not coincidence: Do you get 48 Hz when you set the PSX limit to 48?

Ahhh, always some subversive questions/suggestions...
(It is bad practice to try and improve or extend results which already fit perfectly.) ;D
But luckily, here it turns out indeed that the read frequency perfectly fits the settings in PSX for the boost record generation frequency.
Here is the full science:
   
For each "Frame rate limit" setting in PSX, 1,000 (A) and 10,000 (B) data reads, resp., were timed.
(For the "73" setting, also 50,000 reads (C).)
The A and B (and C) results are mostly identical to the first decimal, but did differ in the second and third, not shown here.  ;) ).


Results:

  FPS            reads/sec   
setting        A       B    C
  73         73.8    73.8  73.8 
60/2        30.2    30.3   --
48/2        24.4    24.4   --
60/3        20.1    20.2   --
48/3        16.3    16.3   --


Cheers,
Martin "ReadSpeeder" E.

Josip Gracin

Hi Martin,

I'm having some issues with setting host IP and port. The values I set in .ini file seem to be ignored and the app always tries to connect to localhost:10749. (I determined this by inspecting the network traffic.)

I couldn't find the source code for Whereamium so I suppose it's not published. Have you considered publishing it? I'm a Java developer myself and I'd be happy to contribute (or at least try).

Thanks.

martin

Hello Josip,
Quote from: JosipHave you considered publishing [the source code]

Not really -- it is too much of an embarrassing mess intricately structured for that...  :-\

Besides, the problem you describe is a new one (i.e. I never heard about an issue like this before), so I have a hunch this is a network and setup question rather than an issue with the source code; I suggest that we pursue this line first. If need be, we can of course still dive into the source (about which I have forgotten all by now, so some archaeology will be required  ;D )

For starters, some additional information would help:

¤ As localhost apparently is not correct for you, you are probably trying to run things on more than one computer, but what exactly is the setup?

¤ Have you (just for a test) tried to run everything on the same machine with the default PSX ports, so that localhost:10749 "should" work?

¤ Moreover, are the ports in the *.ini file in agreement with what is set up in the PSX  Preferences/Network page?

¤ Is it possible that Wheramium is reading a wrong *.ini file from somewhere else (which still has the default entries instead of yours)?

(Sorry if these are very obvious questions, but experience (not least with myself...) advises that they still have to be asked.)

In any case, I'll dig out the source and have a look if/how non-localhost adresses are properly treated.

In the meantime, if anyone else is running PSX / Wheramium on more than one box, don't hesitate to let us know if/how it works for you.  ;)

Cheers,
Martin





Josip Gracin

Hey Martin,

Thanks for the response! I'll check everything you mentioned once again.