Weird behaviour in PSX

[GodAtum]

https://youtu.be/uPnnl0_9njI

As you can see I am descending but the altitude line is level. This is a client connected to the PSX server/boost. After closing the client and reopening it again it was fine.

[Hardy Heinlin]

When the client is connected and not promptly synchronized with the server's flight situation, the client goes into smooth sync mode which is normal. E.g. if the baro pressure or the aircraft altitude is different, it slowly moves to the target value. If the difference is large, it will take some seconds. If something like that happens, there must have been a dropout in your network stream.

This is an effect of the altitude or baro variable shared via the network, not an effect of the internal aerodynamics, therefore it doesn't affect the V/S indication.

[GodAtum]

Not sure what is wrong with my network, or as you mentioned elsewhere the server being overloaded. I have:

1 PSX server + boost

SwitchPSX connects:
3 PSX clients
AdaptPSX
WidePSX
PSXACARS
FDS CDU
FDS Glare
PSX InterfaceIT

Maybe that's too much for something in there to handle?

I have found that when closing PSX server, the clients carry on. When I restart the server the clients do not pickup the server. Gary uses SwicthPSX successfully so not sure what is wrong with my setup?

Having a look at the PSX server resources, the 100% is surprisingly a 100%! https://imgur.com/a/4ma7fBH

[Hardy Heinlin]

I would start with the PSX server.

Then add PSX clients and watch the performance.

Then connect the next add-on and watch the performance. And so on ...

[GodAtum]

With everything running, performance was looking good on the Sydney approach situ (CPU around 50%). When suddenly Java crashed with PSX becoming a brown screen and the CPU dropping to 5%. The clients still said connected but could not click on anything. Even after restarting the clients, they still said connected but nothing was clickable.

I then restart PSX server + boost. The clients don't update unless I force them to by disconnecting them and reconnecting. Is that correct behaviour?

[Hardy Heinlin]

The correct behaviour is just start and forget.

What kind of computer are you using?

What network hardware?

[GodAtum]

PSX server is running in a VM with Intel X5570 @ 2.94GHz, 16GB RAM.

Clients are running in PCs with Intel i5-5300 @ 2.30GHZ and 8GB RAM.

Network is full GB with a Netgear switch.

[Hardy Heinlin]

You've been on the forum now since 2014 and, if I see it correctly, the sim worked OK before and you've changed a lot in your home system meanwhile. Do you remember what have been changed in your system shortly before that problem first occured?

[Jeroen Hoppenbrouwers]

The clients never noticed that PSX stopped. This is the equivalent of pulling the complete network out under them by cutting a cable so that PSX's network shutdown messages did not even get through. As PSX or Java cannot cause this kind of problem, because there is an Operating System in between them and the hardware guarding for all this, the issue is the OS or the VM.

Do PSX clients require some kind of keepalive? Do they auto-shutdown their connection (and possibly retry) when there has been no server sign of life for n seconds? Such a keepalive is about the only way I know of to detect sudden death in networks when the whole chain forgets about its state.

Hoppie

[Hardy Heinlin]

Each PSX client has a "checkError" function, but this is only executed whenever a client has written a message to the network. As long as the client isn't writing, the "checkError" isn't executed.

public boolean checkError()

    Flushes the stream if it's not closed and checks its error state.

    Returns:
        true if the print stream has encountered an error, either on the underlying output stream or during a format conversion.

When "checkError" returns true, the client will close its network socket.

To execute "checkError", you need to make the client write something. E.g. on the client's Instructor go to Situation > Human > Pilot and play with the checkboxes on that page. Each checkbox action will generate a message that will be sent into the network. If "checkError" reports an error to the client after such an action, the client will disconnect from the network.

Or, for example, the client also sends messages whenever the client's flight controls are being moved (by mouse, keyboard, or USB).

But this is just to answer Hoppie's question. This is not a problem nor a solution. It's just a description.

In addition to that, there's an option on Preferences > Basics: "This client reconnects after server error". -- When selected, it will retry to connect every 5 seconds.


|-|ardy

[Jeroen Hoppenbrouwers]

Yes, if you programmed it like this, you will only trigger a client disconnect/reconnect on a failed TCP write. And it usually takes a few seconds until the TCP/IP stack times out on the packet it sent, waiting for the TCP ACK that never comes.

If you want to catch broken cables and sudden death of VMs and OSes, you cannot avoid to either send or monitor some form of keepalive, even over TCP. Even an "empty" message from the client to the server would do, as long as it is a few network bytes; the server may ignore it completely. Sending keepalives will eventually time out the TCP stack and force a disconnect. Listening to incoming messages and timing out on a predefined gap also works and usually is a bit quicker.

Probably you could implement this without too much impact on anything, especially the "empty sends". Stupidly repeating a non-eco message every n seconds will do it.

If you want to test it, you cannot work with two PCs connected directly by Ethernet. Unplugging will signal to the OS that the link was lost and the OS will force the connections closed. Instead, you need to have at least one Ethernet switch between the PCs. If you disconnect the Ethernet on the far side of the switch, the PC on the near side does not get a Link Lost -- instead, the line just goes dead as the feed is broken, but the link to the switch is still there.

We had quite some trouble with this kind of @%#$@%#$@ in real aircraft, hence I know that the only way to fix it, is to monitor what happens with regular messages. Not every ms; every 5 seconds is fine.


Hoppie

[Markus Vitzethum]

Quick question by the way ...

> PSXACARS

Is PSXACARS = PSX BACARS .... or anything new? The search function did not come up with useful answers?

Markus