Author Topic: PSX Server redundancy? (Read 1231 times)

GodAtum · « **on:** Sun, 3 Jan 2021 16:01 »

I was wondering how professional customers make the PSX server redundant, so if it or the PC (by that I mean physical machine, VM or operating system) fails, it will carry on running? At the moment, if the PC running PSX server crashes, it will take the PSX server down with it, which must be unacceptable downtime in a professional environment with customers paying by the hour?

All I can think of is running another PC that takes a copy of the autosave situ file every 7 minutes, so if the 1st PSX server PC fails, one can start a PSX server on the 2nd PC with the latest situ file quickly, thereby minimising downtime.

Gary Oliver · « **Reply #1 on:** Sun, 3 Jan 2021 16:11 »

Having run PSX in an exceptionally complicated Sim environment since its release I have had 2 PSX server crashes..... Both of these with badly written addons or doing something entirely stupid when testing out stuff.

Since adding SimStack Router in front of the PSX server on port 10746 so anything 3rd party talks to that and the PSX instances all talk to each other on 10747 I have not had a single PSX server issue.

I think you might be overthinking a problem that doesn't exist.

Regarding OS or hardware failures, not a single one on the PSX server. Its still running a windows 8.1 install with all updates disabled that was installed when I installed the PC in the sim.

Windows 10 LTSB with patching disabled on all other PC's.

Cheers
G

Jeroen Hoppenbrouwers · « **Reply #2 on:** Sun, 3 Jan 2021 16:38 »

I'm not even sure whether professional simulators have any true redundancy. Yes they are expensive to run, and there are most certainly safety features, but nobody gets hurt when the thing freezes in mid-vlight.

Furthermore I don't know whether there are any pro sim setups (commercial) running PSX? That would be interesting to know.

Lastly a lot of pro PSX use is classroom education, and the teacher's PC most certainly won't be redundant.

Hoppie

Hardy Heinlin · « **Reply #3 on:** Sun, 3 Jan 2021 17:57 »

Some airlines use PSX also for a multimonitor cockpit mockup.

GodAtum · « **Reply #4 on:** Sun, 3 Jan 2021 19:10 »

Thanks for the info all, very interesting.

The reason I ask is that I've finally got round to fixing my setup this holiday. Either, on 50% of flights, the PSX server window goes all brown, and I have to force close it, then restart.

Or the PSX server keeps running, but the clients go out of sync. So they say they are connected but all the data is out of sync. When I try and reconnect, it says no server found on port 10747. When I run a Powershell command "Test-NetConnection <host> -port 10747" it errors. When I try to restart the server in the instructor station window it errors saying port 10747 is not available. The only way to fix this is to restart PSX entirely.

I am using Java 1.8.0_231 on Windows 10.

Hope someone with a lot more knowledge can offer some advice

Hardy Heinlin · « **Reply #5 on:** Mon, 4 Jan 2021 04:55 »

GodAtum, for a test, can you make some flights without any add-ons? Just use your computers to run some networked PSX instances. Pure PSX server with pure PSX clients. No routers, no add-ons.

Will that fail as well on 50% of flights? If not, you may continue the test with more and more add-ons and then see at which additional add-on the problem starts.

|-|ardy

GodAtum · « **Reply #6 on:** Mon, 4 Jan 2021 08:45 »

Quote from: Hardy Heinlin on Mon, 4 Jan 2021 04:55

GodAtum, for a test, can you make some flights without any add-ons? Just use your computers to run some networked PSX instances. Pure PSX server with pure PSX clients. No routers, no add-ons.

Will that fail as well on 50% of flights? If not, you may continue the test with more and more add-ons and then see at which additional add-on the problem starts.

|-|ardy

Good idea, I'll do that thanks

GodAtum · « **Reply #7 on:** Mon, 4 Jan 2021 22:12 »

PSX worked Ok with no addons.

When I added SimStack with the FDS Glare software, the client got out of sync. On the client GUI, when I try to change a dial like the heading or alt, it won't update the value on the client, but will on the server. Same with the CDU, if I press a button, the display updates on the server but not on the client.

On the client, when I stop and start the server connection, it fails with "No server found on port 10747". I then need to restart the Server for it to resync.

Like Gary, I have my PSX client pointed to the server. And the addons like FDS Glare pointed to SimStack.

Jeroen Hoppenbrouwers · « **Reply #8 on:** Mon, 4 Jan 2021 23:05 »

We will need a bit more detail on what is connected to what. If you say "client", you probably mean the PSX Client. But where is the SimStack software located? Does that drive the PSX Client from hardware? Does the hardware lose sync, or the client, or both? Is the SimStack software controlling SimStack hardware, or is the FDS Glare something completely independent?

A simple block diagram may help here.

Hoppie

GodAtum · « **Reply #9 on:** Tue, 5 Jan 2021 09:32 »

At the moment, I'm just testing psx-server (with only PSX server & SwitchPSX running), psx-boot and avionics-a (with only PSX client & FDS Glare running).

hoppie added the width=700 configuration

GodAtum · « **Reply #10 on:** Wed, 6 Jan 2021 16:11 »

Doing some more testing. When using only the PSX Server and PSX clients, the PSX Server connection dies after an extended period.

As you can see from the screenshot, PSX server is running, but 127.0.0.1 port 10747 refuses connection. When I restart the PSX server, I can then make the connection. Windows firewall is completely off.

Hardy Heinlin · « **Reply #11 on:** Wed, 6 Jan 2021 16:30 »

"127.0.0.1" is "localhost".

PSX clients connect to "localhost" if they are on the same computer as the PSX server.

Are your PSX clients on the same computer as your PSX server?

GodAtum · « **Reply #12 on:** Wed, 6 Jan 2021 16:35 »

Quote from: Hardy Heinlin on Wed, 6 Jan 2021 16:30

"127.0.0.1" is "localhost".

PSX clients connect to "localhost" if they are on the same computer as the PSX server.

Are your PSX clients on the same computer as your PSX server?

I'm keeping it simple and just testing the local connection on the same PC in the screenshot. When trying to connect to 127.0.0.1 port 10747, it is refused, even though the PSX server is running.

Hardy Heinlin · « **Reply #13 on:** Wed, 6 Jan 2021 17:45 »

What's in Aerowinx/Logs/errors.txt?

GodAtum · « **Reply #14 on:** Wed, 6 Jan 2021 18:22 »

It's empty.

Hardy Heinlin · « **Reply #15 on:** Thu, 7 Jan 2021 02:07 »

On your screenshot I see 4 client buttons. The 4th client has the number 1000. What kind of client is that? It disconnected and reconnected 1000 times during this test run. For performance reasons, the PSX server doesn't allow more than 1000 connections. It will deny further connection attempts. To reset this limit, you need to cycle the checkbox to "Disconnected" and back to "The main server".

It's not normal when an add-on connects thousands of times within a few hours. Is this add-on on the same computer? If not, are the network wires OK?

|-|ardy

Jeroen Hoppenbrouwers · « **Reply #16 on:** Thu, 7 Jan 2021 10:21 »

Extremely useful feature, Hardy. Consider coloring the button differently when this limit is reached, as it is too easy to overlook if you don't know it's there.

Hoppie

Hardy Heinlin · « **Reply #17 on:** Thu, 7 Jan 2021 11:32 »

Good idea ...

GodAtum · « **Reply #18 on:** Sun, 10 Jan 2021 18:16 »

Thanks so much Hardy, it seems obvious now!

Markus Vitzethum · « **Reply #19 on:** Tue, 12 Jan 2021 14:51 »

Hi Hardy,

> The 4th client has the number 1000. What kind of client is that? It disconnected and
> reconnected 1000 times during this test run. For performance reasons, the PSX server
> doesn't allow more than 1000 connections. It will deny further connection attempts.
> To reset this limit, you need to cycle the checkbox to "Disconnected" and back to "The
> main server".

interesting - I've experienced this issue before. It is documented in this message which includes a video:

https://aerowinx.com/board/index.php?topic=3629.msg61207#msg61207

In my situation the affected add-on was Externalsim.
However, I don't see Externalsim in Godatum's setup diagram (using WidePSX instead).

What can possibly go wrong that different external programs fall into a reconnect loop?

Markus