Feature request: override screen size, disable layout rescale/fit on screen

[Daniel Neugebauer]

It isn't urgently needed but since it was quiet regarding PSX updates over the past months I thought I may as well request the feature now.

I started working on connecting PSX into a virtual cockpit last year already*. One issue I found is that there does not seem to be any way to let PSX ignore actual OS display resolutions and to allow using oversized windows. For the VC connection I built a layout that fits all 6 "main" displays (PFDs, NDs, MFDs) on screen in the resolution the target VC needs. The window gets captured, split and rearranged to transport the texture into the other ("visual") flightsim running on the same PC (so simply increasing the desktop size beyond actual display resolution is impractical). Unfortunately the PSX window (at least on Linux) always rescales ("too big for this screen (will be shifted and cropped)") the layout to fit on screen (2239x1508 on 3440x1440 minus task bar and window title = 2239x1386). Attempts to circumvent that via the window manager (KDE Plasma/kwin) resulted in PSX still ignoring the extra space when drawing. On Linux there is a workaround (using a nested X11 session via Xephyr) but I loose OpenGL in that case and get some performance issue when capturing that window (PSX slows to half frame rate; that doesn't happen for the native window).

If there isn't already some hidden option that can be set (or I missed something else), then I would like to request to be able to either

- provide an override for the detected display/screen resolution to PSX or
- have an option to simply disable the "resize to fit on screen" feature so that PSX simply uses the original screen size defined in the layout and draws into whatever size the window currently has.

Such options may also benefit hardware cockpit builders (who probably use some similar technique already). Other workarounds (system-independent) would be to either use a reduced PSX display resolution (which would degrade quality in a VC) or to run two PSX sessions, each rendering just 3 displays (in addition to yet another session already required for the 3 CDUs).

I would be very happy if one of the options could be implemented as it would make it a little bit easier to use the VC connection once it is done*.


*) Please don't ask about the VC project progress - it is still a long way until the first release as I got side-tracked by one depending and one unrelated project this year and I also only work on these projects in my spare-time. I really cannot give any time estimates other than that it would at earliest be ready the next summer, that's also why the feature request isn't urgent.

[Hardy Heinlin]

That's easy. I'll add this option. But I cannot guarantee that it will work on every OS and every Java version. MacOs, for example, may crop the window also.


Regards,

|-|ardy

[Daniel Neugebauer]

Great, thanks a lot!

[Hardy Heinlin]

Unfortunately the PSX window (at least on Linux) always rescales ("too big for this screen (will be shifted and cropped)") the layout to fit on screen (2239x1508 on 3440x1440 minus task bar and window title = 2239x1386).

Just a system test and a question:
• Your monitor is circa 3000 pixels wide.
• Your PSX frame is circa 2000 pixels wide.
• You move your PSX frame to the left beyond the left monitor border so that 1000 pixels of the PSX frame width are hidden beyond the left monitor border, and 1000 pixels are visible within the monitor area.
• Now you stretch the right edge of your PSX frame by circa 1000 pixels so that 2000 pixels of the PSX frame width are visible (and 1000 are still hidden).

Question:
Is the right half of that visible 2000 pixel wide PSX area (the part that you stretched) solid brown?


Regards,

|-|ardy

[Daniel Neugebauer]

Is the right half of that visible 2000 pixel wide PSX area (the part that you stretched) solid brown?

On my system PSX renders everything as it should be: https://www.energiequant.de/images/foren/aerowinx/2023/10/3000_shift_test.jpg (OBS is shown to confirm that the window contents still render off-screen; please ignore the coordinates/dimensions shown in OBS, also on the following screenshots)

That's on Linux using Adoptium/Temurin JDK 1.8.0_362-b09 on KDE Plasma 5.27.8, X org 21.1.8, Nvidia drivers 535.113.01 (GeForce 1070).

I can extend the window further until PSX doesn't let me continue: https://www.energiequant.de/images/foren/aerowinx/2023/10/full_width_shift_test.jpg

I only get solid brown while I resize the window or if I force a maximum window size through the window manager (overriding PSX's window configuration outside PSX) that is larger than the actual display size: https://www.energiequant.de/images/foren/aerowinx/2023/10/forced_oversize_test.jpg (that's one of the workarounds I attempted)

Edit: And just to be sure I confirmed again that oversized off-screen rendering (in general) actually works with other applications (but I didn't try it with Java) - for example I can have an extremely oversized Chrome window, also displaying hardware accelerated 3D graphics and still capture everything:  https://www.energiequant.de/images/foren/aerowinx/2023/10/oversize_chrome_test.jpg

[Hardy Heinlin]

Thanks. OK, so you just want to get rid of the frame size limiter. Your flightdeck graphics fill a 3000 pixel wide PSX canvas even when the monitor width is smaller than 3000. (PSX by default sets its image buffer size to the size of the monitor which can be a single monitor or the sum of multiple monitors if "allow stretching across monitors" is selected.)

[Daniel Neugebauer]

In case it causes too much trouble to adapt the canvas size "dynamically" if a layout would need a larger screen resolution then I would also be happy if I could just override it once during startup (CLI or config option) by specifying the wanted "virtual" display size for the sim session somewhere.

[Hardy Heinlin]

I can add that later if it's really required. Let's see if that "frame size unlimiter" alone solves your problem already. The new PSX version is finished now and I'll upload it tomorrow.

[Hardy Heinlin]

I forgot to ask: Did you select the checkbox "Allow frame stretching across multiple monitors"?
It's on Instructor > Preferences > Basics.

[Hardy Heinlin]

The new option is now available in PSX version 10.172:

https://aerowinx.com/board/index.php/topic,4191.0.html


Regards,

|-|ardy

[Daniel Neugebauer]

I forgot to ask: Did you select the checkbox "Allow frame stretching across multiple monitors"?

I had that option activated but I also didn't see any difference if I disabled it (I only have a single monitor connected).

The new option is now available in PSX version 10.172

I've tested version 172 and set CustomCode=128 (it was 0 before) in my preferences. I see that it has some effect but unfortunately it's not what would be needed:

1. The maximum window size is still limited to my display resolution (3440x1440) unless I also override the maximum size via KDE (same as before). Other windows (incl. the instructor panel) can be larger than my screen size without such workarounds, so it appears that PSX still limits the window size.

2. The "too big" warning is gone from the layout tab but when loading/switching to a layout it still seems to behave like it did before - the window still scales down to the maximum available on-screen size (display resolution minus task bar and window title). In addition to loading the screen layout I have to manually override the current (and maximum) window size outside of PSX.

3. If the maximum window size is overridden, I can see the buffer has been extended to 1.3x my display size (beyond that it's solid brown). That would be enough to render the needed layout with a sufficiently high display resolution as on my system but it wouldn't be enough for other users who are for example only using 1920x1080 displays (1080 x 1.3 = 1404 which is less than the required 1508 layout screen height).

I uploaded the layout file, in case you want to try the exact use-case: https://www.energiequant.de/files/psxvc/Remote_231021.9pack

For a virtual cockpit integration layout 1 from that file would need to be rendered full size (2239x1508) independent of the actual display resolution on a user's system. Even if the user would only have a native 1920x1080 resolution then PSX should still use a window with inner dimensions of 2239x1508 so that the layout appears exactly like it was defined (1:1) and positions do not get shifted. The layout is not fully optimized yet but from what I estimated it would still exceed my screen height even in best case. For users with standard HD resolution displays it would also exceed the available screen width.

Regarding the fixed buffer size factor: Virtual cockpit integration will be released publicly (as open source) when it's done, so it's not limited to my system. While it probably cannot be an easy turn-key solution and will require some preparations to be made by users I don't think they can be expected to have ultra-wide or 4k displays just to have the correct buffer size for rendering the required layout for screen capture (within a single PSX instance running in the background). The layout is pretty much fixed to avoid scaling or quality degradation in the VC and to be able to provide users with correct texture mapping. A fixed factor for the buffer size based on display resolution doesn't reliably work in that case, it would be better if users could set an absolute buffer size for the exact use-case/layout.

The current PSX layout is specific to one aircraft model (X-Plane 11 default B744). If another 3D model would be used, then the VC panel texture may be different, requiring another layout in PSX (which might be even larger), so the window/buffer size would need to be freely controllable by users.

[Hardy Heinlin]

I had that option activated but I also didn't see any difference if I disabled it (I only have a single monitor connected).

When you selected and deselected that checkbox, did you also make some actions like stretching the frame, reloading the 9pack file, restarting PSX?

1. The maximum window size is still limited to my display resolution (3440x1440) unless I also override the maximum size via KDE (same as before).

Does that also happen with the width or just with the height?

On my macOS I can't make the frame height greater than the monitor height. This affects all applications, not just PSX. I move a full-height frame down so that I get a headroom above it. When I'm dragging the upper frame border towards the top of the monitor, the frame height is increasing with it. But when I release the mouse button, the frame height resets to monitor height, and the frame jumps to the top. I thought this to be macOS specific and that your Linux might behave differently. Well, maybe it's Java specific. But, as I just wrote, on my macOS it happens on all applications.

This problem does not occur with the width. Can you oversize your PSX width?

If the width can be oversized, can you use that horizontal addition instead of the vertical addition?

For example, like this: Upper row for 6 EFIS/EICAS sized screens (large stuff), lower row for 3 CDU screens (smaller stuff):





Another question:
Do you really need all those brown PSX panel pixels above and below the EFIS screens? Or do you fill those areas with brown panel stuff from X-Plane? Did you know that the brown bezels can be removed in PSX? (Instructor > Preferences > Screen: Checkbox "EFIS/EICAS/CDU bezels removed".)

[Daniel Neugebauer]

When you selected and deselected that checkbox, did you also make some actions like stretching the frame, reloading the 9pack file, restarting PSX?

Yes, but I didn't notice any change in behaviour.

Does that also happen with the width or just with the height?

Both, when I reach an inner window size of 3440x1440 the window just stops expanding (tested by moving the window partially off-screen by Alt+Drag, then dragging the window frame). It does work for the instructor window, though, so it can't be a general limitation of Java/Swing, assuming that the main window is also managed through Swing. If there's some (other) LayoutManager on the main window or a single Swing component is used without a LayoutManager around it, maybe that is restricting the main window from growing further, unless it gets overridden by the OS window manager outside PSX? In case the restriction comes from inside the JFrame, then whatever component/manager may tell the OS to limit the window size still works fine if that restriction gets lifted outside PSX and the window receives a larger-than-expected size. Maybe the JFrame receives an unintended call to setMaximumSize/setMaximumBounds or needs those values to be explicitly set if not called before?

The general OS-side window size restriction may actually only be present on MacOS; I didn't test it but I'm sure I've seen that Windows doesn't care about display sizes either if you move a window partially off-screen and then resize it from the upper or left border in the way you described it for MacOS.

This problem does not occur with the width. Can you oversize your PSX width?

If the width can be oversized, can you use that horizontal addition instead of the vertical addition?

It's only possible after manipulating the "maximum size" for the window in KDE, thus unlocking the window restriction sent to the OS window manager but then it works in both directions (vertically and horizontally).

Even with the "maximum size" override in place PSX still will not use the full layout size when reloading/switching the layout.

Do you really need all those brown PSX panel pixels above and below the EFIS screens? Or do you fill those areas with brown panel stuff from X-Plane? Did you know that the brown bezels can be removed in PSX? (Instructor > Preferences > Screen: Checkbox "EFIS/EICAS/CDU bezels removed".)

Bezels get disabled for capturing into the VC, the screenshots were just not showing the exact setup I would use for VC integration. This is what it looks like in a nested X session (virtual screen size of 2239x1508) and how layout 1 from the "Remote" 9pack should appear if the window can grow to the full layout size:

https://www.energiequant.de/images/foren/aerowinx/2023/10/with_bezel.png
https://www.energiequant.de/images/foren/aerowinx/2023/10/without_bezel.png

Edit: And this is the remapped texture in a debug-window (incl. window frame), inner size 2048x2048, written also with Swing: https://www.energiequant.de/images/foren/aerowinx/2023/10/remapped-viewer.png

[Hardy Heinlin]

The problem is I can't reproduce the width limit problems you're seeing. But I'll recheck if there are any other hidden limits that maybe only macOS ignores.

In case I can't fix the problem: How about this?
• Make a frame half as high.
• Start a second networked PSX instance so that you have two frames, each with that smaller height.
• Place them whereever you want.

[Daniel Neugebauer]

I already tried and/or thought of the same workarounds before making the request.

The original idea was to run PSX in a Docker container on a networked PC (with any arbitrary virtual screen size) and capture it via VNC. FPS weren't satisfying enough for PFD/ND/MFD so instead I moved it to the physical machine (using Intel on-CPU graphics acceleration) with an arbitrary screen size (no display connected). Rendering performance was fine but there still was a noticeable lag due to VNC (bandwidth also became a concern because I store the scenery on that machine), so I went with local capture instead which has almost no latency. The only issue with that is that the PSX window size is limited, so I ended up using a nested display server (Xephyr). Unfortunately, even with existing OpenGL proxy solutions (which don't seem to work in Xephyr or are not picked up by Java), I wasn't able to resolve rendering performance issues (it's still okay on my machine but it could be better). Also, that setup is very specific to Linux and (maybe even worse) to X as a window system (which is getting replaced by Wayland). It also exhibits some performance issues when capturing the nested X session (cutting FPS in half). The resulting frame rate is still okay but marginal. That issue is not present when PSX runs on the main X session.

What I got working at the beginning of this year: https://www.youtube.com/watch?v=UqVPaooVDvw (CDUs can be combined from a second PSX instance, just not shown in that video; only Linux, only displays are transferred, no input, no movement; no progress since then as I started work on a dependency first)

Using two instances (one for the upper half, one for the lower half of the layout) is of course possible but "wastes" one PSX instance. If the required window width would still not be possible, then it would even need to be three instances (each showing 2 displays). You may then want another instance for the CDUs to be rendered with the original vector fonts and maybe yet another one to control the simulation from e.g. a notebook while the visual sim is running full-screen. That adds up to 4 or 5 instances running at the same time. I would need to look it up again but I think the maximum allowed number of instances per license was 4, which would get exceeded in that case (also incurring unnecessary processing overhead in the overall system)?

So there are workarounds but they are not very practical. If it's possible to resolve whatever restriction is on the main window that would be greatly appreciated.

The window I posted in the edit of my post above is a basic JFrame holding a single JPanel that I directly draw on. The JPanel is created with just setPreferredSize(new Dimension(width, height)) followed by pack() on the JFrame. Maybe that helps to find the difference?

Alternatively (but that would probably be a major change to PSX?) it would also be helpful to be able to split the 4 views rendered inside the PSX main window into 4 separate windows, each showing just one of those views. That way all displays would fit on screen (regardless of the OS) and would just need to be grabbed individually. For the extraction into a VC the PSX views actually could be non-interactive as all interaction is done through network anyway (VC, another PSX instance or external tools).

[Hardy Heinlin]

Hi Daniel,

let's continue in the "Networkers" subforum which is a bit more private than "Hangar 7".

There I have a test version for you:
https://aerowinx.com/board/index.php/topic,7263.0.html


Regards,

|-|ardy