Broker Communication

[mikeindevon]

As part of the x-plane scenery generator for PS13 / PSX I have build software that gets the position data from the Broker periodically, fits a quadratic to it

say lat=a+b*t+c*t*t

and uses that formula to extrapolate the plane's position forward in time until the next data arrives from the Broker.

I time stamp the data on arrival in my program.  As part of the development analysis I have generated an output showing the "speed" of the plane, ie change in position divided by the time between the time stamps.

I have noticed that the speed varies quite significantly over very short periods, say 0.1 secs.  On the standard run into Sydney this can vary between 165 and 125kts.  Obviously a real plane's speed, even allowing for wind variation, would be much more constant than this.  I get this affect using both the "get" and "notify" methods.  In fact the "get" works better as any variation is spread over a longer time and is therefore less significant.  I see this effect even when all the software is running on one machine.  

I wonder if you guys have any insight into what is causing this.  My thoughts are that it might be:
- quantisation effects within the Broker polling
- variable delays in transmission (as I do the time stamping on arrival).

Any thoughts?

Related to this
- it would be useful to have a time stamp from PS with the data as this would remove the effects of network delays.
- it would be good to have the position data transmitted at a precise constant interval.  If this was done then the matrix inversion to calculate a, b, c above would have to be done only once during the entire simulation setting.

Mike

[Jeroen Hoppenbrouwers]

PSX will be totally different in all aspects, except that it is a socket (not even necessarily TCP). The 'fast update' position and attitude system is not even present yet, only the machine parameter system is.

The Broker was never intended to relay hard real time data -- it just happens to go a long way. It was built for one second data intervals!

The smoothing that Garry does for his scenery driver (and the motion platform) probably goes further than you assume right now. Don't try to push the data transfer -- you will saturate the Broker and make it worse. GET only makes sense if the frequency is less than 10 Hz or so. If higher, you will just collide with data arriving and compete for CPU. The Broker has absolutely no multithreading, it is one serial state machine, and it can easily get bogged down. It comes from a different century!


Jeroen

[mikeindevon]

Last word of my previous post should be session, not setting.

Jeroen, thanks for the info.  I am currently polling with "get"  every 600ms and doing 60 cycles of x-pane within that.  In that time, the plane has traveled some 40m and the extrapolation error over the 600ms is typically 0.7m with occasional jumps to 3-5m when the speed blips.  I will see what I get by throwing out rogue speed data as I go along.  Accuracy is about 0.3m at taxi speeds.

From your inside knowledge of the Broker, could you suggest an ideal polling rate?

Mike

[Jeroen Hoppenbrouwers]

Are you using flat Broker, or also IPC (with the associated different keys)? Flat Broker runs at 1 Hz updates, so anything faster is useless. IPC runs at 18.2 Hz which is the PS1 internal update rate.

I would always suggest to just use Notify. You don't get time stamps, but it will be more accurately reflecting the real data arrival time than any GET ever will. Polling is typically less efficient with the Broker and it was only provided for the once-per-hour data needs. I personally never ever used GET in any add-on.

Background: GET does not reach all the way into PS1. It just returns the last value the Broker got from PS1, either during the last 1Hz poll cycle (using the disk file) or via the IPC 18.2Hz push. So if you use GET, you retrieve out of date values all the time. With NOTIFY, you get real time data with a relatively fixed small delay.

Again, it was never meant to drive scenery generators -- you have to fly yourself and steer your simulated camera to track PS1. Garry will have loads of tips on this.


Jeroen

[Garry Richards]

Hi Mike,

These are the keys I use in VisualPS1 to make the FS aircraft follow PS1 smoothly:

            ReferenceKeys(0) = "gnd" 'ground speed in kt
            ReferenceKeys(1) = "IPC.latdeg" 'latitude deg integer S -ve
            ReferenceKeys(2) = "IPC.latmin" 'latitude min * 100000 integer
            ReferenceKeys(3) = "IPC.londeg" 'longitude deg integer S -ve
            ReferenceKeys(4) = "IPC.lonmin" 'longitude min * 100000 integer
            ReferenceKeys(5) = "IPC.talt" 'true altitude float
            ReferenceKeys(6) = "IPC.pitch" 'Pitch float Down -ve
            ReferenceKeys(7) = "GJR.Bank180" 'Bank float right down -ve
            ReferenceKeys(8 ) = "IPC.thdg" 'true heading float
            ReferenceKeys(9) = "IPC.elev" 'ground elevation feet integer

I tried using 'get' but it was very inefficient and I now use 'notify'. It only has to be called once and will supply fresh data whenever it changes.

I have a high speed timer (40 ms) calling the reading procedure for each line of Broker data. It may contain multiple values.

These raw data are then passed to another thread that smooths them and sends them to FS, again using a 40ms timing loop.

I have tried extrapolation methods but could not remove the jerkiness (just as you experienced) so I use a smoothing method based and improved on that in the original Visual744 from which VisualPS1 was developed.

The smoothing algorithm uses a circular buffer that efficiently maintains an average value, but introduces a slight lag in the output. Here is a code sample:

(The forum removes my tabbed indentation so this may be a little difficult to read.)

Code Select Expand

[SmoothingPositionUbound is 39]
[SmoothingPositionSize is 40]

            'Smooth ground speed

            'Subtract oldest value from sum
            GndSpdSum -= GndSpds(GndSpdCurrent)

            'Get current value
            GndSpds(GndSpdCurrent) = GroundSpeedBrokerUse

            'Add newest value to sum
            GndSpdSum += GndSpds(GndSpdCurrent)

            'Update newest value pointer
            GndSpdCurrent += 1
            If GndSpdCurrent > SmoothingPositionUbound Then
                GndSpdCurrent = 0
            End If

            'Get smoothed value
            GndSpdPS1 = GndSpdSum / SmoothingPositionSize 'average from selected array

            'Update smoothed value
            'Convert to FS value (same units as PS1 in this case)
            Try 'in case of initial invalid result
                GndSpdSmoothed = GndSpdPS1 'kt
            Catch ae As ArithmeticException
                GndSpdSmoothed = 0
            End Try

Hope this helps.



[edit by JH] I fixed the formatting

[Jeroen Hoppenbrouwers]

I tried using 'get' but it was very inefficient and I now use 'notify'. It only has to be called once and will supply fresh data whenever it changes.

I have a high speed timer (40 ms) calling the reading procedure for each line of Broker data. It may contain multiple values.

The 'notify' is the correct approach.

Instead of a 40 ms polling loop for incoming data over the socket, most programming systems nowadays support setting up an interrupt handler or event handler on I/O channels. This may save you some cycles as well, and also improve the data latence a wee bit (from max 40 ms down to whatever your system can do).


Jeroen

[Avi]

(The forum removes my tabbed indentation so this may be a little difficult to read.)

Hi Garry.

Put your code between the code tags ( [ code ] and [ /code ] without the spaces) and it should solve the problem.

It will look like this:

Code Select Expand

aaa...
   bbb...
       ccc...


Cheers,


(edit: fixed it, JH)

[Garry Richards]

Thanks for the tip Avi and for the edit Hoppie - I forgot about the useful / code switch.

I considered using event handlers but in practice a thread timer works efficiently to call several read/write procedures consecutively. The Broker's input buffer state is tested first and if it is empty the read procedure exits.

In flight high speed changing data from the IPC keys are being received at about 18.2 Hz (about every 55 ms). The timer also controls injection of high speed FS ground elevation data into PS1. Ground elevation has to be injected repeatedly even if it is not changing or PS1 will revert to its own value.

I found by experimentation that a timer period close to but less than 55 ms resulted in reliable high speed data transfer but anything less than about 40 ms was wasting cycles.

PSX is of course another matter and processing its high speed data might require a different approach.  

[mikeindevon]

Thanks for all the input - most appreciated.

I have had some success with the extrapolation method, although it needs more sea trials.  The current state of development is as follows:
- I am using the notify method with a similar set of parameters to Garry
- I take a sample of the notify data at about 1Hz, this longer period reduces the S/N ratio, ie the random variations in the position data are a smaller percentage of the movement between samples.
- I compute the extrapolation coefficients  for lat, lon and alt in the thread dealing with the broker using a fit to the last n data points, n = 5 seems to work.
- I get called by X-plane every simulation frame and I extrapolate the data.  This calculation is very quick and I can achieve approx 50 frames per second with high rendering options on a Core 2 duo machine that was high-end spec four years ago - so we are not talking bleeding edge here.
- When the next data comes in from the Broker thread I compute a delta from my estimated position, start extrapolating from the new data point but apply the delta so that in fact I appear to continue from where I was.  The delta is then gradually reduced over N frames, N to be tuned but currently equivalent to about 5 seconds.  Any new data modifies the current value of delta.
- Over a one second period during an approach the plane moves about 125m and a typical delta is about 0.5m, jumping to 3-5m for extreme data errors.
-The overall effect of this is a very smooth approach.
I have just been running the standard approaches into Sydney and Heathrow and haven't tried flying the plane manually yet.  Perhaps the smoothing will need to be reduced to provide adequate response.

Garry, you say that you feed the altitude back into PS1.  Is this just to keep the instrumentation up-to-date (ground proximity, radio alt etc) or is there some deeper requirement for this.

Mike

[Jeroen Hoppenbrouwers]

Feeding radio altitude is indeed to keep the indicated height equal to the visual height, and therefore also to induce flight into terrain effects.

[ivan]

Better late than never...

In late 2009 I did some work on slaving FlightGear to PS1 as a scenery generator:
http://open744.wordpress.com/2009/11/27/fgvisual/

And here is where you can get source code and binaries:
http://open744.sourceforge.net/fgvisual.html

[John Golin]

Hello Ivan!

[Jeroen Hoppenbrouwers]

WorldFlight coming up. Everybody surfaces!