[Sis-dtn] R: ION and in-order delivery

[Carlo Caini]

Dear Keith,
        not only do out-of-order bundles are allowed, but they actually happen in DTN tests for a variety of reasons. However, it is difficult to know why, unless a very detailed ("microscopic" i.e. bundle-by-bundle) analysis is carried out.
Your test however is very simple, thus I would exclude both routing and TCPCL. Most likely it is LTP at the origin of the problem. Note that with your configuration 10 sessions are allowed in parallel. In the absence of losses the LTP blocks corresponding to the sessions should arrive in order, but this is not the case in the presence of losses. In fact, the delivery time is roughly ½ RTT in the absence of any LTP segment losses, and 1+1/2 RTT if one or more losses have to be recovered (and you have no losses on the retransmitted segments, of course).

Note that you could have losses not only on the channel, but also inside the receiver machine because of a UDP buffer overflow on the Rx buffer, if the LTP packets are sent too fast. You can check if this is the case with the Linux command netstat. ION, however, has a UDPLSO pacer on the sender side, set on the basis of the nominal Tx speed declared in the contact (byte/s), thus I would exclude this too, but is worth checking. You can also use either the LTP log to see if you have retransmissions or not, or more simply sniff the traffic with tcpdump and then analyze it with Wireshark. If you have losses on LTP red data segments the out of order is unavoidable, as blocks arrive with different delays. By the way, in the past the UDPLSO  pacer was ruled by the speed (in bit/s) declared after the IP adress at the end of the span command (1000000 in your case), now (in recent ION versions) I think that this number, if inserted, is just neglected, which is correct as it is better to rely on the Tx speed declared in the contact . In the udplso man it is no more mentioned.

Another possible reason for out of order delivery could be bundle aggregation. In your test I suppose you have a lot of bping requests aggregated by A when the contact between A and B starts after a pause, and the same for B for echo replies. When one block with multiple aggregated bundles arrives, basically all bundles are delivered at (roughly) the same time, but I am not sure that the original order is preserved. It is fairly easy to check: it is enough to set bundle aggregation size to 1B.
Let me know if you manage to find the culprit!
Yours,
   Carlo



________________________________
Da: SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> per conto di Keith Scott via SIS-DTN <sis-dtn at mailman.ccsds.org>
Inviato: giovedì 15 febbraio 2024 20:59
A: sis-dtn at mailman.ccsds.org <sis-dtn at mailman.ccsds.org>
Oggetto: [Sis-dtn] ION and in-order delivery

I ran a quick test with a 3-node ION network A-B-C where for the first minute A is connected to B (via LTP) and B is connected to C (via TCP).  After that the topology repeats every two minutes; during the first minute A is connected to B and during the second B is connected to C.  At t=540s full connectivity (A-B and B-C) is restored for one minute.  There is no artificially induced loss in this scenario.  The max aggregation size for LTP was 100kbytes and the max aggregation time was 1s, e.g.:
    a span 2 10 10 64000 100000 1 'udplso 10.44.3.2:1113<http://10.44.3.2:1113> 1000000'


[image.png]

I ran bping (will do a unidirectional test later).  The chart below shows the received bping sequence number as a function of the order in which bpings were received.

[image.png]

So yeah, it looks like ION will occasionally misorder bundles.  I think that's not ideal, but I strongly believe that it is compliant with both the spec and the intent of the service (bundle delivery) BP purports to provide.  (And most bundles were in order, which is a feat considering that between 60 and 540s the network is never end-to-end connected).

    --keith

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/sis-dtn/attachments/20240215/a73cf2d8/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-image.png.png
Type: image/png
Size: 79905 bytes
Desc: Outlook-image.png.png
URL: <http://mailman.ccsds.org/pipermail/sis-dtn/attachments/20240215/a73cf2d8/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Outlook-image.png.png
Type: image/png
Size: 48080 bytes
Desc: Outlook-image.png.png
URL: <http://mailman.ccsds.org/pipermail/sis-dtn/attachments/20240215/a73cf2d8/attachment-0003.png>