[Sis-dtn] [EXTERNAL] R: Special use of sequence number in bundle creation time stamp

[Keith Scott]

I'm all for the capability, but an extension block seems the way to go.  I
think either SIS-DTN, or even SOIS, could do this (and/or in conjunction
with the IETF), and get the extension block registered with IANA for all to
use.  In the meantime, it seems like it's something that's only used at the
endpoints, so you could just use one of the private block types until the
block is registered, then switch to the allocated type number.

    --keith


On Wed, Aug 9, 2023 at 10:19 PM Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE
SYSTEMS INC] via SIS-DTN <sis-dtn at mailman.ccsds.org> wrote:

> All,
>
>
>
>    The primary purpose of the proposed Source timestamp sequence number is
> to indicate if DTN lost anything between the source and destination. Not
> being able to tell a user of your network if you dropped any of their data
> is an issue with many in the science community. What they put in the bundle
> or how they sequence it is not a DTN problem.
>
>
>
> We are trying to infuse DTN into a very conservative community. We should
> include mechanisms to determine where things may have gone wrong.
>
>
>
> Kind regards,
>
>
>
>      Jonathan Wilmot
>
>
>
> CCSDS SOIS Area Director
>
> GSFC DTN Systems Engineer
>
> cFS Architecture
>
> Cell 301-751-2658
>
>
>
>
>
>
>
>
>
>
>
> *From:* Gao, Jay L (US 332C) <jay.l.gao at jpl.nasa.gov>
> *Sent:* Wednesday, August 9, 2023 3:49 PM
> *To:* sburleig.sb at gmail.com; 'Carlo Caini' <carlo.caini at unibo.it>; 'Felix
> Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via SIS-DTN' <
> sis-dtn at mailman.ccsds.org>; Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE
> SYSTEMS INC] <jonathan.j.wilmot at nasa.gov>
> *Cc:* Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied
> Physics Lab] <edward.j.birrane at nasa.gov>; Singh, Somendra {Simon}
> (GSFC-5820) <simon.singh at nasa.gov>
> *Subject:* Re: [Sis-dtn] [EXTERNAL] R: Special use of sequence number in
> bundle creation time stamp
>
>
>
> All,
>
>
>
> The proposed use of the sequence number seems to suggest it is design only
> for a kind of application that simply streams data.
>
>
>
> If you have an application that runs on top of BP that implements any kind
> of interactive messages as well as data. Then the sequence number is no
> longer a valid gap detector because not all bundles carries in-order data
> payload.
>
>
>
> I am not sure we try to standardize a new use of the sequence numbers to
> fit only a specific type of application that could be supported over BP.
>
>
>
> Jay
>
>
>
> Jay L. Gao
> _____________________________________________
> Communications and Architecture Research Section
> Jet Propulsion Laboratory
> 4800 Oak Grove Drive
> M/S: 238-343
> Pasadena, CA 91109
> Jay.L.Gao at jpl.nasa.gov
> TEL: 818-354-9528
> ------------------------------
>
> *From:* SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> on behalf of Wilmot,
> Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] via SIS-DTN <
> sis-dtn at mailman.ccsds.org>
> *Sent:* Wednesday, August 9, 2023 10:16 AM
> *To:* sburleig.sb at gmail.com <sburleig.sb at gmail.com>; 'Carlo Caini' <
> carlo.caini at unibo.it>; 'Felix Flentge' <Felix.Flentge at esa.int>; 'Dr.
> Keith L Scott via SIS-DTN' <sis-dtn at mailman.ccsds.org>
> *Cc:* Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied
> Physics Lab] <edward.j.birrane at nasa.gov>; Singh, Somendra {Simon}
> (GSFC-5820) <simon.singh at nasa.gov>
> *Subject:* Re: [Sis-dtn] [EXTERNAL] R: Special use of sequence number in
> bundle creation time stamp
>
>
>
> Scott,
>
>
>
>    There seems to be some pros and cons in play that we have not
> documented well. I will create some material for a future DTN WG (week or
> two) to discuss and include the security aspects as Vint commented.
>
>
>
> Just want to ensure we all are on the same page, and we base a forward
> path on documented analysis.
>
>
>
>
>
> Kind regards,
>
>
>
>      Jonathan Wilmot
>
>
>
> CCSDS SOIS Area Director
>
> GSFC DTN Systems Engineer
>
> cFS Architecture
>
> Cell 301-751-2658
>
>
>
>
>
>
>
>
>
> *From:* sburleig.sb at gmail.com <sburleig.sb at gmail.com>
> *Sent:* Wednesday, August 9, 2023 12:53 PM
> *To:* Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <
> jonathan.j.wilmot at nasa.gov>; 'Carlo Caini' <carlo.caini at unibo.it>; 'Felix
> Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via SIS-DTN' <
> sis-dtn at mailman.ccsds.org>
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>;
> Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied Physics
> Lab] <edward.j.birrane at nasa.gov>
> *Subject:* RE: [EXTERNAL] R: [Sis-dtn] Special use of sequence number in
> bundle creation time stamp
>
>
>
> *CAUTION:* This email originated from outside of NASA.  Please take care
> when clicking links or opening attachments.  Use the "Report Message"
> button to report suspicious messages to the NASA SOC.
>
>
>
> Jonathan, it sounds to me like the core issue here is whether to add all
> of this bundle labeling and sequence checking functionality to the existing
> requirements on primary block processing or to instead define it rigorously
> in an extension block and leave primary block processing alone.
>
>
>
> If the additional processing requirements are imposed on primary block
> processing, then all deployments of all implementations of BPv7 are
> affected, including those for whom none of this functionality is needed in
> any way.  They will of course not recognize the new bundle processing
> control flags and will ignore them, but bundle uniqueness will routinely be
> lost unaccountably if (for example) bundle creation time is zero and bundle
> sequence numbers roll over at 2^16 per sequence number length flag values
> 01.
>
>
>
> It may be possible to constrain the required supplementary BPv7
> specification in a way that shields existing BPv7 users from its impact.
> But I believe that will be a difficult specification to write and the cost
> of getting it wrong will be significant.
>
>
>
> I think it would be a much better use of SCaN’s DTN resources to draft a
> clean extension block specification.  At the data rates being contemplated
> here, I think the small increment in bundle size should be an acceptable
> price to pay for these capabilities.
>
>
>
> Scott
>
>
>
> *From:* Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <
> jonathan.j.wilmot at nasa.gov>
> *Sent:* Wednesday, August 9, 2023 9:20 AM
> *To:* Carlo Caini <carlo.caini at unibo.it>; 'Felix Flentge' <
> Felix.Flentge at esa.int>; 'Dr. Keith L Scott via SIS-DTN' <
> sis-dtn at mailman.ccsds.org>; sburleig.sb at gmail.com
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>;
> Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied Physics
> Lab] <edward.j.birrane at nasa.gov>
> *Subject:* RE: [EXTERNAL] R: [Sis-dtn] Special use of sequence number in
> bundle creation time stamp
>
>
>
> Carlo,
>
>
>
>     For the use cases that we are considering, any time a source node
> sends the same bundle to two or more destinations it uses Scott’s proposed
> multi-destination protocol. If a node wants to send different bundles to
> different destinations, then they are different services.
> (org.node.service)    In our flight system data model source “.service” is
> a discriminator for different bundle types/ADU. NASA.PACE.QuickLookScience
> is one service and NASA.PACE.BulkData is another.
>
>
>
> I can see an issue with the set of common “service” numbers, but maybe
> those are in the destination EID not the source.
>
>
>
> I agree we don’t want to break the original use case.
>
>
>
> Kind regards,
>
>
>
>      Jonathan Wilmot
>
>
>
> CCSDS SOIS Area Director
>
> GSFC DTN Systems Engineer
>
> cFS Architecture
>
> Cell 301-751-2658
>
>
>
>
>
>
>
> *From:* Carlo Caini <carlo.caini at unibo.it>
> *Sent:* Wednesday, August 9, 2023 11:50 AM
> *To:* 'Felix Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via
> SIS-DTN' <sis-dtn at mailman.ccsds.org>; sburleig.sb at gmail.com
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>; Wilmot,
> Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <jonathan.j.wilmot at nasa.gov>
> *Subject:* [EXTERNAL] R: [Sis-dtn] Special use of sequence number in
> bundle creation time stamp
>
>
>
> *CAUTION:* This email originated from outside of NASA.  Please take care
> when clicking links or opening attachments.  Use the "Report Message"
> button to report suspicious messages to the NASA SOC.
>
>
>
> I fully agree with Scott on his comments and suggestions. More generally,
> I would avoid, whenever possible, the use of a parameter designed to a
> specific end, such as timestamp sequence number here, to different
> purposes. However, appealing it may be on a short term, it is looking for
> troubles on the long term.
>
> Specifically, the timestamp sequence number is not the same as an
> hupothetical bundle sequence number, as pointed out by Scott. Moreover, if
> we assume that a source generates bundles to a give destination to specific
> rate, e.g. one bundle per second, the sequence number increment between two
> consecutive bundles of this flow may be larger than one. For example, the
> first sequence number of the bundle flow could be 1 and the second 5. It is
> enough that the same source generates in the meantime other 3 bundles to
> different destinations.
>
> Yours,
>
>    Carlo
>
>
> ------------------------------
>
> *Da:* SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> per conto di
> sburleig.sb--- via SIS-DTN <sis-dtn at mailman.ccsds.org>
> *Inviato:* mercoledì 9 agosto 2023 17:31
> *A:* 'Felix Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via
> SIS-DTN' <sis-dtn at mailman.ccsds.org>
> *Cc:* 'Singh, Somendra {Simon} (GSFC-5820)' <simon.singh at nasa.gov>;
> 'Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC]' <
> jonathan.j.wilmot at nasa.gov>
> *Oggetto:* Re: [Sis-dtn] Special use of sequence number in bundle
> creation time stamp
>
>
>
> Hi, Felix.  One thing to keep in mind is that the purpose of the sequence
> number in the bundle creation timestamp is to distinguish among bundles
> that have the same bundle creation time.  This can occur when bundles are
> created at a very high rate or when bundle creation time is zero due to the
> absence of an accurate clock.
>
>
>
> I would suggest that using bundle creation timestamp sequence numbers for
> other purposes makes it more difficult to use those numbers for their
> original purpose.
>
>
>
> If sequence numbering of bundles in these various ways is important, I
> think it would be better to encode those numbers (and the flags describing
> their significance) in an immutable extension block, i.e., an extension
> block that is one of the additional targets of a block integrity block
> covering the primary block.
>
>
>
> Scott
>
>
>
> *From:* SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> *On Behalf Of *Felix
> Flentge via SIS-DTN
> *Sent:* Wednesday, August 9, 2023 7:43 AM
> *To:* Dr. Keith L Scott via SIS-DTN <sis-dtn at mailman.ccsds.org>
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>; Wilmot,
> Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <jonathan.j.wilmot at nasa.gov>
> *Subject:* [Sis-dtn] Special use of sequence number in bundle creation
> time stamp
>
>
>
> Dear All,
>
>
>
> I think we have had some initial exchange on the use of the sequence
> number in the bundle creation timestamp some time ago. Meanwhile, we had a
> number of side meetings with Jonathan, Simon and others to further discuss
> the topic of ‘flows’, compressed bundle reporting and network management.
>
>
>
> We have arrived at a point where we think that it makes sense to allow
> using the sequence number in the primary header for identifying gaps in
> sequences of bundles or to even allow for in-sequence delivery.
>
>
>
> The basic idea is that the sequence number does not get reset to zero but
> is incrementing by one until a specified maximum. Further we might use
> different, individual sequence numbers per source node ID or per
> destination endpoint ID. Such a behaviour might be governed by policy or
> even indicated in the Primary Block’s Processing Control Flags, like
>
>
>
>    1. Use two bits to specify whether the creation timestamp sequence
>    number is used in special way:
>
>
>
>    - 00 – single sequence number - no specific usage
>    - 01 – an individual sequence number is used per source node id
>    - 10 – an individual sequence number is used per destination endpoint
>    id
>    - 11 – reserved (until we find a good usage; maybe individual sequence
>    number per next hop)
>
>
>
> *In case 10 with different destination endpoint IDs, the BPA has to
> guarantee that the same values can only appear with different bundle
> creation times!*
>
>
>
>    1. Use of two bits to indicate the maximum length of the sequence
>    number (00 above) or individual sequence numbers (01 and 10 above):
>
>
>    - 00 - sequence numbers are assigned somehow (reset to zero, randomly,
>    ..)
>    - 01 - increasing sequence number up to 2^16-1
>    - 10 - increasing sequence number up to 2^32-1
>    - 11 - increasing sequence number up to 2^64-1
>
> My main questions:
>
>    1. Does it make sense to use flags in the primary header for this
>    purpose or is it sufficient to document that sequence numbers can be used
>    in that way and leave it up to policy to specify certain behaviours? I have
>    a slight preference for using the BPCF as it seems more formal and could
>    ease interoperability (and the sequence number is in the primary header, so
>    why not indicate it there).
>    2. If we want to use BPCF, what is the way to specify this? Should we
>    / do we need to go IETF or could CCSDS be sufficient (to start with)?
>    Ideally we would use some of the reserved first 16 bits (e.g. Bit 7 – 10)
>    3. No matter whether we want to use BCPF or not, I think something
>    should end up in the CCSDS BB as soon as possible. So, maybe there is a
>    chance for a late RID (if it does not delay the overall publication
>    process). We should start discussing (maybe tomorrow).
>
>
>
> Regards,
>
> Felix
>
>
>
> This message is intended only for the recipient(s) named above. It may
> contain proprietary information and/or protected content. Any unauthorised
> disclosure, use, retention or dissemination is prohibited. If you have
> received this e-mail in error, please notify the sender immediately. ESA
> applies appropriate organisational measures to protect personal data, in
> case of data privacy queries, please contact the ESA Data Protection
> Officer (dpo at esa.int).
> _______________________________________________
> SIS-DTN mailing list
> SIS-DTN at mailman.ccsds.org
> https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/sis-dtn/attachments/20230809/1498a20d/attachment-0001.htm>