[Sis-dtn] [EXTERNAL] R: Special use of sequence number in bundle creation time stamp
Keith Scott
keithlscott at gmail.com
Thu Aug 10 08:35:32 UTC 2023
If you want to implement such a hack for your mission, by all means do it.
Ensure that the endpoints you're interested in know the secret handshake
(and that the use of such doesn't violate any rules of the specs) and have
at it. At that point it's a purely mission/implementation-specific
modification, which is fine. Doing an Orange book to document a
'temporary' solution using timestamp sequence numbers seems like overkill,
especially if the consensus is that there are better ways to achieve the
same goals.
Others could weigh in here, but I suspect the chances that the IETF would
support adding the creation timestamp sequence number method to the BPv7
spec are pretty low, so we'd bifurcate from the yet-to-be-realized numerous
terrestrial BP implementations, whereas a specified and registered
extension block, while it might not be picked up by every implementation,
would at least have grounding in the BPv7 framework.
--------
* There's a bigger issue here, in that nowhere in 9171 nor the CCSDS spec
does it say that the timestamp OR any specific extension block (metadata)
MUST be provided to the receiving application agent. So to use either
method we're going to need an implementation that understands the streaming
sequence mechanism makes that information available (to the application, to
network management via an implementation-specific ADM or other
interface?). Such an implementation would (could) be entirely
spec-compliant, but the capability isn't guaranteed by the specs.
The extension block method also suffers from the risk that some joker might
encrypt it so that any processing would have to be done post-decryption
(presumably at the destination node itself). At least the timestamp
sequence number is guaranteed to be in the clear.
9171 definition of 'Delivery' (my emphasis):
A bundle is considered to have been delivered at a node subject to a
registration as soon as the ADU that is the payload of the bundle, together
with any relevant metadata (*an implementation matter*), has been presented
to the node's application agent in a manner consistent with the state of
that registration.
CCSDS BPv7 spec (my emphasis):
4.3.7 BUNDLE DELIVERY INDICATION PARAMETERS
4.3.7.1 The delivery indication parameters shall be the ADU and the
metadata from 4.3.7.2 below pertaining to the ADU.
4.3.7.2 The value of the delivery indications parameters shall include the
following:
a) application data unit is an administrative record;
b) acknowledgement by application is requested.
NOTE – Implementations may also include other information with the Bundle
Delivery Indication Parameters such as the source EID, creation time,
and/or information from extension blocks.
--keith
On Thu, Aug 10, 2023 at 3:38 AM Sanchez Net, Marc (US 332H) <
marc.sanchez.net at jpl.nasa.gov> wrote:
> All,
>
>
>
> I would suggest we consider Felix’s and Jonathan’s proposal in the context
> of an CCSDS orange book (maybe green or yellow?), as it does address a
> critical issue (that of bundle accountability) to operationalize DTN in
> flight projects. I do agree that the best way to implement this
> functionality is with an extension block. However, in the near-term, and
> from an infusion standpoint, having a “poor man’s way” of getting the job
> done is valuable (e.g., new extension block means more dev time, more
> requirements, more V&V, more mission cost, etc.), as long as we understand
> and document its limitations.
>
>
>
> From a technical standpoint:
>
> - I am not too convinced by the use of the BPCF as currently proposed.
> There seems to be side effects (e.g., “*In case 10 with different
> destination endpoint IDs, the BPA has to guarantee that the same values can
> only appear with different bundle creation times”*) and RFC9171 states
> that “Bundle processing control flags that are unrecognized be ignored”, so
> we risk having DTN nodes with different behavior?
> - I am more in favor of documenting that by not resetting the sequence
> number, which is explicitly allowed in RF9171 and (as Josh points out)
> still serves the original purpose of the sequence number, projects may
> identify gaps and perform data accountability. Since this is method is a
> “poor man’s way” of achieving this in the near-term, management by policy
> (internal to the project, or by SLA agreement with the network provider)
> seems ok.
> - I am not too convinced that timestamps and sequence numbers are the
> best way of doing in-order delivery. The difference between in-order
> delivery and gap identification is that the former requires modifying how a
> DTN node delivers bundles to the application (maybe forwarding is affected
> too?), so it seems a change in bundle processing behavior. On the other
> hand, the latter is just a change to the reporting mechanism, which seems
> less intrusive. For example, a DTN node could implement an indication to
> flag a gap (as detected by the sequence number), which then triggers flight
> SW to generate an EVR destined for the MOC (which may flow as a payload to
> another bundle or go via a non-DTN channel). It seems to me that such
> behavior would not need to be interoperable (after all, EVRs aren’t).
>
>
>
> Thanks,
>
>
> -----------------------------------------------------------------------------------------------------------------------
>
> Marc Sanchez Net (332H)
>
> Telecommunications Engineer
>
> Jet Propulsion Laboratory
>
> Cell: (617) 953-7977 <(617)%20953-7977> | Email:
> marc.sanchez.net at jpl.nasa.gov
>
>
> -----------------------------------------------------------------------------------------------------------------------
>
>
>
> *From:* SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> * On Behalf Of *sburleig.sb---
> via SIS-DTN
> *Sent:* Wednesday, August 9, 2023 2:42 PM
> *To:* 'Keith Scott' <keithlscott at gmail.com>; 'Wilmot, Jonathan J.
> (GSFC-580.0)[VANTAGE SYSTEMS INC]' <jonathan.j.wilmot at nasa.gov>
> *Cc:* 'Dr. Keith L Scott via SIS-DTN' <sis-dtn at mailman.ccsds.org>;
> 'Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied Physics
> Lab]' <edward.j.birrane at nasa.gov>; 'Singh, Somendra {Simon} (GSFC-5820)' <
> simon.singh at nasa.gov>; 'Carlo Caini' <carlo.caini at unibo.it>
> *Subject:* Re: [Sis-dtn] [EXTERNAL] R: Special use of sequence number in
> bundle creation time stamp
>
>
>
> I really think that’s best; simpler, more powerful, much easier to
> standardize.
>
>
>
> Scott
>
>
>
> *From:* Keith Scott <keithlscott at gmail.com>
> *Sent:* Wednesday, August 9, 2023 2:17 PM
> *To:* Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <
> jonathan.j.wilmot at nasa.gov>
> *Cc:* Gao, Jay L (JPL-332C)[JPL Employee] <jay.l.gao at jpl.nasa.gov>;
> sburleig.sb at gmail.com; Carlo Caini <carlo.caini at unibo.it>; Felix Flentge <
> Felix.Flentge at esa.int>; Dr. Keith L Scott via SIS-DTN <
> sis-dtn at mailman.ccsds.org>; Singh, Somendra {Simon} (GSFC-5820) <
> simon.singh at nasa.gov>; Birrane, Edward J. (GSFC-460.0)[Johns Hopkins
> University Applied Physics Lab] <edward.j.birrane at nasa.gov>
> *Subject:* Re: [Sis-dtn] [EXTERNAL] R: Special use of sequence number in
> bundle creation time stamp
>
>
>
> I'm all for the capability, but an extension block seems the way to go. I
> think either SIS-DTN, or even SOIS, could do this (and/or in conjunction
> with the IETF), and get the extension block registered with IANA for all to
> use. In the meantime, it seems like it's something that's only used at the
> endpoints, so you could just use one of the private block types until the
> block is registered, then switch to the allocated type number.
>
>
>
> --keith
>
>
>
>
>
> On Wed, Aug 9, 2023 at 10:19 PM Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE
> SYSTEMS INC] via SIS-DTN <sis-dtn at mailman.ccsds.org> wrote:
>
> All,
>
>
>
> The primary purpose of the proposed Source timestamp sequence number is
> to indicate if DTN lost anything between the source and destination. Not
> being able to tell a user of your network if you dropped any of their data
> is an issue with many in the science community. What they put in the bundle
> or how they sequence it is not a DTN problem.
>
>
>
> We are trying to infuse DTN into a very conservative community. We should
> include mechanisms to determine where things may have gone wrong.
>
>
>
> Kind regards,
>
>
>
> Jonathan Wilmot
>
>
>
> CCSDS SOIS Area Director
>
> GSFC DTN Systems Engineer
>
> cFS Architecture
>
> Cell 301-751-2658
>
>
>
>
>
>
>
>
>
>
>
> *From:* Gao, Jay L (US 332C) <jay.l.gao at jpl.nasa.gov>
> *Sent:* Wednesday, August 9, 2023 3:49 PM
> *To:* sburleig.sb at gmail.com; 'Carlo Caini' <carlo.caini at unibo.it>; 'Felix
> Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via SIS-DTN' <
> sis-dtn at mailman.ccsds.org>; Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE
> SYSTEMS INC] <jonathan.j.wilmot at nasa.gov>
> *Cc:* Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied
> Physics Lab] <edward.j.birrane at nasa.gov>; Singh, Somendra {Simon}
> (GSFC-5820) <simon.singh at nasa.gov>
> *Subject:* Re: [Sis-dtn] [EXTERNAL] R: Special use of sequence number in
> bundle creation time stamp
>
>
>
> All,
>
>
>
> The proposed use of the sequence number seems to suggest it is design only
> for a kind of application that simply streams data.
>
>
>
> If you have an application that runs on top of BP that implements any kind
> of interactive messages as well as data. Then the sequence number is no
> longer a valid gap detector because not all bundles carries in-order data
> payload.
>
>
>
> I am not sure we try to standardize a new use of the sequence numbers to
> fit only a specific type of application that could be supported over BP.
>
>
>
> Jay
>
>
>
> Jay L. Gao
> _____________________________________________
> Communications and Architecture Research Section
> Jet Propulsion Laboratory
> 4800 Oak Grove Drive
> M/S: 238-343
> Pasadena, CA 91109
> Jay.L.Gao at jpl.nasa.gov
> TEL: 818-354-9528
> ------------------------------
>
> *From:* SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> on behalf of Wilmot,
> Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] via SIS-DTN <
> sis-dtn at mailman.ccsds.org>
> *Sent:* Wednesday, August 9, 2023 10:16 AM
> *To:* sburleig.sb at gmail.com <sburleig.sb at gmail.com>; 'Carlo Caini' <
> carlo.caini at unibo.it>; 'Felix Flentge' <Felix.Flentge at esa.int>; 'Dr.
> Keith L Scott via SIS-DTN' <sis-dtn at mailman.ccsds.org>
> *Cc:* Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied
> Physics Lab] <edward.j.birrane at nasa.gov>; Singh, Somendra {Simon}
> (GSFC-5820) <simon.singh at nasa.gov>
> *Subject:* Re: [Sis-dtn] [EXTERNAL] R: Special use of sequence number in
> bundle creation time stamp
>
>
>
> Scott,
>
>
>
> There seems to be some pros and cons in play that we have not
> documented well. I will create some material for a future DTN WG (week or
> two) to discuss and include the security aspects as Vint commented.
>
>
>
> Just want to ensure we all are on the same page, and we base a forward
> path on documented analysis.
>
>
>
>
>
> Kind regards,
>
>
>
> Jonathan Wilmot
>
>
>
> CCSDS SOIS Area Director
>
> GSFC DTN Systems Engineer
>
> cFS Architecture
>
> Cell 301-751-2658
>
>
>
>
>
>
>
>
>
> *From:* sburleig.sb at gmail.com <sburleig.sb at gmail.com>
> *Sent:* Wednesday, August 9, 2023 12:53 PM
> *To:* Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <
> jonathan.j.wilmot at nasa.gov>; 'Carlo Caini' <carlo.caini at unibo.it>; 'Felix
> Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via SIS-DTN' <
> sis-dtn at mailman.ccsds.org>
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>;
> Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied Physics
> Lab] <edward.j.birrane at nasa.gov>
> *Subject:* RE: [EXTERNAL] R: [Sis-dtn] Special use of sequence number in
> bundle creation time stamp
>
>
>
> *CAUTION:* This email originated from outside of NASA. Please take care
> when clicking links or opening attachments. Use the "Report Message"
> button to report suspicious messages to the NASA SOC.
>
>
>
> Jonathan, it sounds to me like the core issue here is whether to add all
> of this bundle labeling and sequence checking functionality to the existing
> requirements on primary block processing or to instead define it rigorously
> in an extension block and leave primary block processing alone.
>
>
>
> If the additional processing requirements are imposed on primary block
> processing, then all deployments of all implementations of BPv7 are
> affected, including those for whom none of this functionality is needed in
> any way. They will of course not recognize the new bundle processing
> control flags and will ignore them, but bundle uniqueness will routinely be
> lost unaccountably if (for example) bundle creation time is zero and bundle
> sequence numbers roll over at 2^16 per sequence number length flag values
> 01.
>
>
>
> It may be possible to constrain the required supplementary BPv7
> specification in a way that shields existing BPv7 users from its impact.
> But I believe that will be a difficult specification to write and the cost
> of getting it wrong will be significant.
>
>
>
> I think it would be a much better use of SCaN’s DTN resources to draft a
> clean extension block specification. At the data rates being contemplated
> here, I think the small increment in bundle size should be an acceptable
> price to pay for these capabilities.
>
>
>
> Scott
>
>
>
> *From:* Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <
> jonathan.j.wilmot at nasa.gov>
> *Sent:* Wednesday, August 9, 2023 9:20 AM
> *To:* Carlo Caini <carlo.caini at unibo.it>; 'Felix Flentge' <
> Felix.Flentge at esa.int>; 'Dr. Keith L Scott via SIS-DTN' <
> sis-dtn at mailman.ccsds.org>; sburleig.sb at gmail.com
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>;
> Birrane, Edward J. (GSFC-460.0)[Johns Hopkins University Applied Physics
> Lab] <edward.j.birrane at nasa.gov>
> *Subject:* RE: [EXTERNAL] R: [Sis-dtn] Special use of sequence number in
> bundle creation time stamp
>
>
>
> Carlo,
>
>
>
> For the use cases that we are considering, any time a source node
> sends the same bundle to two or more destinations it uses Scott’s proposed
> multi-destination protocol. If a node wants to send different bundles to
> different destinations, then they are different services.
> (org.node.service) In our flight system data model source “.service” is
> a discriminator for different bundle types/ADU. NASA.PACE.QuickLookScience
> is one service and NASA.PACE.BulkData is another.
>
>
>
> I can see an issue with the set of common “service” numbers, but maybe
> those are in the destination EID not the source.
>
>
>
> I agree we don’t want to break the original use case.
>
>
>
> Kind regards,
>
>
>
> Jonathan Wilmot
>
>
>
> CCSDS SOIS Area Director
>
> GSFC DTN Systems Engineer
>
> cFS Architecture
>
> Cell 301-751-2658
>
>
>
>
>
>
>
> *From:* Carlo Caini <carlo.caini at unibo.it>
> *Sent:* Wednesday, August 9, 2023 11:50 AM
> *To:* 'Felix Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via
> SIS-DTN' <sis-dtn at mailman.ccsds.org>; sburleig.sb at gmail.com
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>; Wilmot,
> Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <jonathan.j.wilmot at nasa.gov>
> *Subject:* [EXTERNAL] R: [Sis-dtn] Special use of sequence number in
> bundle creation time stamp
>
>
>
> *CAUTION:* This email originated from outside of NASA. Please take care
> when clicking links or opening attachments. Use the "Report Message"
> button to report suspicious messages to the NASA SOC.
>
>
>
> I fully agree with Scott on his comments and suggestions. More generally,
> I would avoid, whenever possible, the use of a parameter designed to a
> specific end, such as timestamp sequence number here, to different
> purposes. However, appealing it may be on a short term, it is looking for
> troubles on the long term.
>
> Specifically, the timestamp sequence number is not the same as an
> hupothetical bundle sequence number, as pointed out by Scott. Moreover, if
> we assume that a source generates bundles to a give destination to specific
> rate, e.g. one bundle per second, the sequence number increment between two
> consecutive bundles of this flow may be larger than one. For example, the
> first sequence number of the bundle flow could be 1 and the second 5. It is
> enough that the same source generates in the meantime other 3 bundles to
> different destinations.
>
> Yours,
>
> Carlo
>
>
> ------------------------------
>
> *Da:* SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> per conto di
> sburleig.sb--- via SIS-DTN <sis-dtn at mailman.ccsds.org>
> *Inviato:* mercoledì 9 agosto 2023 17:31
> *A:* 'Felix Flentge' <Felix.Flentge at esa.int>; 'Dr. Keith L Scott via
> SIS-DTN' <sis-dtn at mailman.ccsds.org>
> *Cc:* 'Singh, Somendra {Simon} (GSFC-5820)' <simon.singh at nasa.gov>;
> 'Wilmot, Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC]' <
> jonathan.j.wilmot at nasa.gov>
> *Oggetto:* Re: [Sis-dtn] Special use of sequence number in bundle
> creation time stamp
>
>
>
> Hi, Felix. One thing to keep in mind is that the purpose of the sequence
> number in the bundle creation timestamp is to distinguish among bundles
> that have the same bundle creation time. This can occur when bundles are
> created at a very high rate or when bundle creation time is zero due to the
> absence of an accurate clock.
>
>
>
> I would suggest that using bundle creation timestamp sequence numbers for
> other purposes makes it more difficult to use those numbers for their
> original purpose.
>
>
>
> If sequence numbering of bundles in these various ways is important, I
> think it would be better to encode those numbers (and the flags describing
> their significance) in an immutable extension block, i.e., an extension
> block that is one of the additional targets of a block integrity block
> covering the primary block.
>
>
>
> Scott
>
>
>
> *From:* SIS-DTN <sis-dtn-bounces at mailman.ccsds.org> *On Behalf Of *Felix
> Flentge via SIS-DTN
> *Sent:* Wednesday, August 9, 2023 7:43 AM
> *To:* Dr. Keith L Scott via SIS-DTN <sis-dtn at mailman.ccsds.org>
> *Cc:* Singh, Somendra {Simon} (GSFC-5820) <simon.singh at nasa.gov>; Wilmot,
> Jonathan J. (GSFC-580.0)[VANTAGE SYSTEMS INC] <jonathan.j.wilmot at nasa.gov>
> *Subject:* [Sis-dtn] Special use of sequence number in bundle creation
> time stamp
>
>
>
> Dear All,
>
>
>
> I think we have had some initial exchange on the use of the sequence
> number in the bundle creation timestamp some time ago. Meanwhile, we had a
> number of side meetings with Jonathan, Simon and others to further discuss
> the topic of ‘flows’, compressed bundle reporting and network management.
>
>
>
> We have arrived at a point where we think that it makes sense to allow
> using the sequence number in the primary header for identifying gaps in
> sequences of bundles or to even allow for in-sequence delivery.
>
>
>
> The basic idea is that the sequence number does not get reset to zero but
> is incrementing by one until a specified maximum. Further we might use
> different, individual sequence numbers per source node ID or per
> destination endpoint ID. Such a behaviour might be governed by policy or
> even indicated in the Primary Block’s Processing Control Flags, like
>
>
>
> 1. Use two bits to specify whether the creation timestamp sequence
> number is used in special way:
>
>
>
> - 00 – single sequence number - no specific usage
> - 01 – an individual sequence number is used per source node id
> - 10 – an individual sequence number is used per destination endpoint
> id
> - 11 – reserved (until we find a good usage; maybe individual sequence
> number per next hop)
>
>
>
> *In case 10 with different destination endpoint IDs, the BPA has to
> guarantee that the same values can only appear with different bundle
> creation times!*
>
>
>
> 1. Use of two bits to indicate the maximum length of the sequence
> number (00 above) or individual sequence numbers (01 and 10 above):
>
>
> - 00 - sequence numbers are assigned somehow (reset to zero, randomly,
> ..)
> - 01 - increasing sequence number up to 2^16-1
> - 10 - increasing sequence number up to 2^32-1
> - 11 - increasing sequence number up to 2^64-1
>
> My main questions:
>
> 1. Does it make sense to use flags in the primary header for this
> purpose or is it sufficient to document that sequence numbers can be used
> in that way and leave it up to policy to specify certain behaviours? I have
> a slight preference for using the BPCF as it seems more formal and could
> ease interoperability (and the sequence number is in the primary header, so
> why not indicate it there).
> 2. If we want to use BPCF, what is the way to specify this? Should we
> / do we need to go IETF or could CCSDS be sufficient (to start with)?
> Ideally we would use some of the reserved first 16 bits (e.g. Bit 7 – 10)
> 3. No matter whether we want to use BCPF or not, I think something
> should end up in the CCSDS BB as soon as possible. So, maybe there is a
> chance for a late RID (if it does not delay the overall publication
> process). We should start discussing (maybe tomorrow).
>
>
>
> Regards,
>
> Felix
>
>
>
> This message is intended only for the recipient(s) named above. It may
> contain proprietary information and/or protected content. Any unauthorised
> disclosure, use, retention or dissemination is prohibited. If you have
> received this e-mail in error, please notify the sender immediately. ESA
> applies appropriate organisational measures to protect personal data, in
> case of data privacy queries, please contact the ESA Data Protection
> Officer (dpo at esa.int).
>
> _______________________________________________
> SIS-DTN mailing list
> SIS-DTN at mailman.ccsds.org
> https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn
> <https://urldefense.us/v3/__https:/mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-dtn__;!!PvBDto6Hs4WbVuu7!O9YNN-1tejMB-n4VHyJZoduZkXN6wDvc26SiZRhT4ZH7xsGHq_ea5xF8_xMEsNxWuVLgeefJP6NY7BwzI5MH6sonvV-tsoE$>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/sis-dtn/attachments/20230810/e99a71b2/attachment-0001.htm>
More information about the SIS-DTN
mailing list