[SIS-CFDPV1] revised draft specification

Wed Dec 11 09:13:06 UTC 2019

Hi Felix,

I think that, if we even suggest that users have the choice of private checksums, we have to provide some guidance with regards to the processing of these checksums. Otherwise we'll wind up in the scenario which Cheol pointed out where the receiving entity assumes that it can process the private checksum, but the file fails validation and generates a "file checksum" failure, since the checksum is private. In my opinion, that isn't really the correct behavior, since a Unsupported Checksum Type fault would be more appropriate.

It doesn't have to be much, "the private checksum MAY be implemented." & "If the private checksum is implemented, the sending and receiving entities must ensure that an unambiguous selection of checksum algorithms is ensured for a given file transfer. If this cannot be ensured, an Unsupported Checksum Type fault SHALL be generated". This also covers the interoperability testing: if the sending entity can support the selection of a private checksum while the receiving entity can't determine it, then it's compliant with the specification.

Given that the filestore request does not contain the desired checksum type (and my suggestion to add it was simply a workaround to increasing the MIB complexity), the receiving entity will not know what checksum is in use until the initial metadata PDU, at which point we have to be able to deterministically process or reject it.

Thanks,

Jeremy

________________________________
From: Felix.Flentge at esa.int <Felix.Flentge at esa.int>
Sent: Wednesday, December 11, 2019 9:25:52 AM
To: Mayer, Jeremy
Cc: chkoo at kari.re.kr; sis-cfdpv1 at mailman.ccsds.org; sis-cfdpv1-bounces at mailman.ccsds.org
Subject: Re: [SIS-CFDPV1] revised draft specification

Hmmh,

I wouldn't like to introduce too much change in the draft at the current stage.

In general, I think:
1) We could have checksum ID for private / experimental use but its use should be discouraged.
2) Implementations may or may not support private / experimental checksums (or be configurable). This could go to the MIB but does not have to (as for the other checksum ID). It's an implementation matter.
3) If an implementation chooses to support private / experimental checksums, it will try to apply a checksum which may either succeed or fail. Which exact checksum to apply is again an implementation matter, could be per source or generic or guessing based on some other information ...
4) If an implementation chooses not to support private / experimental checksums, it will raise the Unsupported Checksum Type fault.

For the interoperability testing, I am not sure how much of this has to be covered. As the private / experimental checksum is, well 'private/experimental', I don't think we have to test something for the standard interoperability. Dealing with unsupported checksums seems also more a 'local' entity issue and not so much required for interoperability testing.

Regards,
Felix

From:        <Jeremy.Mayer at dlr.de>
To:        <chkoo at kari.re.kr>, <Felix.Flentge at esa.int>, <sis-cfdpv1 at mailman.ccsds.org>, <sis-cfdpv1-bounces at mailman.ccsds.org>
Date:        11/12/2019 08:49
Subject:        Re: [SIS-CFDPV1] revised draft specification
________________________________

Hi Cheol,

Understood, and to respond to both of your emails at once: I think that the "private/experimental" checksum ID will cover this scenario, without requiring the allocation of the limited range of checksum ID's which are supported by CFDP.

To use your example, if the sending entity uses a private checksum algorithm, then they would put checksum ID private_use (whichever ID that may be) into the metadata PDU. Upon receiving that checksum, the receiving entity MUST search the MIB in order to determine which checksum is used for that source CFDP entity ID, and process accordingly. If it doesn't find a checksum, then the "Unsupported Checksum Type" error is raised.

This would still increase the complexity of the MIB and checksum processing pipeline, but we can outline that the private_use checksum ID MUST search the MIB to determine the actual checksum algorithm, and shall fail if  more or less than 1 checksum matches for a given CFDP source entity ID.

Thanks,

Jeremy

________________________________

From: 구철회 <chkoo at kari.re.kr>
Sent: Wednesday, December 11, 2019 8:46:23 AM
To: Mayer, Jeremy; Felix.Flentge at esa.int; sis-cfdpv1 at mailman.ccsds.org; sis-cfdpv1-bounces at mailman.ccsds.org
Subject: RE: [SIS-CFDPV1] revised draft specification

Hi, Felix.

If we choose one ID as 'Reserved for Private and/or Experimental Use, that means a CFDP implementation can support the ID; therefore when the ID is applied to a filestore request, a receiving entity (who can handle the ID) will have no problem to process the request. No fault condition can be issued.

Jeremy’s idea can handle the situation. But a ‘prior coordination between agencies’ should be made before entering a real testing in order to know which one is an implemented checksum algorithm (because the sending entity have to process a file delivery request), or a fake algorithm (non-support algorithm). I think that could be not what we want in this context.

Cheol

From: Jeremy.Mayer at dlr.de <Jeremy.Mayer at dlr.de>
Sent: Wednesday, December 11, 2019 4:17 PM
To: Felix.Flentge at esa.int; 구철회 <chkoo at kari.re.kr>; sis-cfdpv1 at mailman.ccsds.org; sis-cfdpv1-bounces at mailman.ccsds.org
Subject: Re: [SIS-CFDPV1] revised draft specification

Felix,

I agree. Checksum ID's 0-15 are reserved for CFDP,  so that checksum can be added to the blue book without altering the registry. However, it may make sense to allocate two checksum ID's for general (non-CFDP specific) use: one for private and one for experimental.

If we did that, checksums 0-3 would be "real" checksums, 4-12 would be reserved for CFDP, 13 & 14 would be private/experimental, and 15 would NULL.

Thanks,

Jeremy

________________________________

From: Felix.Flentge at esa.int<mailto:Felix.Flentge at esa.int> <Felix.Flentge at esa.int<mailto:Felix.Flentge at esa.int>>
Sent: Wednesday, December 11, 2019 8:10:41 AM
To: chkoo at kari.re.kr<mailto:chkoo at kari.re.kr>; sis-cfdpv1 at mailman.ccsds.org<mailto:sis-cfdpv1 at mailman.ccsds.org>; SIS-CFDPV1; Mayer, Jeremy
Subject: Re: [SIS-CFDPV1] revised draft specification

Dear All,

I would support that idea. We could be even more general (also to avoid changing the current draft standard wrt to the MIB) to just have one ID specified as 'Reserved for Private and/or Experimental Use'.

Regards,
Felix

From:        <Jeremy.Mayer at dlr.de<mailto:Jeremy.Mayer at dlr.de>>
To:        <chkoo at kari.re.kr<mailto:chkoo at kari.re.kr>>, <sis-cfdpv1 at mailman.ccsds.org<mailto:sis-cfdpv1 at mailman.ccsds.org>>
Date:        11/12/2019 08:01
Subject:        Re: [SIS-CFDPV1] revised draft specification
Sent by:        "SIS-CFDPV1" <sis-cfdpv1-bounces at mailman.ccsds.org<mailto:sis-cfdpv1-bounces at mailman.ccsds.org>>

________________________________

Hi Cheol,

I can answer your questions about the Checksum registry: First, I'm not sure why the checksum ID's are non-sequential, since I'm not sure what the SANA website is sorting by, but if you click the "ChecksumID" column header, it should sort correctly.

I would suggest that, in the scenario which you outlined, the MIB containing the supported checksums for a receiving entity should be updated via TC and/or the desired checksum should be sent during a filestore request, but that's a more impactful change. I am adverse to a one-to-one allocation of checksum ID's to agencies, since it ultimately negates the purpose of the checksum ID and complicates mission operations and cross support: a single receiving CFDP entity at a ground station can (and likely will) support multiple missions, some of which may be performed from different agencies, each of which may have their own checksumming capabilities and requirements. If a single checksum ID is allocated to each agency, the receiving entity will be unable to determine the "real" checksum in use, unless that data is conveyed via some out-of-band mechanism.

Additionally, different files from a single mission may require different checksums, either due to bandwidth constraints, mission phase, and/or avionics capabilities, meaning that the receiving entity will have to be aware of the mapping between entities, sessions, and checksum configurations. This is further complicated by the rise of file-based operations and advanced FDIR systems: it may be possible for the sending entity to change checksum modes in the event of a failure, in order in order to reduce power utilization and/or computational complexity.

What would be possible is to allocate a single checksum ID to be "MIB defined". If that ID is in use, then it's up to the sender/receiver to determine the checksum out-of-band, but I think that would require some changes to the blue book.

Thanks,

Jeremy

________________________________

From: SIS-CFDPV1 <sis-cfdpv1-bounces at mailman.ccsds.org<mailto:sis-cfdpv1-bounces at mailman.ccsds.org>> on behalf of 구철회 <chkoo at kari.re.kr<mailto:chkoo at kari.re.kr>>
Sent: Wednesday, December 11, 2019 1:48:18 AM
To: sis-cfdpv1 at mailman.ccsds.org<mailto:sis-cfdpv1 at mailman.ccsds.org>
Subject: Re: [SIS-CFDPV1] revised draft specification

Hi, Scott.

I like your idea. I am currently working on the modification of CFDP software from my side, and I love to perform the supplementary interoperability test when the updated ION and interoperability test plan are available.

I don’t see many testing items are required for this supplementary testing. However as I sent it earlier to you there is a tricky issue with regard to handling condition code of ‘Unsupported checksum type(1011)’. In order to make that situation, the ‘Unsupported checksum type’ is generated, happen, a CFDP (sending) entity should be able to generate a checksum field by a checksum algorithm that the other CFDP (receiving) entity does not support. As there are not much choices in selection of checksum algorithms, it can be tricky because one CFDP entity may have to implement an unnecessary checksum algorithm that is even not consistent with CFDP Checksum Identifiers in SANA registry (https://sanaregistry.org/r/checksum_identifiers).

   * short question : why are the number of the checksum ID in the SANA not consistently sequential? E.g., 4,5,(6?),7,(8?),9,10,11,12,13,14,6,8,15.

After thinking about this issue, I like to propose an idea. Why don’t we allocate a Checksum ID to each space agencies, e.g., KARI reserve 5 as a checksum ID for potential use. Then KARI can freely choose the checksum algorithm for checksum ID 5, e.g., just a copy of modular, CRC32 or whatever or in-house algorithm for sure. When KARI put the 5 at the checksum type field in Metadata PDU, with all the interoperability testing with space agencies, the ‘Unsupported checksum type’ condition code should be designated at the FIN PDU from a receiving CFDP entity since all space agencies except KARI doesn’t have any reference for the checksum ID of 5. How do you think?

Cheol

From: SIS-CFDPV1 <sis-cfdpv1-bounces at mailman.ccsds.org<mailto:sis-cfdpv1-bounces at mailman.ccsds.org>> On Behalf Of Burleigh, Scott C (US 312B) via SIS-CFDPV1
Sent: Wednesday, December 11, 2019 1:21 AM
To: sis-cfdpv1 at mailman.ccsds.org<mailto:sis-cfdpv1 at mailman.ccsds.org>
Subject: [SIS-CFDPV1] revised draft specification

Hi, all.  Following some additional discussion of the RID dispositions proposed in the Darmstadt meeting, we now have got a final (I hope) draft specification posted to CWE at https://cwe.ccsds.org/sis/docs/SIS-CFDPV1/Draft%20Documents/727x0b5%20--%20Specification%20--%20Blue%20Book/727x0p42_working%204.doc?Web=1; PDF is at https://cwe.ccsds.org/sis/docs/SIS-CFDPV1/Draft%20Documents/727x0b5%20--%20Specification%20--%20Blue%20Book/727x0p42_working%204.pdf.

This draft has gone to the Secretariat for final editing, but the technical stuff won’t be changing unless we discover a problem in interoperability testing.

Speaking of which, I hope to do that (small, supplementary) interoperability test in January.  Cheol and Chenyunjun, please take one more look at the markups in the revised spec to make sure the final tweaks are reflected in your code?  I need to modify ION’s CFDP as well and hope to have that done in early January.  We’ve discussed a plan for this supplementary testing, and we should try to tie off that discussion and finalize the plan sometime in the next couple of weeks.  (Holidays in the U.S. will complicate this, but it should still be doable.)

The end is in sight, I swear.

Scott

_______________________________________________
SIS-CFDPV1 mailing list
SIS-CFDPV1 at mailman.ccsds.org<mailto:SIS-CFDPV1 at mailman.ccsds.org>
https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-cfdpv1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/sis-cfdpv1/attachments/20191211/0bd476c3/attachment-0001.html>