[SIS-CFDPV1] revised draft specification
Felix.Flentge at esa.int
Felix.Flentge at esa.int
Wed Dec 11 15:37:12 UTC 2019
Hi Scott,
100% agree. Let's get the standard out (we can still improve in 5 years if
necessary).
Regards,
Felix
From: "Burleigh, Scott C (US 312B)" <scott.c.burleigh at jpl.nasa.gov>
To: "Jeremy.Mayer at dlr.de" <Jeremy.Mayer at dlr.de>,
"Felix.Flentge at esa.int" <Felix.Flentge at esa.int>
Cc: "sis-cfdpv1 at mailman.ccsds.org" <sis-cfdpv1 at mailman.ccsds.org>,
"sis-cfdpv1-bounces at mailman.ccsds.org"
<sis-cfdpv1-bounces at mailman.ccsds.org>
Date: 11/12/2019 16:24
Subject: RE: [SIS-CFDPV1] revised draft specification
Guys, please. I believe the draft specification is already perfectly
clear on this, and I am strongly opposed to revising the text in any way
at this point. We had several months of Agency Review, RIDs were filed,
and all of the RIDs have been addressed to the satisfaction of the RID
authors. If there were a significant technical problem here then we would
be obliged to fix it, but I am pretty sure there is none.
The checksum algorithm IDs that currently populate the registry are the
ones that we can select among for interoperability testing. If an agency
wants to interoperate CFDP with another agency, then it must use one of
the checksum algorithms that are listed in the registry.
If an agency wants to use an algorithm that’s not in the registry, for
CFDP interoperation with one or more other agencies, then it needs to add
that algorithm to the registry so that the peer agency can know how to
support it. We have procedures for adding things to registries.
If an agency wants to use an algorithm that’s not in the registry for
private CFDP operation, it can use whatever it likes. It won’t be
compliant, strictly speaking, but since it isn’t interoperating that may
not matter. If it wants to be strictly compliant then it will need to add
the new algorithm to the registry. Again, we have procedures for doing
that.
Let’s please not overthink this. I am pretty sure the current draft
specification enables agencies to use CFDP in an interoperable way. Let’
s move on to testing and publication.
Scott
From: SIS-CFDPV1 <sis-cfdpv1-bounces at mailman.ccsds.org> On Behalf Of
Jeremy.Mayer at dlr.de
Sent: Wednesday, December 11, 2019 1:13 AM
To: Felix.Flentge at esa.int
Cc: sis-cfdpv1 at mailman.ccsds.org; sis-cfdpv1-bounces at mailman.ccsds.org
Subject: [EXTERNAL] Re: [SIS-CFDPV1] revised draft specification
Hi Felix,
I think that, if we even suggest that users have the choice of private
checksums, we have to provide some guidance with regards to the processing
of these checksums. Otherwise we'll wind up in the scenario which Cheol
pointed out where the receiving entity assumes that it can process the
private checksum, but the file fails validation and generates a "file
checksum" failure, since the checksum is private. In my opinion, that
isn't really the correct behavior, since a Unsupported Checksum Type fault
would be more appropriate.
It doesn't have to be much, "the private checksum MAY be implemented." &
"If the private checksum is implemented, the sending and receiving
entities must ensure that an unambiguous selection of checksum algorithms
is ensured for a given file transfer. If this cannot be ensured, an
Unsupported Checksum Type fault SHALL be generated". This also covers the
interoperability testing: if the sending entity can support the selection
of a private checksum while the receiving entity can't determine it, then
it's compliant with the specification.
Given that the filestore request does not contain the desired checksum
type (and my suggestion to add it was simply a workaround to increasing
the MIB complexity), the receiving entity will not know what checksum is
in use until the initial metadata PDU, at which point we have to be able
to deterministically process or reject it.
Thanks,
Jeremy
From: Felix.Flentge at esa.int <Felix.Flentge at esa.int>
Sent: Wednesday, December 11, 2019 9:25:52 AM
To: Mayer, Jeremy
Cc: chkoo at kari.re.kr; sis-cfdpv1 at mailman.ccsds.org;
sis-cfdpv1-bounces at mailman.ccsds.org
Subject: Re: [SIS-CFDPV1] revised draft specification
Hmmh,
I wouldn't like to introduce too much change in the draft at the current
stage.
In general, I think:
1) We could have checksum ID for private / experimental use but its use
should be discouraged.
2) Implementations may or may not support private / experimental checksums
(or be configurable). This could go to the MIB but does not have to (as
for the other checksum ID). It's an implementation matter.
3) If an implementation chooses to support private / experimental
checksums, it will try to apply a checksum which may either succeed or
fail. Which exact checksum to apply is again an implementation matter,
could be per source or generic or guessing based on some other information
...
4) If an implementation chooses not to support private / experimental
checksums, it will raise the Unsupported Checksum Type fault.
For the interoperability testing, I am not sure how much of this has to be
covered. As the private / experimental checksum is, well
'private/experimental', I don't think we have to test something for the
standard interoperability. Dealing with unsupported checksums seems also
more a 'local' entity issue and not so much required for interoperability
testing.
Regards,
Felix
From: <Jeremy.Mayer at dlr.de>
To: <chkoo at kari.re.kr>, <Felix.Flentge at esa.int>, <
sis-cfdpv1 at mailman.ccsds.org>, <sis-cfdpv1-bounces at mailman.ccsds.org>
Date: 11/12/2019 08:49
Subject: Re: [SIS-CFDPV1] revised draft specification
Hi Cheol,
Understood, and to respond to both of your emails at once: I think that
the "private/experimental" checksum ID will cover this scenario, without
requiring the allocation of the limited range of checksum ID's which are
supported by CFDP.
To use your example, if the sending entity uses a private checksum
algorithm, then they would put checksum ID private_use (whichever ID that
may be) into the metadata PDU. Upon receiving that checksum, the receiving
entity MUST search the MIB in order to determine which checksum is used
for that source CFDP entity ID, and process accordingly. If it doesn't
find a checksum, then the "Unsupported Checksum Type" error is raised.
This would still increase the complexity of the MIB and checksum
processing pipeline, but we can outline that the private_use checksum ID
MUST search the MIB to determine the actual checksum algorithm, and shall
fail if more or less than 1 checksum matches for a given CFDP source
entity ID.
Thanks,
Jeremy
From: 구철회 <chkoo at kari.re.kr>
Sent: Wednesday, December 11, 2019 8:46:23 AM
To: Mayer, Jeremy; Felix.Flentge at esa.int; sis-cfdpv1 at mailman.ccsds.org;
sis-cfdpv1-bounces at mailman.ccsds.org
Subject: RE: [SIS-CFDPV1] revised draft specification
Hi, Felix.
If we choose one ID as 'Reserved for Private and/or Experimental Use, that
means a CFDP implementation can support the ID; therefore when the ID is
applied to a filestore request, a receiving entity (who can handle the ID)
will have no problem to process the request. No fault condition can be
issued.
Jeremy’s idea can handle the situation. But a ‘prior coordination
between agencies’ should be made before entering a real testing in order
to know which one is an implemented checksum algorithm (because the
sending entity have to process a file delivery request), or a fake
algorithm (non-support algorithm). I think that could be not what we want
in this context.
Cheol
From: Jeremy.Mayer at dlr.de <Jeremy.Mayer at dlr.de>
Sent: Wednesday, December 11, 2019 4:17 PM
To: Felix.Flentge at esa.int; 구철회 <chkoo at kari.re.kr>;
sis-cfdpv1 at mailman.ccsds.org; sis-cfdpv1-bounces at mailman.ccsds.org
Subject: Re: [SIS-CFDPV1] revised draft specification
Felix,
I agree. Checksum ID's 0-15 are reserved for CFDP, so that checksum can
be added to the blue book without altering the registry. However, it may
make sense to allocate two checksum ID's for general (non-CFDP specific)
use: one for private and one for experimental.
If we did that, checksums 0-3 would be "real" checksums, 4-12 would be
reserved for CFDP, 13 & 14 would be private/experimental, and 15 would
NULL.
Thanks,
Jeremy
From: Felix.Flentge at esa.int <Felix.Flentge at esa.int>
Sent: Wednesday, December 11, 2019 8:10:41 AM
To: chkoo at kari.re.kr; sis-cfdpv1 at mailman.ccsds.org; SIS-CFDPV1; Mayer,
Jeremy
Subject: Re: [SIS-CFDPV1] revised draft specification
Dear All,
I would support that idea. We could be even more general (also to avoid
changing the current draft standard wrt to the MIB) to just have one ID
specified as 'Reserved for Private and/or Experimental Use'.
Regards,
Felix
From: <Jeremy.Mayer at dlr.de>
To: <chkoo at kari.re.kr>, <sis-cfdpv1 at mailman.ccsds.org>
Date: 11/12/2019 08:01
Subject: Re: [SIS-CFDPV1] revised draft specification
Sent by: "SIS-CFDPV1" <sis-cfdpv1-bounces at mailman.ccsds.org>
Hi Cheol,
I can answer your questions about the Checksum registry: First, I'm not
sure why the checksum ID's are non-sequential, since I'm not sure what the
SANA website is sorting by, but if you click the "ChecksumID" column
header, it should sort correctly.
I would suggest that, in the scenario which you outlined, the MIB
containing the supported checksums for a receiving entity should be
updated via TC and/or the desired checksum should be sent during a
filestore request, but that's a more impactful change. I am adverse to a
one-to-one allocation of checksum ID's to agencies, since it ultimately
negates the purpose of the checksum ID and complicates mission operations
and cross support: a single receiving CFDP entity at a ground station can
(and likely will) support multiple missions, some of which may be
performed from different agencies, each of which may have their own
checksumming capabilities and requirements. If a single checksum ID is
allocated to each agency, the receiving entity will be unable to determine
the "real" checksum in use, unless that data is conveyed via some
out-of-band mechanism.
Additionally, different files from a single mission may require different
checksums, either due to bandwidth constraints, mission phase, and/or
avionics capabilities, meaning that the receiving entity will have to be
aware of the mapping between entities, sessions, and checksum
configurations. This is further complicated by the rise of file-based
operations and advanced FDIR systems: it may be possible for the sending
entity to change checksum modes in the event of a failure, in order in
order to reduce power utilization and/or computational complexity.
What would be possible is to allocate a single checksum ID to be "MIB
defined". If that ID is in use, then it's up to the sender/receiver to
determine the checksum out-of-band, but I think that would require some
changes to the blue book.
Thanks,
Jeremy
From: SIS-CFDPV1 <sis-cfdpv1-bounces at mailman.ccsds.org> on behalf of 구철
회 <chkoo at kari.re.kr>
Sent: Wednesday, December 11, 2019 1:48:18 AM
To: sis-cfdpv1 at mailman.ccsds.org
Subject: Re: [SIS-CFDPV1] revised draft specification
Hi, Scott.
I like your idea. I am currently working on the modification of CFDP
software from my side, and I love to perform the supplementary
interoperability test when the updated ION and interoperability test plan
are available.
I don’t see many testing items are required for this supplementary
testing. However as I sent it earlier to you there is a tricky issue with
regard to handling condition code of ‘Unsupported checksum type(1011)’.
In order to make that situation, the ‘Unsupported checksum type’ is
generated, happen, a CFDP (sending) entity should be able to generate a
checksum field by a checksum algorithm that the other CFDP (receiving)
entity does not support. As there are not much choices in selection of
checksum algorithms, it can be tricky because one CFDP entity may have to
implement an unnecessary checksum algorithm that is even not consistent
with CFDP Checksum Identifiers in SANA registry (
https://sanaregistry.org/r/checksum_identifiers).
* short question : why are the number of the checksum ID in the SANA
not consistently sequential? E.g., 4,5,(6?),7,(8?),9,10,11,12,13,14,6,8
,15.
After thinking about this issue, I like to propose an idea. Why don’t we
allocate a Checksum ID to each space agencies, e.g., KARI reserve 5 as a
checksum ID for potential use. Then KARI can freely choose the checksum
algorithm for checksum ID 5, e.g., just a copy of modular, CRC32 or
whatever or in-house algorithm for sure. When KARI put the 5 at the
checksum type field in Metadata PDU, with all the interoperability testing
with space agencies, the ‘Unsupported checksum type’ condition code
should be designated at the FIN PDU from a receiving CFDP entity since all
space agencies except KARI doesn’t have any reference for the checksum ID
of 5. How do you think?
Cheol
From: SIS-CFDPV1 <sis-cfdpv1-bounces at mailman.ccsds.org> On Behalf Of
Burleigh, Scott C (US 312B) via SIS-CFDPV1
Sent: Wednesday, December 11, 2019 1:21 AM
To: sis-cfdpv1 at mailman.ccsds.org
Subject: [SIS-CFDPV1] revised draft specification
Hi, all. Following some additional discussion of the RID dispositions
proposed in the Darmstadt meeting, we now have got a final (I hope) draft
specification posted to CWE at
https://cwe.ccsds.org/sis/docs/SIS-CFDPV1/Draft%20Documents/727x0b5%20--%20Specification%20--%20Blue%20Book/727x0p42_working%204.doc?Web=1
; PDF is at
https://cwe.ccsds.org/sis/docs/SIS-CFDPV1/Draft%20Documents/727x0b5%20--%20Specification%20--%20Blue%20Book/727x0p42_working%204.pdf
.
This draft has gone to the Secretariat for final editing, but the
technical stuff won’t be changing unless we discover a problem in
interoperability testing.
Speaking of which, I hope to do that (small, supplementary)
interoperability test in January. Cheol and Chenyunjun, please take one
more look at the markups in the revised spec to make sure the final tweaks
are reflected in your code? I need to modify ION’s CFDP as well and hope
to have that done in early January. We’ve discussed a plan for this
supplementary testing, and we should try to tie off that discussion and
finalize the plan sometime in the next couple of weeks. (Holidays in the
U.S. will complicate this, but it should still be doable.)
The end is in sight, I swear.
Scott
_______________________________________________
SIS-CFDPV1 mailing list
SIS-CFDPV1 at mailman.ccsds.org
https://mailman.ccsds.org/cgi-bin/mailman/listinfo/sis-cfdpv1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/sis-cfdpv1/attachments/20191211/be404582/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 8524 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mailman.ccsds.org/pipermail/sis-cfdpv1/attachments/20191211/be404582/attachment-0001.bin>
More information about the SIS-CFDPV1
mailing list