[Smwg] action item 2018-1211-01, Clarify constraints on presence of payload and metadata files in an XFDU packages

Tue Jan 8 21:31:24 UTC 2019

John,

I must admit to not having done a detailed analysis (inspecting the updated book), but the approach and items listed below look good to me.  Some minor notes on the items 1 and 2:

1.       From: "...accompanied in in the same XFDU Package.."  To:  ""...accompanied in the same XFDU Package..."

2.      From "...only when it associated..."  To:  "only when it is associated..."

Best regards,
-Erik

From: SMWG <smwg-bounces at mailman.ccsds.org> On Behalf Of John Pietras
Sent: Friday, December 14, 2018 12:18
To: CCSDS SMWG ML (smwg at mailman.ccsds.org) <smwg at mailman.ccsds.org>
Subject: Re: [Smwg] action item 2018-1211-01, Clarify constraints on presence of payload and metadata files in an XFDU packages

CSSMWG colleagues ---
At the 11 December 2018 telecon we discussed the ambiguity in the specification of the relationship between the elements of the XFDU Manifest and the numbers and types of files within the XFDU Package; i.e., there is currently no prohibition on the XFDU Package containing "other" files that are not accounted for in the DataObject and MetadataObject elements of the Manifest.

Serge has suggested deleting 4.2.7.1 ("The metadataSection element shall contain one metadataObject element for each Metadata Object (i.e., metadata file) associated with the payload data file(s) being transferred in the XFDU Package") and 4.2.8.1 ("The dataObjectSection element shall contain one dataObject element for each payload data file in the XFDU Package"), and replacing them with a requirement such as "Each file present in the XFDU package archive other than the manifest shall be described by either a dataObject element or by a metadataObject element." Serge justified this wording by saying that (a) there is nothing stated that normatively  limits  the contents of the XFDU Package to just a single manifest file, one of more payload files, and zero or more metadata files, and furthermore (b) "payload data file" and "metadata file" are not even formally defined. The suggested approach sidesteps the need to distinguish between payload files and metadata files for the purposes of ensuring correspondence between files in the Package and metadataObject/ dataObject elements in the Manifest.

I sympathize and concur with the desire to not create distinctions where it is otherwise not necessary to do so. However, in this case  I think that the distinction between the two kinds of files is very important in understanding what's in the XFDU Package and how those contents relate to the elements of the Manifest. Furthermore, "payload data" is defined in 1.5.2 (g) and we have sections named "Payload Data File Names" (3.2.1.3), "Metadata File Names" (3.2.1.4), "Payload Data File Enclosed within the XFDU Package" (4.3), and "Metadata File Enclosed within the XFDU Package" (4.4). To try to remove these terms in some globally-uniform way would involve more effort than I think we have resources/time to do.

So instead I chose to clarify and formalize the concepts of payload data file and metadata file, and specify that an XFDU Package shall contain only: a single XFDU Manifest file; one or more payload data files; and zero or more metadata files. I have made the following changes to the TGFT White Book:

1.       Changed the "payload data" item in 1.5.2 to "payload data file", and changed the definition to "a file that contains the essential data of the data transfer. One or more payload data files are contained in an XFDU Package, which is the "service data unit" of TGFT. A payload data file may be accompanied in in the same XFDU Package by one or more metadata files that provide information necessary to the interpretation or processing of the payload data."

2.       Added a "metadata file" item in 1.5.2, with the definition "a file that contains data that is used to interpret or process data contained in a payload data file. A metadata file may be contained within and XFDU Package only when it associated with one or more payload data files within that XFDU Package."

3.       Added some wording to section 2.3 to better describe the contents of and constraints on files in XFDU Packages.

4.       Refined the General section (4.1) under Composition Rules for XFDU Packages for TGFT to include payload data files and metadata files. This specifically states that each dataObject element corresponds to a payload file, and each metadataObject corresponds to a metadata file.

5.       Changed 4.2.6.3.1 to "The optional anyMdID attribute of the contentUnit element shall be present if the XFDU Manifest contains a metadataSection element (4.2.7) that contains any metadataObject elements that correspond to metadata files that are needed to interpret or process the payload data file that is represented by the contentUnit. If present, this attribute shall contain the IDREF values of the ID attributes of the metadataObject elements (see 4.3.7.2.1 (a)) that correspond to those related metadata files."

This change corrects an error that had slipped by me until Serge's email referenced it. As originally phrased, the anyMdID attribute would be present in all contentUnit elements is there were any metadataObject elements, whether or not the metadata file(s) had anything to do with the specific payload data file. The text now indicates
that the anyMdID attribute is to be populated only when there are metadata files that are relevant to the specific payload data file.

6.       Changed 4.2.7.1 to "The metadataSection element shall contain one metadataObject element for each metadata file associated with the payload data file(s) being transferred in the XFDU Package."

7.       In section 4.5, [Composition Rules for] TGFT XFDU Package:

a.       Changed 4.5.2 to "The TGFT XFDU package shall contain one enclosed payload data file for every dataObject element in the dataObjectSection of the XFDU manifest."

b.       Changed 4.5.3 to "The TGFT XFDU package shall contain one enclosed metadata file for every metadataObject element in the metadataSection of the XFDU manifest for which the metadataReference element has an href value that begins "file:".

c.       Inserted a new requirement (4.5.4) "Other than the XFDU Manifest file, the TGFT XFDU package shall not contain any files that do not have either (a) a corresponding dataObject element in the XFDU manifest or (b) a corresponding metadataObject element in the XFDU manifest for which the metadataReference element has an href value that begins "file:".

d.       Inserted a new requirement (4.5.5) "Other than the XFDU Manifest, each file in the TGFT XFDU package shall correspond to one and only one of either (a) a dataObject element in the XFDU manifest or (b) a metadataObject element in the XFDU manifest for which the metadataReference element has an href value that begins "file:".
Items (a), (b), and (c) above are specifically aimed at limiting the files in the XFDU Package to only those files that are identified in the XFDU Manifest. I added item (d) because I'm not completely sure that otherwise it is prohibited that a dataObject element and a metadataObject element point to the same file.

These changes have been made to the draft version that has been uploaded to the CWE at https://cwe.ccsds.org/css/docs/CSS-SM/CWE%20Private/Book%20Production/Blue/Terrestrial%20Generic%20File%20Transfer/White%20Book/Drafts/927x01w0_09%20-%20Terrestrial%20Generic%20File%20Transfer%20-%20Working-JVP%2020181214.doc?Web=1

Please review it to see if it solves the issue.

Best regards,
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20190108/3bcc8fee/attachment-0001.html>