[Smwg] Uniqueness and identification of manifest file within the XFDU Pakcage
John Pietras
john.pietras at gst.com
Tue Feb 6 16:46:24 UTC 2018
Dear Claudia,
As we discussed in today's WG telecon, the issue comes down to ensuring that (a) there is only one manifest file in each XFDU Package, and that manifest file is uniquely identified by the ".xfdu" extension. Here are the sections of the TGFT White Book that (I believe) state these normatively:
4.4.2 Each TGFT XFDU package shall contain one XFDU manifest XML document, as defined in 4.2;
3.2.1.2 (b) The file type (extension) ".xfdu" shall be used to uniquely identify it as the XFDU Manifest and to differentiate it from any other XML-formatted file that might be in XFDU Package.
NOTE - Use of the ".xfdu" extension instead of ".xml" will require XFDU Package parsers to recognize files with the .xfdu extension as XML files and perhaps to convert the extension before
submitting the file to a standard XML parser.
I think that this closes the issue.
Best regards,
John
From: SMWG [mailto:smwg-bounces at mailman.ccsds.org] On Behalf Of John Pietras
Sent: Monday, February 05, 2018 2:29 PM
To: Ciocirlan Claudia <Claudia.Ciocirlan at cnes.fr>; CCSDS SMWG ML (smwg at mailman.ccsds.org) <smwg at mailman.ccsds.org>
Subject: Re: [Smwg] TGFT UPDATE RESULTING FROM EMAIL "TGFT yellow book updates"
Dear Claudia,
Let me respond to your last comment first - yes, I forgot to change the extension on the manifest file from ".xml" (which it had to be in order for XMLSpy to allow me to edit it) to ".xfdu". I have fixed that error, and I have replaced the version of the contained TGFT White Book to Colin's latest (W-0.03) and updated the manifest file accordingly. The resulting zipped XFDU Package file can be found at URL:
https://cwe.ccsds.org/css/docs/CSS-SM/CWE%20Private%20-%20Beta/Book%20Production/Blue/Terrestrial%20Generic%20File%20Transfer/White%20Book/Drafts/xfdu_for_tgft-package.zip-2018-036T16-50-00Z
Now let's talk about your original question, which I now think that I understand with your expanded explanation.
The structure of the XFDU Package file follows this model:
XFDU Package Name (directory/folder)
- <manifest file>.xfdu
- <payload data file>.??? (1..*)
- <metadata file>.??? (0..*)
where the manifest file and the other content files are contained within the XFDU Package directory/folder at the same (immediate) sublevel.
What I think that you are asking for is something like:
XFDU Package Name (directory/folder)
- <manifest file>.xfdu
- <contents> (subdirectory/subfolder)
- - <payload data file>.??? (1..*)
- - <metadata file>.??? (0..*)
i.e., two levels of hierarchy within the XFDU Package.
However, both the XFDU Structure and Construction Rules Blue Book (CCSDS 661.0-B-1) and the (single) ExoMars example XFDU Package that has been provided to me treat the contain both the manifest file and the content file(s) at the same level of hierarchy, directly contained within the XFDU Package.
Here is the pertinent diagram from the XFDU Blue Book:
[Slide1]
The terminology here is a bit inconsistent with not only our usage for TGFT but also the terminology elsewhere in the XFDU Blue Book itself. However, if you read the text that accompanies this diagram, you see that "Package Interchange File" is actually the XFDU Package archive file (e.g., ZIP. TAR). As used in this diagram (and explained in the XFDU Blue Book), "XFDU" is a logical entity that includes the XFDU Package (Package Interchange File) and any data entities that are referenced by the Manifest document within that XFDU Package "file".
But for the purposes of this discussion, the important part is the internal structure of that XFDU Package archive, which shows the Manifest and the other files contained directly within the Package with no enclosed hierarchy. And as I said earlier, that's also the way that that the ExoMars XFDU Package example that Colin provided to me is organized.
Now it may be possible that we don't have to abide by this "flat hierarchy" precedent in order to still be considered "XFDU Blue Book compliant". We could explore that with the MOIMS Data Archive Ingestion WG (the WG responsible for XFDU), but I have asked them several questions for clarification over the past year plus and got back only one response ("we'll get back to you", on 30 December 2016), so I wouldn't hold my breath waiting for feedback on this question if we were to pose it to them. And if we *were* to decide on our own to establish some internal hierarchy within the XFDU Package, then we'd also have to address the issue of what conventions would apply (if any) to the naming of the subdirectory/subfolder that contains the payload data and metadata files.
The intention of the use of the".xfdu" extension was to identify the manifest file, but it sounds like that may not be working well enough. An alternative to creating a two-tiered hierarchy within the XFDU Package folder might be to adopt a more-conspicuous (and more easily detectable) naming convention for the manifest file names. E.g., requiring that each manifest file name start with the string "manifest_", and prohibit any other file in and XFDU Package from starting with that string.
I'm sure that we'll talk about this tomorrow.
Best regards,
John
From: Ciocirlan Claudia [mailto:Claudia.Ciocirlan at cnes.fr]
Sent: Monday, February 05, 2018 12:27 PM
To: John Pietras <john.pietras at gst.com<mailto:john.pietras at gst.com>>; CCSDS SMWG ML (smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>) <smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>>
Subject: RE: TGFT UPDATE RESULTING FROM EMAIL "TGFT yellow book updates"
Dear all,
please find attached the new version of the yellow book. Karen hope you did not modified the last one. If this is the case I will merge the modifications.
* reference to WebDAV changed to HTTP
* reference to the PassReport nominal test case added
* error test cases updated to reflect changes in the standard white book
* "some" has been removed when related to constraints added in the xml schema
John, after reading your email again I still had this item in stand by, waiting for my answer.
Manifest file
Few rules apply to the naming of the XFDU manifest file. However, looking at the examples, I wonder if one rule could be added concerning the location of this file. I expected it to be found at the top level in the archive, however it has been set in the first level directory.
I don't understand this question. Let's talk about it at the January WebEx.
The question is influenced by a general need to find the manifest file. When testing we have to automatically find this manifest file. But this issue is more general than this: getting the metadata associated to the package.
The white book describes the manifest as a single file having the .xfdu extension of the archive. It would be better, in our opinion, to fix the exact position of this file inside the archive. Our first expectations after reading the specification were to find the manifest file at the top level of the archive and not under a directory.
generally speaking we expect to have an archive containing files and directories. The manifest file is a file that applies to the whole archive, a top level file. In the example provided by John we have an archive containing a single folder and this folder contains a manifest and all the files. John isn't that only a packaging misunderstood?
And speaking of XFDU manifest file, I believe it is missing in the last archive xfdu_for_tgft-package.zip-2018-004T22-13-00Z
Thank you. Regards,
Claudia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20180206/ac738f2b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 18965 bytes
Desc: image004.jpg
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20180206/ac738f2b/attachment.jpg>
More information about the SMWG
mailing list