[Smwg] TGFT UPDATE RESULTING FROM EMAIL "TGFT yellow book updates"
John Pietras
john.pietras at gst.com
Mon Feb 5 19:28:50 UTC 2018
Dear Claudia,
Let me respond to your last comment first - yes, I forgot to change the extension on the manifest file from ".xml" (which it had to be in order for XMLSpy to allow me to edit it) to ".xfdu". I have fixed that error, and I have replaced the version of the contained TGFT White Book to Colin's latest (W-0.03) and updated the manifest file accordingly. The resulting zipped XFDU Package file can be found at URL:
https://cwe.ccsds.org/css/docs/CSS-SM/CWE%20Private%20-%20Beta/Book%20Production/Blue/Terrestrial%20Generic%20File%20Transfer/White%20Book/Drafts/xfdu_for_tgft-package.zip-2018-036T16-50-00Z
Now let's talk about your original question, which I now think that I understand with your expanded explanation.
The structure of the XFDU Package file follows this model:
XFDU Package Name (directory/folder)
- <manifest file>.xfdu
- <payload data file>.??? (1..*)
- <metadata file>.??? (0..*)
where the manifest file and the other content files are contained within the XFDU Package directory/folder at the same (immediate) sublevel.
What I think that you are asking for is something like:
XFDU Package Name (directory/folder)
- <manifest file>.xfdu
- <contents> (subdirectory/subfolder)
- - <payload data file>.??? (1..*)
- - <metadata file>.??? (0..*)
i.e., two levels of hierarchy within the XFDU Package.
However, both the XFDU Structure and Construction Rules Blue Book (CCSDS 661.0-B-1) and the (single) ExoMars example XFDU Package that has been provided to me treat the contain both the manifest file and the content file(s) at the same level of hierarchy, directly contained within the XFDU Package.
Here is the pertinent diagram from the XFDU Blue Book:
[Slide1]
The terminology here is a bit inconsistent with not only our usage for TGFT but also the terminology elsewhere in the XFDU Blue Book itself. However, if you read the text that accompanies this diagram, you see that "Package Interchange File" is actually the XFDU Package archive file (e.g., ZIP. TAR). As used in this diagram (and explained in the XFDU Blue Book), "XFDU" is a logical entity that includes the XFDU Package (Package Interchange File) and any data entities that are referenced by the Manifest document within that XFDU Package "file".
But for the purposes of this discussion, the important part is the internal structure of that XFDU Package archive, which shows the Manifest and the other files contained directly within the Package with no enclosed hierarchy. And as I said earlier, that's also the way that that the ExoMars XFDU Package example that Colin provided to me is organized.
Now it may be possible that we don't have to abide by this "flat hierarchy" precedent in order to still be considered "XFDU Blue Book compliant". We could explore that with the MOIMS Data Archive Ingestion WG (the WG responsible for XFDU), but I have asked them several questions for clarification over the past year plus and got back only one response ("we'll get back to you", on 30 December 2016), so I wouldn't hold my breath waiting for feedback on this question if we were to pose it to them. And if we *were* to decide on our own to establish some internal hierarchy within the XFDU Package, then we'd also have to address the issue of what conventions would apply (if any) to the naming of the subdirectory/subfolder that contains the payload data and metadata files.
The intention of the use of the".xfdu" extension was to identify the manifest file, but it sounds like that may not be working well enough. An alternative to creating a two-tiered hierarchy within the XFDU Package folder might be to adopt a more-conspicuous (and more easily detectable) naming convention for the manifest file names. E.g., requiring that each manifest file name start with the string "manifest_", and prohibit any other file in and XFDU Package from starting with that string.
I'm sure that we'll talk about this tomorrow.
Best regards,
John
From: Ciocirlan Claudia [mailto:Claudia.Ciocirlan at cnes.fr]
Sent: Monday, February 05, 2018 12:27 PM
To: John Pietras <john.pietras at gst.com>; CCSDS SMWG ML (smwg at mailman.ccsds.org) <smwg at mailman.ccsds.org>
Subject: RE: TGFT UPDATE RESULTING FROM EMAIL "TGFT yellow book updates"
Dear all,
please find attached the new version of the yellow book. Karen hope you did not modified the last one. If this is the case I will merge the modifications.
· reference to WebDAV changed to HTTP
· reference to the PassReport nominal test case added
· error test cases updated to reflect changes in the standard white book
· "some" has been removed when related to constraints added in the xml schema
John, after reading your email again I still had this item in stand by, waiting for my answer.
Manifest file
Few rules apply to the naming of the XFDU manifest file. However, looking at the examples, I wonder if one rule could be added concerning the location of this file. I expected it to be found at the top level in the archive, however it has been set in the first level directory.
I don't understand this question. Let's talk about it at the January WebEx.
The question is influenced by a general need to find the manifest file. When testing we have to automatically find this manifest file. But this issue is more general than this: getting the metadata associated to the package.
The white book describes the manifest as a single file having the .xfdu extension of the archive. It would be better, in our opinion, to fix the exact position of this file inside the archive. Our first expectations after reading the specification were to find the manifest file at the top level of the archive and not under a directory.
generally speaking we expect to have an archive containing files and directories. The manifest file is a file that applies to the whole archive, a top level file. In the example provided by John we have an archive containing a single folder and this folder contains a manifest and all the files. John isn't that only a packaging misunderstood?
And speaking of XFDU manifest file, I believe it is missing in the last archive xfdu_for_tgft-package.zip-2018-004T22-13-00Z
Thank you. Regards,
Claudia
De : John Pietras [mailto:john.pietras at gst.com]
Envoyé : lundi 5 février 2018 14:55
À : Ciocirlan Claudia <Claudia.Ciocirlan at cnes.fr<mailto:Claudia.Ciocirlan at cnes.fr>>; CCSDS SMWG ML (smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>) <smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>>
Objet : RE: TGFT UPDATE RESULTING FROM EMAIL "TGFT yellow book updates"
Dear Claudia,
I'm glad that the updates that I made satisfy your concerns in the areas of XFDU structure and the user/provider, sender/receiver, etc., terminology. My edits were confined strictly to items related to those two areas. I did not attempt (nor am I qualified) to comment on file transfer protocols (WebDAV, PROPFIND, etc.). Those issues were still up to Colin to address.
I see that Colin has posted an updated version (0.03). I haven't looked at it yet to see if and/or how he's addressed WebDAV. I also see that Erik has included TGFT updates on the agenda for tomorrow's telecon, so I'm sure that we'll discuss it then.
Best regards,
John
From: Ciocirlan Claudia [mailto:Claudia.Ciocirlan at cnes.fr]
Sent: Monday, February 05, 2018 3:50 AM
To: John Pietras <john.pietras at gst.com<mailto:john.pietras at gst.com>>; CCSDS SMWG ML (smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>) <smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>>
Subject: RE: TGFT UPDATE RESULTING FROM EMAIL "TGFT yellow book updates"
Hi John,
Great, thanks for the updates. Now it is in line with the other documents.
I am also ok with all your answers to my previous email, thank you.
I went through your update and I am wondering. WebDAV is still present everywhere. For example is section 2.2 "With this in mind the protocol chosen for the TGFT is WebDAV."
In my comprehension WebDAV was no longer imposed, it was at least HTTPs with the possibility to use WebDAV if needed. Did I got it wrong?
Thank you and hear you tomorrow.
Claudia
De : John Pietras [mailto:john.pietras at gst.com]
Envoyé : lundi 29 janvier 2018 15:29
À : Ciocirlan Claudia <Claudia.Ciocirlan at cnes.fr<mailto:Claudia.Ciocirlan at cnes.fr>>; CCSDS SMWG ML (smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>) <smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>>
Objet : TGFT UPDATE RESULTING FROM EMAIL "TGFT yellow book updates"
Dear Claudia,
In updating the Tech Note diagram to use terminology consistently throughout the book, I accidentally switched the in-tray labels and I also mislabeled the files sources for files V and W. The correct diagram is this:
[cid:image001.png at 01D39E5B.F1A102F0]
which is now consistent with your Yellow Book diagram if one assumes, for example, that file T = file V and the Agency B file recipient for file T simples turns the file around to send it back to Agency A.
I have updated the draft TGFT White Book to correct the figure and have posted it at URL:
https://cwe.ccsds.org/css/docs/CSS-SM/CWE%20Private%20-%20Beta/Book%20Production/Blue/Terrestrial%20Generic%20File%20Transfer/White%20Book/Drafts/Terrestrial%20Generic%20File%20Transfer%20927x1w0.02-JVP-rev2-180129.doc
Have you had a chance to look at the other changes that I made in response to your comments, and if so, did my responses satisfy your concerns? Thanks.
Finally, in thinking about the in-tray concept in which an "Agency" (more likely a given node of an Agency) conceptually has a single in-tray for all XFDU Packages from another "Agency", it occurs to me that if different applications (programs or human operators) set their own XFDU Package naming conventions on their own they might accidentally gravitate to conventions that are so simple that it is impossible to tell who the intended file recipient is from the file names in the in-trays. So even though the TGFT Blue Book doesn't establish any naming conventions beyond the limitations on the character set and the timetag convention for the XFDU Package, the SANA Considerations section of Annex C (which is currently To Be Written) should establish (or mandate the establishment of) a TGFT file naming convention registry to ensure that any application that uses TGFT uses file names that are unique to that application and cannot be confused for any other application.
Best regards,
John
From: SMWG [mailto:smwg-bounces at mailman.ccsds.org] On Behalf Of Ciocirlan Claudia
Sent: Thursday, January 25, 2018 5:26 AM
To: CCSDS SMWG ML (smwg at mailman.ccsds.org<mailto:smwg at mailman.ccsds.org>)
Subject: [Smwg] TGFT yellow book updates,
Hello all,
During the last TGFT splitter telecon I took one action to update the yellow book. One of the updates mentioned the image bellow.
I had to switch the A in tray with the B in tray.
[cid:image002.png at 01D39E5B.F1A102F0]
The problem is that this schema is based on the TGFT tech note Figure 2-1 and also specified in section 3.2.3 of the White Book
[cid:image003.png at 01D39E5B.F1A102F0]
I propose to keep the yellow book as it is and only to update the presence of WebDAV; Is that ok with you? If not we need to update all the documents related to this concept.
Regards,
Claudia
Claudia Ciocirlan CNES/DNO/OP/SSO
Mission Operations Ground Segment
phone : +33 (0)5 61 28 17 11
email : claudia.ciocirlan at cnes.fr<mailto:claudia.ciocirlan at cnes.fr>
P Save a tree ! Think before printing
________________________________
Spam<https://filter.gst.com/canit/b.php?c=s&i=01V2yqdRR&m=a4b4368df320>
Not spam<https://filter.gst.com/canit/b.php?c=n&i=01V2yqdRR&m=a4b4368df320>
Forget previous vote<https://filter.gst.com/canit/b.php?c=f&i=01V2yqdRR&m=a4b4368df320>
________________________________
Spam<https://filter.gst.com/canit/b.php?c=s&i=01V6UNHfw&m=b297b37ac04d>
Not spam<https://filter.gst.com/canit/b.php?c=n&i=01V6UNHfw&m=b297b37ac04d>
Forget previous vote<https://filter.gst.com/canit/b.php?c=f&i=01V6UNHfw&m=b297b37ac04d>
________________________________
Spam<https://filter.gst.com/canit/b.php?c=s&i=01V75rjA6&m=ca929a6f01d6>
Not spam<https://filter.gst.com/canit/b.php?c=n&i=01V75rjA6&m=ca929a6f01d6>
Forget previous vote<https://filter.gst.com/canit/b.php?c=f&i=01V75rjA6&m=ca929a6f01d6>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20180205/eaad7988/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 87532 bytes
Desc: image001.png
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20180205/eaad7988/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 37718 bytes
Desc: image002.png
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20180205/eaad7988/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 72058 bytes
Desc: image003.png
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20180205/eaad7988/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 18965 bytes
Desc: image006.jpg
URL: <http://mailman.ccsds.org/pipermail/smwg/attachments/20180205/eaad7988/attachment.jpg>
More information about the SMWG
mailing list