<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Georgia;
panose-1:2 4 5 2 5 4 5 2 3 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Georgia",serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">John,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">On the face of it, it seems rather audacious for the ARD to state that any automated capability of any kind can be stated as “currently-available” (as you have noted “…</span>I
am reluctant to state (as the SCCS-ARD currently does) that automated transfer of nonvalidated RM data TDM files over TGFT is a currently-available capability.”) <span style="font-size:12.0pt;font-family:"Georgia",serif">To the best of my knowledge, ARD
== Architecture Requirements Document and is not any kind of service catalog. I think the ARD is supposed to say that if you want to achieve a certain function in standard way these are then the standards that are required, right?
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">Validated vs non-validated radio metric data service has always been something outside the purview of the CSS Area. I believe these concerns/definition of just what validated or
non-validated radio metric data service does are best addressed by the MOIMS NAVWG. And, to the best of my knowledge they generally do not do so – the TDM is a data format designed to support delivery of tracking/radio-metric data, but it does not address/specify
how that data is produced, how it is processed for validation, etc. Indeed, this is what the TDM book says right at the start of section 2 “…</span>This section provides a high-level overview of the CCSDS recommended Tracking Data Message, a message format
designed to facilitate standardized exchange of spacecraft tracking data between space agencies..”
<span style="font-size:12.0pt;font-family:"Georgia",serif"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">We get a near real-time data transfer service for RM/tracking data with TD-CSTS as this leverages the TDM to supply streaming tracking and it indicates what parameters to put in
the TDM segments, But certainly TD-CSTS says nothing about how, for example, ranging data is arrived at in the first place (i.e, it says nothing about sequential tone vs PN ranging, square waves, sine waves, etc.) And I think this is exactly as it should
be – the CCSDS specs for the CSS Area indicate how to transfer and what to transfer but not the service that is producing, in this case, the radio metric data per se.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">I fail to see how the situation is any different for TGFT. It is just files of TDMs packaged in XFDUs, as opposed to TDMs that are streamed in realtime. In either case, the TDM
is produced and then put into some other context (streaming or file) for delivery. In both the TD-CSTS and TGFT cases, they do not say how to go about running the service. It is then specific implementations that in fact work out how to meet the interface
spec given what that implementation’s service does. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">Now, if there is to be standard practice for how a particular file type is to be constructed in slightly-TGFT-restricted XFDUs for subsequent transfer via TGFT, then there can certainly
be a service spec for that service which just points to TGFT for doing the transfer. But it can not be the “job” of the TGFT spec to state how it itself is to used in all the various contexts in which CCSDS member agencies choose to put it to use (by definition,
“mission impossible”). How many TDMs to pack in a file, etc., is clearly a concern for the “upstream” using service and not a TGFT concern. The TGFT recommendation includes a SANA registry so that the various filetypes (called package types as more general
term in TGFT) are recorded, “known”. So I think your concern re “</span>XFDU Package file name has to convey enough information to allow the target application to recognize it”
<span style="font-size:12.0pt;font-family:"Georgia",serif">is already addressed</span>.
<span style="font-size:12.0pt;font-family:"Georgia",serif">Similarly the TGFT recommendation contains service agreement parameters that address your concerns re “…</span>the method (e.g., managed parameters?) by which the TGFT Sender knows the URL of the "in-tray"
of the target TGFT Receiver…” <span style="font-size:12.0pt;font-family:"Georgia",serif">
(please see Annex B of the TGFT document).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">With regard to HTTP(s) PUT, the TGFT document includes a reference to RFC 2616 which is the spec for HTTP (see
<a href="https://datatracker.ietf.org/doc/html/rfc2616">https://datatracker.ietf.org/doc/html/rfc2616</a> ). I believe you will find the necessary details for the PUT method along with other details for HTTP in general. I don’t think it is the job of TGFT
to explain how HTTP works. Please keep in mind that CNES and CAST were able to successfully exchange files via the TGFT specifications – this was accomplished quite simply via unix curl command line calls.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">Best regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif">-Erik<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Georgia",serif"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> John Pietras <john.pietras@gst.com> <br>
<b>Sent:</b> Thursday, August 5, 2021 13:01<br>
<b>To:</b> CCSDS SMWG ML (smwg@mailman.ccsds.org) <smwg@mailman.ccsds.org>; SEA-SA <sea-sa@mailman.ccsds.org>; CCSDS_CSTSWG (css-csts@mailman.ccsds.org) <css-csts@mailman.ccsds.org>; Wolfgang Hell <wo_._he@t-online.de><br>
<b>Cc:</b> Barkley, Erik J (US 3970) <erik.j.barkley@jpl.nasa.gov>; Colin.Haddow@esa.int; Holger.Dreihahn@esa.int; Shames, Peter M (US 312B) <peter.m.shames@jpl.nasa.gov>; Pham, Timothy T (US 3300) <timothy.t.pham@jpl.nasa.gov>; matteo.renesto@telespazio.de;
Wolfgang Hell <wo_._he@t-online.de><br>
<b>Subject:</b> [EXTERNAL] Non-validated EM data delivery via TGFT<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">All ---<o:p></o:p></p>
<p class="MsoNormal">The (proposed) delivery of non-validated radiometric (RM) via Terrestrial Generic File Transfer (TGFT) raises issues that affect the Space Communications Cross Support Architecture Requirements Document (SCCS-ARD) and the Functional Resource
Model (FRM). In a nutshell, I think there is some missing “glue” specification that automates the “pushing” of the NonVal RM data file through TGFT.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’ll start with an overview of the situation. Interestingly, the NonVal RM “service” does NOT correspond either of the IOAG Service Catalog’s “Radio Metric” services (Validated Data RM service and Raw Data RM service). The Validated Data
RM service delivers post-pass (off-line) Tracking Data Message (TDM) files containing RM data that has been validated through a provider-specific, presumably-operator-supervised process. The Raw Data RM service provides TM data samples “as soon as they are
received/built by a Ground Tracking Asset” (aka in “real time”) – typically, every few seconds. The Raw Data RM service is realized by the CSTS Tracking Data Service (CCSDS 922.2).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The NonVal RM service is intended to meet the requirement for what (according to ESA members of the CSTSWG assert} “ESA actually does” – collect raw RM data into TDM-formatted files that are generated over consideably longer intervals than
the TD-CSTS samples – e.g., 20 minutes – and send them via a file transfer protocol (as opposed to the messaging protocol that underlies CSTS service including TD-CSTS). Also, in addition to the tracking data types that TD-CSTS transfers, the NonVal RM service
transfers meteorological, tropospheric, and slant total electron count (STEC) data.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">As of this time, a candidate Non-validated Radiometric Data Collection functional resource (FR) has been defined in the FRM and an associated set of parameters, events, and directives (PEDs) have been defined for registration in the SANA
Functional Resource Registry. The NonVal RM Data Collection FR includes the functions of acquiring the RM data, forming the resultant TDM files, and placing those files is an associated Non-validated RM Data Store. However, for reasons that will be explained
shortly, the FRM does *<b>not</b>* address how those files are subsequently transferred their end-recipient, instead simply stating that the files are transferred by bilaterally-defined means.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The SCCS-ARD, on the other hand, asserts a standard “interface binding signature” in which the transfer of NonVal RM TDMs occurs over TGFT. Indeed this is the desired standardized approach, but not all the standard pieces are in place to
implement it unambiguously introperable.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The issue is in the specification (or, rather, lack thereof) of the process by which the TDM files created by the NonVal RM Data Collection FR are transformed into the TGFT XFDU Packages that are required by TGFT, and subsequently “pushed”
through TGFT to the recipient.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The TGFT XFDU Package is a refined (slimmed-down) and (very slightly) extended version of the CCSDS XFDU Package, which was designed for archiving and retrieval of space data, and supports merging of data from different archives in different
formats. Simply put, the CCSDS XFDU has tons of "features" that are not applicable to the simpler type of file transfer that was envisioned for TGFT. Nevertheless, it was selected as the file packaging mechanism for TGFT in order to reuse an existing CCSDS
standard and "not re-invent the wheel" (although in this case the "wheel" is a military-grade puncture-proof, fire-proof, bomb-proof wheel that we are putting onto a bicycle).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The TGFT XFDU Package is a ZIP or TAR file that contains (a) one or more "payload" data files, (b) zero or more metadata files, and (c) an XML formatted Manifest file that defines the relationships among the enclosed payload data file and
metadata files, and optionally relationships to external metadata files (e.g., externally-defined XML schema files). If the application's "payload" data file format already includes all necessary metadata, then the XFDU Package reduces to just the payload
data file itself and a simple Manifest file. For Non-Val RM data, the payload data file is the TDM file. Among the things that have not yet been specified are whether or not there need to be any accompanying (or referenced) metadata files, and if so, the contents
and/or locations of those metadata files. Can a provider pack more than one TDM into an XFDU Package, or should a new XFDU Package be generated for each TDM file (i.e., are there any timeliness requirements for this less-than-real-time service)?<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">TGFT is a push-only "protocol", so once a NonVal TDM file is generated, the resulting TGFT XFDU Package must be created and pushed through the TGFT Sender to the TGFT Receiver associated with the target application. So the "service
request"(so to speak) to the TGFT Sender entity has the arguments TGFT XFDU Package and the URL of the "in-tray" of that target TGFT Receiver. On the receiving side, the target application somehow "knows" to poll the in-tray for files being sent to it. The
bottom line implication of this is that the XFDU Package file name has to convey enough information to allow the target application to recognize it. So other missing pieces of the puzzle are the naming convention that translates the name of the TDM file to
the more-general file-naming conventions already specified for TGFT, and the method (e.g., managed parameters?) by which the TGFT Sender knows the URL of the "in-tray" of the target TGFT Receiver.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">TGFT uses HTTP over TLS, also known as HTTPS. By my (almost nonexistent) knowledge of HTTP, the "client" must open a TCP connection and then send an HTTP request, at which point the server uses the same TCP connection to return the corresponding
HTTP response. Presumably the TGFT Sender (in the ESLT) is the “client” in this case. I don’t know how this paradigm is intended to be used in a push-only scenario. Does the HTTP request contain the XFDU Package itself (the POST method), or something else
(such as just the URL of the XFDU Package that the TGFT Receiver subsequently uses to pull the file)? What does the TGFT Receiver send back as the corresponding HTTP response? It's not at all obvious to me how this would work, but perhaps to someone who has
more than my Wikipedia-level understanding of HTTP it is obvious. In any case, this, too, should be included in the missing specification.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoNormal">Unless and until we have “standard” answers to these questions, I am reluctant to state (as the SCCS-ARD currently does) that automated transfer of nonvalidated RM data TDM files over TGFT is a currently-available capability.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I would be very happy if someone already has the answers to all these questions and all that would be required would be to document them somewhere (where to document them would be a follow-on question).
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Comments?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best regards,<o:p></o:p></p>
<p class="MsoNormal">John<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>