[Css-csts] Another issue with Real-time Radiometric Data service

[John Pietras]

Members of the CSTSWG ---
Last week I sent you an email describing issues that I had encountered
with trying to apply the CSTS Buffered Data Delivery procedure to the
Real-time Radiometric Data Delivery CSTS (RRD-CSTS) in light of the
requirement that no metadata be lost when the RRD-CSTS is operating in
"real-time" delivery mode (that is, send the data as soon as it is
available from the relevant "production", and discard any data that
backs up due to network congestion in favor of more-recent data). I
outlined several possible solutions, and identified one that I would
attempt to pursue unless and until one of you had good reasons to adopt
a different approach (I have since heard no comment).

I have attempted to apply the selected approach, and so far I think that
it will work for the real-time delivery mode. However, I seem to have
hit a stumbling block with regard to the "complete" delivery mode, which
is also desired according to the minutes of the joint CSTSWG-NavWG
(included in the CSTSWG MoM). 

To summarize the issue, the complete delivery mode works on the
assumption that the data is stored in a Recorded Buffer from which the
data can be retrieved in a time-indexed manner. The user of the CSTS can
arbitrarily select which data is to be delivered out of the Recorded
Buffer by setting the start-generation-time and stop-generation-time via
the START operation that begins the data transfer. 

In the specification of the Buffered Data Delivery CSTS procedure (which
is the procedure that would be the basis for the prime procedure of the
RRD-CSTS), the Recorded Buffer is defined in terms of containing
pre-formed TransferDataInvocation recorded that are to subsequently
played out of the Recorded Buffer via the continuous transfer service
instance. (The Recorded Buffer may or may not also contain synchronous
notification records - that is an ambiguous point addressed in an email
that I sent earlier today.)

In the case of the radiometric data, however, the tracking data cannot
be recorded in the Recorded Buffer in the exact form and sequence in
which it will be played out, because the first data to be sent must
consist of a TDM Header segment followed by a TDM Metadata segment, as
defined in the CCSDS Tracking Data Message (TDM) specification. The
first TDM Metadata segment must contain the metadata that pertains to
the tracking data that follows it, but since the user of the complete
mode specifies the start time of the tracking data to be transferred, to
have the appropriate metadata stored in a way that would allow it to be
simply played out of the Recorded Buffer would require that a metadata
record be placed in the Recorded Buffer every time a new tracking data
sample is recorded. Furthermore, the Buffered Data Delivery process of
simply playing out every subsequent record in the Recorded Buffer would
result in a complete metadata update being sent for every new tracking
data sample, which is extremely inefficient.

The approach that I am following is to redefine the Recorded Buffer as
part of a new Radiometric Buffered Data Delivery procedure. This
redefined Recorded Buffer is not being described as containing
pre-formed TransferServiceInvocation records, but rather it is described
as a process that stores both the tracking data measurements that have
been generated as well as any ancillary data necessary to generate the
TDM metadata associated with any of those tracking data measurements. It
is also a function of the Radiometric Recorded Buffer process to
generate the TDM Metadata segments that are appropriate for the
start-generation-time specified in the procedure's START invocation.

The question that I now ask is whether such a "derivation" of the
Buffered Data Delivery procedure is legitimate. Or, more practically, it
may be legitimate in the formal sense, but it may not be implementable
by any simple extensions of a toolkit-compliant Buffered Data Delivery
software module. If it is essentially a new procedure that just uses
some of the Buffered Data Delivery terminology, would it be better to
define it as a new, service-specific procedure?

In exploring this issue, I have come to believe that the treatment of
the Recorded Buffer in the specification of the Buffered Data Delivery
procedure is unnecessarily constraining. The Procedures Definition
states (in section 5.6.1.2.2.2) that "While the transfer buffer behavior
is part of this procedure, the recorded buffer is not part of the
procedure and must be considered when implementing the service
production", but then the procedure specification goes on to define HOW
data is to be stored in the Recorded Buffer. I believe that it is only
necessary to define what the data must look like as it emerges from the
Recorded Buffer as input to the Transfer Buffer. Statements such as "For
complete delivery mode, the service provider shall read data from the
recorded buffer.  The stored information shall be a TRANSFER-DATA
invocation or the equivalent thereof" (section 5.6.2.3.3) could be
rephrased, for example, to something like "For complete delivery mode,
the service provider shall read data from the recorded buffer in the
form of TRANSFER-DATA invocations or the equivalent thereof", which
avoids the issue of how the data is stored in the recorded buffer. 	

Best regards,
John


John Pietras
Global Science & Technology, Inc. (GST)
7855 Walker Drive
Suite 200
Greenbelt, Maryland, USA
20770-3239
Direct:   +1-240-542-1155
GST Main: +1-301-474-9696
Fax:      +1-301-474-5970