[Css-csts] RE: Buffered Data Processing Procedure

[John Pietras]

Martin,
I have some comments on the most recent versions of the PROCESS-DATA operation and (simple) DP and BDP procedures. As usual, please accept my apologies in advance if I ask questions for which the reasons/answers were discussed in CSTSWG meetings that I didn't attend.


1.       The name of the parameter in the PROCESS-DATA invocation is "data", but the DP and BDP specifications are written (almost) universally in terms of the "data unit" of the PROCESS-DATA invocation. The change in terminology between operation and procedure might be interpreted as a refinement of the content of the data parameter for the purposes of the procedures, but in fact I don't think that there really is any difference in the content or format of either (i.e., the 'data unit' is still an abstract entity requiring further specification. Is there is reason that "data unit" is used in the procedures and not just "data"?



2.       There are five parameters identified throughout the two procedures that are used to constrain or direct the behavior of the procedures: (1) the size of the input queue (identified as being defined by the service using the procedure), (2) the size of the input queue (identified as being defined by the service that uses the procedure), (3) the BDP transfer mode (how it is set is not identified), (4) maximum transfer buffer size (identified as a managed parameter), and (5) processing-latency-limit (identified as a "global" managed parameter, a managed parameter, and as being specified by the service that uses the procedure - more on this in a moment). The "Parameter Values to be Specified Outside of the Framework" annex of the CSTS SFW is a repository for such parameters - when the DP and BDP procedures are eventually integrated into the CSTS SFW, these parameters should be included in the cited Annex. Regarding the processing-latency-limit parameter, you (Martin) included a comment asking whether being labeled as a "managed parameter" implied being set by Service Management and if so, how that affects the subsequent statement that this parameter is set by the service that uses the procedure. My opinion is that calling something a managed parameter *does* imply Service Management. If we need a more-generic term, we could call such a parameter a "configuration parameter" or "configurable parameter", and then state the method by which it is set. The most general method is to set by means specified by the service that uses the procedures, because such a service can always specify that for that service the parameter is managed. Only if we are sure that a configuration/configurable parameter should always be set by management for all direct and derived applications of a procedure should the specification of that procedure declare a parameter to be managed. Note that deferring how a configurable parameter is set to the derived service also implies that the service can fix such a parameter to a single value - e.g., a service could be defined that uses only the Complete transfer mode (Note, too, that the Transfer Mode should be identified in the BDP specification as a configuration/configurable parameter and a statement made about how it is set (e.g., deferred to the service that uses the procedure)).



3.       Data Processing (DP) procedure, section 2.1.2.2 (Concept), first paragraph (minor point of grammar): "When the Service Provider receives a PROCESS-DATA invocation he stores it on an input queue ..." should be rephrased as either "When the Service Provider receives a PROCESS-DATA invocation the Service Provider stores it on an input queue ..." or "When the Service Provider receives a PROCESS-DATA invocation it stores that PROCESS-DATA invocation on an input queue ...". "He" is normally used to refer only to a male person*, not an automated process (generally this also applies to "she", but sometimes things like ships are personified as "she". ) [*"He" has also traditionally been used to when the sex of the person is unknown or not relevant - e.g., "When the operator starts up the system, he first enters his User ID and password." However, at least in the US, over that past few decades there has been a move toward using either gender in such cases.]



4.       DP procedure and Buffered Data Processing (BDP) procedure, multiple instances. The CSTS SFW has an informal typographical convention (that is, one that is not explicitly stated in 1.6.3.3) of naming formal entities (e.g., Release Timer, Transfer Buffer, Recording Buffer) with initial caps to distinguish them as terms with specific formal behaviors, characteristics, etc.. I suggest applying this convention to Input Queue and (for BDP) Latency Timer. [NOTE regarding CSTS SFW - should we formalize this convention in 1.6.3.3?]. Also, should the BDP specify the units of the Latency Timer (presumably, seconds)?



5.       DP procedure, section 2.1.2.2 (Concept), end of third paragraph "This ['production interrupted' notification] report informs the user that the input queue may fill up and cause the service instance to be aborted if the Production Status remains in a non-operational state." There is no normative behavior specified regarding aborting the service if the queue fills while Production Status is non-operational.


6.       DP procedure, 2.1.3.2.5, NOTE, states "The Service Provider queues the PROCESS-DATA invocations so that the Service User may transfer the PROCESS-DATA invocations well in advance of the processing taking place." However, this rationale is negated by the Timely Transfer mode of the BDP, which has the processing-latency-limit to limit how far in advance the invocations can be transferred. I think there is a more universal reason for the input queue, which could be stated as "The input queue allows the timing of the processing of decouples the timing of the processing of the PROCESS-DATA invocations form the transfer of those invocations." For services that use a derived procedure that does have the equivalent of the processing-latency-limit, that could include allowing long delays between transfer and processing. But it also applies to the Timely mode of BDP in cases where the process pops the invocations from the queue asynchronously with respect to how they are transferred.


7.       DP procedure, 2.1.3.3.1 states "The Service Provider shall remove PROCESS-DATA invocations from the input queue and process the data units included as soon as possible ...". Is the actual processing a part of the derived procedure itself, or is it part of a production process that the DP (or DP-derived) procedure hands the invocation to? The distinction may be important for how service specifications are written using DP procedures. If the processing is part of the derived procedure itself, then services using such derived procedures will have to have the, behavior and state machines for those derived procedures include the production processing. If, on the other hand, the processing is considered to be a separate "state machine", the impact on the DP-derived procedure itself can be minimized.  The existence of the input queue supports this second interpretation if that's the one that applies: the queue is basically a mailbox connecting the data transfer process to the "processing" process. For the Forward Frames service, for example, my plans have been to adopt this second interpretation (I took a similar approach in defining the enhanced functionality of the EFCLTU service).

Note that the current wording of the requirements under 2.1.3.3 is still valid in either case, because "Service Provider" can be interpreted in the broader sense to be not only the transfer service provider but the complete service provider (i.e., the Complex).  But if  interpretation is that it is the procedure that does the processing then 2.1.3.3 contains requirements on the procedure, whereas if it is a separate production process that does the processing then 2.1.3.3 contains common requirements on all production processing associated with the DP-derived procedure, and perhaps we might want to make that distinction.



8.       DP procedure, 2.1.4.2.2.2 (a), "the processing of the data completed without aborting ..." I think that deleting "without aborting" would not change the meaning of this statement. "Abort" has a special (and different) meaning in CSTS and its use in this sentence could cause confusion.



9.       DP procedure, 2.1.5.1, Table 2-4 (State Table): the "interrupt notified" flag can be set to TRUE by incoming events 3  ((ProcessDataInvocation)) and 6 ('production interrupted"), but there is no way to reset the flag to FALSE. It appears that adding "set interrupt notified" to FALSE" after "'notify 'production operational''" for event 8 ('production operational') will fix it.



10.   DP procedure, 2.1.5.1, Table 2-4 (State Table): The entries {'procedure to association abort 'xxx''} should be {procedure to association abort 'xxx'} - no single quotation marks around procedure to association abort 'xxx'.



11.   Buffered Data Processing (BDP) procedure, 1.1.2.2 (Concepts): My apologies if this was discussed in CSTSWG meetings that I didn't attend, but I'm having a hard time conceiving of a use case for having a latency limit when the BDP is operating in Complete mode. Was one ever expressed, or was this possible combination included just in case someone might come up with a useful application?



12.   Buffered Data Processing (BDP) procedure, 1.1.2.2 (Concepts):  It might be useful to have a diagram or two to illustrate the Transfer Buffer and Input Queue and the relationship(s) between them.



13.   Buffered Data Processing (BDP) procedure, 1.1.2.2 (Concepts), last sentence of second paragraph: "The maximum size of the Transfer Buffer expressed as the number of PROCESS-DATA invocations that may be stored in the buffer is defined as a managed parameter." Suggest a slight rewriting for clarity, as follows "The maximum size of the Transfer Buffer is expressed as the number of PROCESS-DATA invocations that may be stored in the buffer. It is defined as a managed parameter."



14.   BDP procedure, 1.1.2.2 (Concepts), last sentence of second paragraph, states that the maximum transfer buffer size is expressed in terms of the number of P-D invocations. There is a relationship between the transfer buffer and the input queue, namely the contents of the transfer buffer aren't put on the input queue until the full buffer contents can be placed there. However,  the "units" of the input queue itself are not defined. Is the intention that the input queue also be specified in terms of P-D invocations? If so this could result in less data being queued than the input queue can physically accommodate, but that might be a secondary consideration if the actual number on the queue is expected to more important than the data volume. Note that if the input queue definition is going to be refined by the BDP procedure, then the statements in 2.1.2.2 (Concepts) and 2.1.3.2.6 of the DP procedure regarding the size of the input queue being defined by the service be modified to allow modification by a procedure that is derived from this procedure or a service that uses this procedure.



15.   BDP procedure, 1.1.2.2 (Concepts), beginning of sixth paragraph, states "In Complete Transfer mode, the Service Provider stops reading data from the underlying data communication service when the available space on the input queue drops below the maximum size of a Transfer Buffer and resumes reading data when there is sufficient room on the input queue to store all PROCESS-DATA invocations that might be included in a maximum sized Transfer Buffer." Unless I am mistaken, operating a CSTS over ISP would have all procedures operating over the same TCP connection. If that is true, then if a service were to allow multiple concurrent instances of the BDP procedure, any one of them running in Complete mode would suspend transfer for all of them (even ones running in Timely mode, which would be absolutely counter to the purpose of Timely mode). I can't think of any reason why someone would want to create a service that would use multiple concurrent BDP procedure instances, but that doesn't mean that someone might not try it sometime in the future. The BDP procedure specification should (a) outright prohibit the use of multiple BDP instances where at least one of the instances is operating in the Complete mode or (b) include a note of some sort  identifying the consequences of defining a service that allows multiple instances of BDP to be instantiated if at least one of the instances can be in Complete mode.



16.    BDP procedure, 1.1.2.2 (Concepts), last sentence of sixth paragraph, states "In effect, data will always be transferred completely accepting an additional delay above the processing latency controlled by the processing-latency-limit parameter." I do not understand what the "accepting additional delay" clause is attempting to say. This same phrasing is used in 1.1.4.2.3.1 NOTE 1.


17.    BDP procedure, 1.1.4.2.2, NOTE: Should "PROCESS-DATA invocations I independent" be "PROCESS-DATA invocations is independent"?



18.   [NOTE - this comment is really a comment on the CSTS SFW itself, but is was raised by the following statements in the BDP specification] BDP procedure, 1.1.4.2.3.3, NOTE 2 states "In this situation [that is, when the Service Provider has stopped reading the data from the underlying comm service while in Complete mode] the Service User will not be able to terminate the procedure nominally by invocation of the STOP operation until all transmitted data have been read and queued by the Service Provider. The only way to terminate the procedure earlier is by invocation of PEER-ABORT." Terminating the procedure while in this suspended-reading condition via the PEER-ABORT is possible when the underlying comm service includes ISP-1 because ISP-1 uses TCP, which provides the urgent data feature. However, the capability to have a PEER-ABORT bypass the flow control of the underlying comm service is never explicitly called out in either the specification of the PEER-ABORT operation or the characteristics of the underlying comm service (1.3.1) of the CSTS SFW. There *are* statements in the CSTS SFW to the effect that ISP-1 is the "default" underlying protocol, but it's still not normative/mandatory in the text sections. The only place in the CSTS SFW that ISP-1 is essentially normative for CSTSes is in the ASN.1 specification of the PEER-ABORT diagnostics. The CSTS SFW should take one of two positions: (a) declare ISP-1 to be the *required* underlying protocol of all CSTSes, in which case the phrases such as "default" underlying service, "assumed to be used", and "applicable to" should be replaced by more direct statement (i.e., ISP-1 is the underling protocol'); or (b) still allow ISP-1 to be one of possibly several underlying protocols, in which case the characteristics of the underlying comm service in 1.3.1 need to be expanded to allow for the bypassing of flow control mechanisms.



19.   BDP procedure, 1.1.4.2.3.4: Insert "queue" after "input" in "In the Timely transfer mode, when the available space on the input does not allow ..."



20.   BDP procedure, 1.6.1.1, table 1-2 (State Table): Some of the actions are italicized, and it appears that these italicized sections are those that are copied from the parent DP procedure. If this is the case a note explaining that would be helpful.


Best regards,
John



From: css-csts-bounces at mailman.ccsds.org<mailto:css-csts-bounces at mailman.ccsds.org> [mailto:css-csts-bounces at mailman.ccsds.org] On Behalf Of Martin Götzelmann
Sent: Thursday, January 24, 2013 9:59 AM
To: CCSDS_CSTSWG (css-csts at mailman.ccsds.org<mailto:css-csts at mailman.ccsds.org>)
Subject: [Css-csts] Buffered Data Processing Procedure

Members of the CSTS WG,

I have had another go at the Buffered Data Processing Procedure and uploaded the result to the CWE folder:
Cross Support Services Area (CSS) > Documents > CSS-CSTS > CWE Private > Working Materials > DPP Working Papers
The name of the file is CSTS Forward _BDP-24Jan2013.docx

I have entered a number of questions related both to the presentation of a derived procedure and with respect to certain specifications and would be very much appreciate your comments.

As I was working on this procedure I discovered a few typos on the simple DPP Version 10. I have corrected these on the version stored on the CWE and have not created a new version.

Kind Regards, Martin

_______________________
Martin Götzelmann
Principal Consultant Technology

Telespazio VEGA Deutschland GmbH
Europaplatz 5
64293 Darmstadt
Germany
Tel        : +49 (0)6151 8257-147
Fax       : +49 (0)6151 8257-799
Email   : martin.goetzelmann at vega.de<mailto:martin.goetzelmann at vega.de>
Web     : www.telespazio-vega.de<http://www.telespazio-vega.de/>

Registered office/Sitz: Darmstadt, Register court/Registergericht: Darmstadt, HRB 89231; Managing Directors/Geschäftsführer: Sigmar Keller, John Lewis, Yves Constantin

Notice of Confidentiality
This transmission is intended for the named addressee only. It contains information which may be confidential and which may also be privileged.  Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else.  If you have received this transmission in error please notify the sender immediately.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ccsds.org/pipermail/css-csts/attachments/20130128/c6c6d82c/attachment-0001.html