[CESG] Re: CESG-P-2012-04-001 Approval to release CCSDS 521.0-P-1.1, Mission Operations Message Abstraction Layer (Pink Sheets, Issue 1.1) for CCSDS Agency review

Tue May 8 15:30:56 EDT 2012

Hi Sam,

It is clear to me that we keep getting wrapped around the axle of just what is the MAL and how it must be specified to really be a Blue Book in the sense meant in the “Organization and Processes for the Consultative Committee for Space Data Systems”. CCSDS A02.1-Y-3, Yellow Book. Issue 3. July 2011.

Namely:

•Recommended Standards must be complete, unambiguous and at a sufficient level of technical detail that they can be directly implemented and used for space-mission interoperability and cross support.
•Recommended Standards must say very clearly, “These are the technical properties of what the implementer must build and how it must behave if it is to be compliant and interoperable.”

In the case of the MAL, and only in the case of the MAL to my knowledge, we have allowed a very abstract specification to be accepted as a Blue Book.  At the CESG insistence the MAL Blue Book also includes the explicit statement, in Sec 2.1:

The MO service specifications and the MAL are abstract in their definition; they do not contain any specific information on how to implement them for a particular programming language or transport encoding. Moving from the abstract to the implemented system, two other specifications are needed. One is the Language Mapping that states how the abstract MAL and MO service specifications are to be realised in some specific language: this defines the API in that language. The second is the transport mapping from the abstract MAL data structures into a specific and unambiguous encoding of the messages and to a defined and unambiguous mapping to a specific data transport. It is only when these mappings are defined that is possible to implement services that use the MAL interface and use the transport bindings to exchange data. (For further information on this, see reference [1].)

The expected functionality of the Transport Interface is described in Sec 3.7.  This section "specifies what facilities must be made available to a compliant MAL and also the required behaviour of the Transport."  Please do keep in mind that it is unprecedented in CCSDS, that a Blue Book, in and of itself, is not a complete, unambiguous, and directly implementable specification that has the required properties of interoperability and cross support.  It is only when an encoding and transport mapping are also specified that all of these properties are met.

The reason for coming back to this is that there is this perceived tendency in the MAL to strive for ever more flexibility in what is permitted.  The issue, from the point of view of providing what are truly exact and unambiguous specifications that can be directly implemented, is that every step that is taken toward implementation flexibility seems to lead further away from this requirement on Blue Books.  And so your Blue Books, and their mods, get an extra careful reading because of this situation.  So here we are, again, trying to find an acceptable path forward.

If I try and parse out the underlying specification structures  that are being addressed here I think it is these:

  1.  Data type specification (how the data structures in a MAL message are defined)
  2.  MAL data types (the primitive data objects used to construct a data type specification)
  3.  Data type specification language (the technical method used for specifying a data type or a data type specification)
  4.  Transport encoding (the translation of an abstract data type specification into an unambiguous data structure specification that can be used interoperably)
  5.  Transport protocol mapping (the unambiguous specification of the MAL abstract message exchanges and behavior onto an implementable transport (or message) protocol that has interoperability properties)

If this is so (please confirm), then I think what these sec 2.2.5, 4.1.1 and 4.16 sections are trying to say is that the MAL spec has defined a specific set of (abstract) MAL data types and a specific (abstract) MAL data types specification.  But what you wish to permit is for the implementor of a service that uses the MAL specification to not be bound by this specification and instead be permitted to use any other specific data type and structure specification language if they wish to do so.  The specific example you mentioned, XML, is one that is now in common use.    Unless I misunderstand something, the effect that this relaxation of the rules has is that these parts of the MAL specification may be ignored and implementers have the option of adopting whatever data type specification language that they wish to use in defining their specific services and transport encodings.

If I have somehow misunderstood that you are actually proposing please let me know.  Maybe I am just not getting the concept.

I understand the motivation for wishing to add this flexibility, but I also believe that it further weakens the status of the MAL as a Blue Book.  To my eyes it says, in essence "We have defined this abstract spec, but you can ignore it if you wish to do something different."  My statement is "be careful what you ask for".   Each step away from really defining a Blue Book, with the required properties, makes it less likely that what you publish will be useful in defining interoperable services.

That said, see below for my specific comments on the approach you wish to take.

Regards, Peter

From: Sam Cooper <Sam.Cooper at scisys.co.uk<mailto:Sam.Cooper at scisys.co.uk>>
Date: Tuesday, 8 May 2012 7:06 AM
To: Peter Shames <peter.m.shames at jpl.nasa.gov<mailto:peter.m.shames at jpl.nasa.gov>>
Cc: Nestor Peccia <Nestor.Peccia at esa.int<mailto:Nestor.Peccia at esa.int>>, MOIMS-SC MOIMS-SC <moims-sc at mailman.ccsds.org<mailto:moims-sc at mailman.ccsds.org>>, Tom Gannett <tomg at aiaa.org<mailto:tomg at aiaa.org>>
Subject: CESG-P-2012-04-001 Approval to release CCSDS 521.0-P-1.1, Mission Operations Message Abstraction Layer (Pink Sheets, Issue 1.1) for CCSDS Agency review

Hi Peter, please find my response to your comments (BTW, I have merged your comments relating to similar issues together and also included the comments from the annotated document with the referenced text (in italics) for completeness):

1) The most significant is the notion that alternative data type specification languages other than those defined in the MAL might be used (Sec 4.1.1 and elsewhere).  How do you achieve interoperability, or even know what encoding has been used, if this sort of free adoption of other data type spec languages is allowed?  Shouldn't there be some sort of universal config message, or MIME type spec, or something else used to signal the encoding actually being used?
2.2.5 Operation Template
…
The listed types are expected to be MAL data types (or compositions derived from these) as defined in section 4 of this specification, however, any data type specification may be used although interoperability with other entities cannot be guaranteed in this case.

4.1.1 MAL Data Types
…
Other data type specification languages (such as XML Schema) may be used to specify the message bodies in service specifications, however support for these other specification languages may not be universal and may limit use of a specification to specific deployments (such as ground only).

4.1.6 Containing Abstract Elements
…
Other data type specification languages may allow the containment of abstract elements however the MAL data type specification does not allow this.
It is unfortunate that I used the word interoperability in the first example as that is misleading. What this is referring to is that a service specification may use a different data type model for the specification than that of the MAL; this means that the MAL defines one type model but others can be used in a service specification. This is completely separate from the encoding used at runtime. As there is only EVER one specification of the service then all users of that service will be using the same specification, and therefore will all see that it defines the specification message types in (for example) XML Schema. So, there is no interoperability problem as all will see the same specification, there is no runtime encoding problem because that is separate from the specification (as these are encoding independent). The only issue for implementers may come from the fact that they may not know how to represent data type model X (XML Schema for example) in language Y (Java for example) and this is the job of the language APIs.
So, the first point in 2.2.5 shall be reworded to remove the last 11 words, specifically ‘although interoperability with other entities cannot be guaranteed in this case’. The second point in 4.1.1 is correct but shall be reworded to show that this is referring to the mappings. The third point in 4.1.6 shall be reworded to show that the containment of abstract elements is a limitation on the MAL data type model, i.e. reverse the ordering of the sentence to ‘The MAL data type specification does not allow the containment of abstract elements however other data type specification languages may.’.

I think if you wish to add in this capability then it would be wise to carefully define these different terms, somewhat like what I did earlier, and state clearly just what is it that you are proposing to allow and what is not permissible.  Otherwise this reads to me like "We have defined this abstract spec, but you can ignore it if you wish to do something different." and that seems to me to be a recipe for having your spec ignored or warped to the point where there is no interoperability possible.

2) This is confusing.  Does it mean that these values are used in this spec as examples or that these are the only permitted values?
3.1 Abstract service specifications: Overview
…
The MAL specification uses the area identifier of ‘MAL’ and the area number of ‘1’.
All MAL based specifications (including the MAL) must define the area identifier and number to be used when referencing types and operations from that area in messages. It is a low level detail of the specification.

I still find this confusing as it is stated.  If this is somehow a recursive or reflexive definition then you need to find a nice way to say that.   Maybe some statement of the form "In the context of the MAL spec the Area Identifier "MAL" and the Area Number of "1" are used.  In the context in which the MAL spec is to be applied the Area Identifier and the Area Number will be defined for any given implementation or deployment."

3) This is confusing since it directly contradicts the new requirements in Sec 3.4.1.c, .d, and .e.  Which is the correct version, the one in 3.4.1 or the one described here and used in examples?
3.4.1 Message Header Field Values
…
NOTE -- An alternate representation for Domain is used in other parts of the specification using a single Identifier. Each part of the Domain Identifier list is concatenated using the ‘.’ character with the most significant first. For example ‘Agency.Mission.Craft.Subsystem’.
What it is trying to define is an alternative way of representing a domain name in a single identifier. It is not used in the message header (as that is fixed as a list of Identifiers as defined in 3.4.1.c, .d , and .e) but could be used in the data messages (not the header as that is fixed here) of future service specifications if they wanted to. However, if it is confusing I shall remove it.

I think this is another of those what have we defined, how are we using it, what freedoms are permitted sort of thing.  If what you mean is that in the MAL Message Header the Domain shall be specified as a list of comma separated Identifiers, with the most significant first, then say that.  If you wish to define a different data type, like DomainName, that shall be specified as a single compound name with Identifiers concatenated using "." then say that.  You can then have a rule that says that Domain shall be used in the Message Header Domain field, but that DomainName may be used elsewhere, such as in Message Body.

That would be less ambiguous than how it is now stated.  Is this what you meant to define as a et of rules?

4) This, and the related discussions on what is permitted in the message body (Sec 2.2.5), is confusing.  Either this is a spec that can be used to produce interoperable implementations, or it is not.  If what is supposed to be normative text includes these sorts of "if", "maybe", "this violates" text then it is hardly normative.
3.4.2 Message Body
…
e) If the message body uses the MAL types then the last part of the message may be defined as a list of an abstract type, this violates the rules on list specification however at runtime this list shall be replaced with a list of a derived concrete type.
The text shall be clarified to:
e) The last part of the message body may be defined as a list of an abstract type.
f) At runtime any list of an abstract type shall be replaced with a list of a derived concrete type.

Is this a "runtime" encoding or is it really an encoding rule / transport mapping function?  This whole "if the message body uses the MAL types ..." is exactly the sort of thing that I find troubling.  Please define, in unambiguous terms, just what it is that is permitted in this spec if an implementation is to be compliant.  If you do not have enough specification capability for your purposes refine the spec until you do have the required capability in the spec.   Do not say "we can't do this with the rules we have defined, but you can break the rules and do it another way if you want to."

Since you seem to want to build in the ability for a user to define a separate set of data type specifications and message structures in XML perhaps what you need to think about doing is to create an actual, concrete, XML specification as part of the ever expanding SM&C document set.  At least then you could deal with these potential ambiguities head on.

5) This proposed change, from using "*" as a wildcard to using "*" for the first sub-key and "0" for subsequent sub-keys strikes me as confusing and potentially error prone.  What possible rationale is there for changing the rules in this way?  This is hardly normal practice.  The fact that you had to add two more requirements just to describe it is indicative of the problem.
3.5.6.5 PubSub: Subscription Matching
…
a) A sub-key specified in the EntityKey structure shall take one of three types of value: an actual value, a NULL value, and the special wildcard identifiervalue of ‘*’ (for the first sub-key only) or zero (for the other three sub-keys).
b) Entities shall not use the zero value for the first sub-key as the wildcard value of ‘*’ is reserved for the first sub-key.
c) Entities shall not use the ‘*’ value for the second, third, and fourth sub-keys as the wildcard value of zero is reserved for these sub-keys.
The structure that holds these keys was previously defined as containing four Identifiers, it is now defined as containing a single Identifier followed by three Integers. It is not error prone due to normal programming language type checking. The rational for the change is that in every case that we had used it in we found that the last three sub-keys were always defined to contain a numeric, and it was deemed in-efficient to store these in a string (which is what an Identifier is). Because ‘*’ is not a valid value for an Integer the value of ‘0’ was chosen as its replacement.

So this is essentially an implementation optimization approach.  I suppose that you could as easily have specified that the "*" was the abstract wildcard value but that the implementation binding of this could be done to an integer.  That binding, to further remove the ambiguity of a binding of zero, which might be a valid filed entry, could have specified the maximum negative signed integer or some such equally unlikely value.  For me it was the "here's a wild card, it's '*'" followed by "here's a wildcard, it's '0'" that stuck me as being really odd and potentially confusing.

If you could let me know if you are happy with my responses I shall make the changes in co-ordination with Tom.

Best regards,
Sam.

Please let me know whether I have interpreted your intentions correctly in and how you plan to resolve these remaining issues.

Cheers, Peter

___________________________________________________________
Sam Cooper
Technical Specialist – Space Division
SciSys UK Limited
T:  +44 (0)117 916 5127
E:  sam.cooper at scisys.co.uk<mailto:sam.cooper at scisys.co.uk> | http://www.scisys.co.uk

SciSys UK Limited. Registered in England and Wales No. 4373530.
Registered Office: Methuen Park, Chippenham, Wiltshire SN14 0GB, UK.

Before printing, please think about the environment.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ccsds.org/pipermail/cesg/attachments/20120508/da9e1335/attachment-0001.html