[CESG]
RE: CESG-P-2012-04-001 Approval to release CCSDS 521.0-P-1.1,
Mission Operations
Message Abstraction Layer (Pink Sheets, Issue 1.1) for CCSDS Agency review
Sam Cooper
Sam.Cooper at scisys.co.uk
Wed May 9 04:43:04 EDT 2012
Hi Peter,
I'm not going to get into the old discussion of what is a magenta book
and what is a blue book, we have been there and we have done that.
So, to focus on your question below:
If I try and parse out the underlying specification structures that are
being addressed here I think it is these:
1. Data type specification (how the data structures in a MAL
message are defined)
2. MAL data types (the primitive data objects used to construct a
data type specification)
3. Data type specification language (the technical method used for
specifying a data type or a data type specification)
4. Transport encoding (the translation of an abstract data type
specification into an unambiguous data structure specification that can
be used interoperably)
5. Transport protocol mapping (the unambiguous specification of the
MAL abstract message exchanges and behavior onto an implementable
transport (or message) protocol that has interoperability properties)
If this is so (please confirm), then I think what these sec 2.2.5, 4.1.1
and 4.16 sections are trying to say is that the MAL spec has defined a
specific set of (abstract) MAL data types and a specific (abstract) MAL
data types specification. But what you wish to permit is for the
implementor of a service that uses the MAL specification to not be bound
by this specification and instead be permitted to use any other specific
data type and structure specification language if they wish to do so.
The specific example you mentioned, XML, is one that is now in common
use. Unless I misunderstand something, the effect that this
relaxation of the rules has is that these parts of the MAL specification
may be ignored and implementers have the option of adopting whatever
data type specification language that they wish to use in defining their
specific services and transport encodings.
Yes, you are correct in the above, in the MAL we provide a data type and
structure specification language that can be used to define the body of
the messages of a specific service. Previously that was the only option
available to use. All we have changed now is to say, for a specific
service you can use an alternative data type and structure specification
language (XML Schema is the most obvious one) if you wish. However, I
strongly believe that it does not reduce interoperability; exactly in
the same way that having a bespoke transport encoding doesn't, the
service is still interoperable as long as you share the relevant
information.
Further comments are inserted below, let me know what you think.
Regards,
Sam.
________________________________
From: Shames, Peter M (313B) [mailto:peter.m.shames at jpl.nasa.gov]
Sent: 08 May 2012 20:31
To: Sam Cooper
Cc: Nestor.Peccia at esa.int; moims-sc at mailman.ccsds.org; Thomas Gannett;
CCSDS Engineering Steering Group - CESG Exec
Subject: Re: CESG-P-2012-04-001 Approval to release CCSDS 521.0-P-1.1,
Mission Operations Message Abstraction Layer (Pink Sheets, Issue 1.1)
for CCSDS Agency review
Hi Sam,
It is clear to me that we keep getting wrapped around the axle of just
what is the MAL and how it must be specified to really be a Blue Book in
the sense meant in the "Organization and Processes for the Consultative
Committee for Space Data Systems". CCSDS A02.1-Y-3, Yellow Book. Issue
3. July 2011.
Namely:
* Recommended Standards must be complete, unambiguous and at a
sufficient level of technical detail that they can be directly
implemented and used for space-mission interoperability and cross
support.
* Recommended Standards must say very clearly, "These are the
technical properties of what the implementer must build and how it must
behave if it is to be compliant and interoperable."
In the case of the MAL, and only in the case of the MAL to my knowledge,
we have allowed a very abstract specification to be accepted as a Blue
Book. At the CESG insistence the MAL Blue Book also includes the
explicit statement, in Sec 2.1:
The MO service specifications and the MAL are abstract in their
definition; they do not contain any specific information on how to
implement them for a particular programming language or transport
encoding. Moving from the abstract to the implemented system, two other
specifications are needed. One is the Language Mapping that states how
the abstract MAL and MO service specifications are to be realised in
some specific language: this defines the API in that language. The
second is the transport mapping from the abstract MAL data structures
into a specific and unambiguous encoding of the messages and to a
defined and unambiguous mapping to a specific data transport. It is only
when these mappings are defined that is possible to implement services
that use the MAL interface and use the transport bindings to exchange
data. (For further information on this, see reference [1].)
The expected functionality of the Transport Interface is described in
Sec 3.7. This section "specifies what facilities must be made available
to a compliant MAL and also the required behaviour of the Transport."
Please do keep in mind that it is unprecedented in CCSDS, that a Blue
Book, in and of itself, is not a complete, unambiguous, and directly
implementable specification that has the required properties of
interoperability and cross support. It is only when an encoding and
transport mapping are also specified that all of these properties are
met.
The reason for coming back to this is that there is this perceived
tendency in the MAL to strive for ever more flexibility in what is
permitted. The issue, from the point of view of providing what are
truly exact and unambiguous specifications that can be directly
implemented, is that every step that is taken toward implementation
flexibility seems to lead further away from this requirement on Blue
Books. And so your Blue Books, and their mods, get an extra careful
reading because of this situation. So here we are, again, trying to
find an acceptable path forward.
If I try and parse out the underlying specification structures that are
being addressed here I think it is these:
6. Data type specification (how the data structures in a MAL
message are defined)
7. MAL data types (the primitive data objects used to construct a
data type specification)
8. Data type specification language (the technical method used for
specifying a data type or a data type specification)
9. Transport encoding (the translation of an abstract data type
specification into an unambiguous data structure specification that can
be used interoperably)
10. Transport protocol mapping (the unambiguous specification of the
MAL abstract message exchanges and behavior onto an implementable
transport (or message) protocol that has interoperability properties)
If this is so (please confirm), then I think what these sec 2.2.5, 4.1.1
and 4.16 sections are trying to say is that the MAL spec has defined a
specific set of (abstract) MAL data types and a specific (abstract) MAL
data types specification. But what you wish to permit is for the
implementor of a service that uses the MAL specification to not be bound
by this specification and instead be permitted to use any other specific
data type and structure specification language if they wish to do so.
The specific example you mentioned, XML, is one that is now in common
use. Unless I misunderstand something, the effect that this
relaxation of the rules has is that these parts of the MAL specification
may be ignored and implementers have the option of adopting whatever
data type specification language that they wish to use in defining their
specific services and transport encodings.
If I have somehow misunderstood that you are actually proposing please
let me know. Maybe I am just not getting the concept.
I understand the motivation for wishing to add this flexibility, but I
also believe that it further weakens the status of the MAL as a Blue
Book. To my eyes it says, in essence "We have defined this abstract
spec, but you can ignore it if you wish to do something different." My
statement is "be careful what you ask for". Each step away from really
defining a Blue Book, with the required properties, makes it less likely
that what you publish will be useful in defining interoperable services.
That said, see below for my specific comments on the approach you wish
to take.
Regards, Peter
From: Sam Cooper <Sam.Cooper at scisys.co.uk>
Date: Tuesday, 8 May 2012 7:06 AM
To: Peter Shames <peter.m.shames at jpl.nasa.gov>
Cc: Nestor Peccia <Nestor.Peccia at esa.int>, MOIMS-SC MOIMS-SC
<moims-sc at mailman.ccsds.org>, Tom Gannett <tomg at aiaa.org>
Subject: CESG-P-2012-04-001 Approval to release CCSDS 521.0-P-1.1,
Mission Operations Message Abstraction Layer (Pink Sheets, Issue 1.1)
for CCSDS Agency review
Hi Peter, please find my response to your comments (BTW, I have
merged your comments relating to similar issues together and also
included the comments from the annotated document with the referenced
text (in italics) for completeness):
1) The most significant is the notion that alternative data type
specification languages other than those defined in the MAL might be
used (Sec 4.1.1 and elsewhere). How do you achieve interoperability, or
even know what encoding has been used, if this sort of free adoption of
other data type spec languages is allowed? Shouldn't there be some sort
of universal config message, or MIME type spec, or something else used
to signal the encoding actually being used?
2.2.5 Operation Template
...
The listed types are expected to be MAL data types (or
compositions derived from these) as defined in section 4 of this
specification, however, any data type specification may be used although
interoperability with other entities cannot be guaranteed in this case.
4.1.1 MAL Data Types
...
Other data type specification languages (such as XML Schema) may
be used to specify the message bodies in service specifications, however
support for these other specification languages may not be universal and
may limit use of a specification to specific deployments (such as ground
only).
4.1.6 Containing Abstract Elements
...
Other data type specification languages may allow the
containment of abstract elements however the MAL data type specification
does not allow this.
It is unfortunate that I used the word interoperability in the
first example as that is misleading. What this is referring to is that a
service specification may use a different data type model for the
specification than that of the MAL; this means that the MAL defines one
type model but others can be used in a service specification. This is
completely separate from the encoding used at runtime. As there is only
EVER one specification of the service then all users of that service
will be using the same specification, and therefore will all see that it
defines the specification message types in (for example) XML Schema. So,
there is no interoperability problem as all will see the same
specification, there is no runtime encoding problem because that is
separate from the specification (as these are encoding independent). The
only issue for implementers may come from the fact that they may not
know how to represent data type model X (XML Schema for example) in
language Y (Java for example) and this is the job of the language APIs.
So, the first point in 2.2.5 shall be reworded to remove the
last 11 words, specifically 'although interoperability with other
entities cannot be guaranteed in this case'. The second point in 4.1.1
is correct but shall be reworded to show that this is referring to the
mappings. The third point in 4.1.6 shall be reworded to show that the
containment of abstract elements is a limitation on the MAL data type
model, i.e. reverse the ordering of the sentence to 'The MAL data type
specification does not allow the containment of abstract elements
however other data type specification languages may.'.
I think if you wish to add in this capability then it would be wise to
carefully define these different terms, somewhat like what I did
earlier, and state clearly just what is it that you are proposing to
allow and what is not permissible. Otherwise this reads to me like "We
have defined this abstract spec, but you can ignore it if you wish to do
something different." and that seems to me to be a recipe for having
your spec ignored or warped to the point where there is no
interoperability possible.
>>> Good suggestion, I will update the text to clarify this in section 2
and make sure the correct terms are used in the other sections.
2) This is confusing. Does it mean that these values are used
in this spec as examples or that these are the only permitted values?
3.1 Abstract service specifications: Overview
...
The MAL specification uses the area identifier of 'MAL' and the
area number of '1'.
All MAL based specifications (including the MAL) must define the
area identifier and number to be used when referencing types and
operations from that area in messages. It is a low level detail of the
specification.
I still find this confusing as it is stated. If this is somehow a
recursive or reflexive definition then you need to find a nice way to
say that. Maybe some statement of the form "In the context of the MAL
spec the Area Identifier "MAL" and the Area Number of "1" are used. In
the context in which the MAL spec is to be applied the Area Identifier
and the Area Number will be defined for any given implementation or
deployment."
>> Good suggestion, however the area number/identifier are service
specific so how about "In the context of the MAL spec the Area
Identifier "MAL" and the Area Number of "1" are used. In the context in
which the MAL spec is to be applied the Area Identifier and the Area
Number will be defined for any given specification."?
3) This is confusing since it directly contradicts the new
requirements in Sec 3.4.1.c, .d, and .e. Which is the correct version,
the one in 3.4.1 or the one described here and used in examples?
3.4.1 Message Header Field Values
...
NOTE -- An alternate representation for Domain is used in other
parts of the specification using a single Identifier. Each part of the
Domain Identifier list is concatenated using the '.' character with the
most significant first. For example 'Agency.Mission.Craft.Subsystem'.
What it is trying to define is an alternative way of
representing a domain name in a single identifier. It is not used in the
message header (as that is fixed as a list of Identifiers as defined in
3.4.1.c, .d , and .e) but could be used in the data messages (not the
header as that is fixed here) of future service specifications if they
wanted to. However, if it is confusing I shall remove it.
I think this is another of those what have we defined, how are we using
it, what freedoms are permitted sort of thing. If what you mean is that
in the MAL Message Header the Domain shall be specified as a list of
comma separated Identifiers, with the most significant first, then say
that. If you wish to define a different data type, like DomainName,
that shall be specified as a single compound name with Identifiers
concatenated using "." then say that. You can then have a rule that
says that Domain shall be used in the Message Header Domain field, but
that DomainName may be used elsewhere, such as in Message Body.
That would be less ambiguous than how it is now stated. Is this what
you meant to define as a et of rules?
>> I shall remove the note, it is only adding confusion and is not
directly needed in the MAL specification. If an alternative
representation is required elsewhere then it shall be defined there.
4) This, and the related discussions on what is permitted in the
message body (Sec 2.2.5), is confusing. Either this is a spec that can
be used to produce interoperable implementations, or it is not. If what
is supposed to be normative text includes these sorts of "if", "maybe",
"this violates" text then it is hardly normative.
3.4.2 Message Body
...
e) If the message body uses the MAL types then the last part of
the message may be defined as a list of an abstract type, this violates
the rules on list specification however at runtime this list shall be
replaced with a list of a derived concrete type.
The text shall be clarified to:
e) The last part of the message body may be defined as a list of
an abstract type.
f) At runtime any list of an abstract type shall be replaced
with a list of a derived concrete type.
Is this a "runtime" encoding or is it really an encoding rule /
transport mapping function? This whole "if the message body uses the
MAL types ..." is exactly the sort of thing that I find troubling.
Please define, in unambiguous terms, just what it is that is permitted
in this spec if an implementation is to be compliant. If you do not
have enough specification capability for your purposes refine the spec
until you do have the required capability in the spec. Do not say "we
can't do this with the rules we have defined, but you can break the
rules and do it another way if you want to."
>>> The above change in green removes the "if the message body uses the
MAL types ..." part and also removes the "we can't do this with the
rules we have defined, but you can break the rules and do it another way
if you want to." part. I shall rephrase the second part (f) to clarify
that runtime means at message encoding time.
Since you seem to want to build in the ability for a user to define a
separate set of data type specifications and message structures in XML
perhaps what you need to think about doing is to create an actual,
concrete, XML specification as part of the ever expanding SM&C document
set. At least then you could deal with these potential ambiguities head
on.
>>> Since we know that it is likely that XML Schema is to be used (for
example Nav messages) then it would make sense that any encoding book
will have to at least mention how it deals with specifications that use
XML Schema (even if to say that it isn't supported). I would say that
any other data type specification language would fall into the bespoke
category; obviously this may change over time when we are all using
quantum computers and the next great thing is upon us! :-)
5) This proposed change, from using "*" as a wildcard to using
"*" for the first sub-key and "0" for subsequent sub-keys strikes me as
confusing and potentially error prone. What possible rationale is there
for changing the rules in this way? This is hardly normal practice.
The fact that you had to add two more requirements just to describe it
is indicative of the problem.
3.5.6.5 PubSub: Subscription Matching
...
a) A sub-key specified in the EntityKey structure shall take one
of three types of value: an actual value, a NULL value, and the special
wildcard identifiervalue of '*' (for the first sub-key only) or zero
(for the other three sub-keys).
b) Entities shall not use the zero value for the first sub-key
as the wildcard value of '*' is reserved for the first sub-key.
c) Entities shall not use the '*' value for the second, third,
and fourth sub-keys as the wildcard value of zero is reserved for these
sub-keys.
The structure that holds these keys was previously defined as
containing four Identifiers, it is now defined as containing a single
Identifier followed by three Integers. It is not error prone due to
normal programming language type checking. The rational for the change
is that in every case that we had used it in we found that the last
three sub-keys were always defined to contain a numeric, and it was
deemed in-efficient to store these in a string (which is what an
Identifier is). Because '*' is not a valid value for an Integer the
value of '0' was chosen as its replacement.
So this is essentially an implementation optimization approach. I
suppose that you could as easily have specified that the "*" was the
abstract wildcard value but that the implementation binding of this
could be done to an integer. That binding, to further remove the
ambiguity of a binding of zero, which might be a valid filed entry,
could have specified the maximum negative signed integer or some such
equally unlikely value. For me it was the "here's a wild card, it's
'*'" followed by "here's a wildcard, it's '0'" that stuck me as being
really odd and potentially confusing.
>>> Well it is not implementation only because the types of the fields
are fixed in the MAL for this, so it will always be Integer.
If you could let me know if you are happy with my responses I
shall make the changes in co-ordination with Tom.
Best regards,
Sam.
Please let me know whether I have interpreted your intentions correctly
in and how you plan to resolve these remaining issues.
Cheers, Peter
___________________________________________________________
Sam Cooper
Technical Specialist - Space Division
SciSys UK Limited
T: +44 (0)117 916 5127
E: sam.cooper at scisys.co.uk | http://www.scisys.co.uk
SciSys UK Limited. Registered in England and Wales No. 4373530.
Registered Office: Methuen Park, Chippenham, Wiltshire SN14 0GB,
UK.
Before printing, please think about the environment.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ccsds.org/pipermail/cesg/attachments/20120509/a19b3ba7/attachment-0001.html
More information about the CESG
mailing list