[Moims-sc] Binary encoding of lists

[Churlaud Olivier]

Hi all,

I’ll try to summarize the issue with an example, and what would be my decision. As usual, feel free to disagree ☺

Description of the issue

Entity A exchanges data with Entity B.

Entity A sends a MAL List<Attribute> to Entity B.
In a very particular exchange, it happens that all Attributes are Integers, for instance: [1, 9, 0]
With the optimization that Stefan proposes, Entity A can :

  1.  either encode it as an heterogenous list, which has the short form part 0: List<Attribute> [ <int> 1, <int> 9, <int> 0] or a
  2.  or encode it as an homogeneous list, which would have the short form part -11 : List<int> [1, 9, 0]

Why it’s an issue

In case 1), Entity B will indeed receive a List<Attribute> (short form part 0), whereas in case 2, Entity B will receive a List<Integer> (short form part -11), so the transport will have changed the MAL object, which is bad.

On the other hand, we said that Entity B knows which operation is being used, and therefore the expected data. So Entity B knows beforehand if the data is a List<int> or a List<Attributes>, so can do the type casting while decoding the message.

The decision

The point here is to decide between 2 options: can we assume that the data type is known before hand, and that the consumer uses this information to do the right type castings or not?

My conclusion
In my opinion, it should be authorized, otherwise we cannot do this kind of optimization.
The consequence is that the encoding of lists must have two decoding paths:

  *   an heterogeneous list which all elements have the same type can be encoded
     *   as an heterogeneous list (short form part = 0, and short form part of each element repeated)
     *   or has an heterogeneous list (short form part = 0, and short form part of each element repeated): it’s an optimization and it’s legal
  *   An homogeneous list can be encoded
     *   as an homogeneous list (short form part = -11 for integers)
     *   or as an heterogeneous list (short form part = 0, and short form part of each element repeated): it’s stupid but legal

  *   the decoders should be compliant of that.

Since MAL is ambiguous on this, the transport should be prescriptive: this should be described in the MAL SPP/Binary Binding.  For HTTP/XML, we should either say the same thing, or forbid it explicitly (since we don’t need optimization here, I would go for the simplest in terms of book changes).

I hope the point is clearer now. Let’s discuss it next Monday ☺

Cheers
Olivier


De : MOIMS-SC <moims-sc-bounces at mailman.ccsds.org> De la part de Serge Lacourte via MOIMS-SC
Envoyé : vendredi 23 octobre 2026 11:17
À : moims-sc at mailman.ccsds.org
Objet : [Moims-sc] Binary encoding of lists

EMAIL EXTERNE : Cet e-mail provient d'un expéditeur externe. Faites preuve de vigilance avant de cliquer sur un lien ou d'ouvrir une pièce jointe. Vérifiez que l'expediteur et le contenu du message sont légitimes.





Dear WG members,

we discussed several optimizations proposed by Stefan for the binary encoding. One of them concerns the encoding of lists. I understood the proposal as the possible encoding of a list holding elements of a single concrete type with a single encoding of the type instead of repeating the encoding of this type for all the elements of the list. This bothered me, and I try to discuss it more formally here.

I explain in the following my understanding of the MAL requirements. I would like to have your opinion on this.

There is no formal API of an encoding. An informal API of the encoding could be to convert a MAL data into an array of bytes(*1), and the reverse. I would go one step further, asserting that the decoding of the array of bytes which results from the encoding of a MAL data should be the exact copy of the original MAL data, including its exact MAL type.

With MAL v1 this was clear and obvious. However MALv2 introduces grey cases that have probably not been cleared enough, and I believe that some ambiguity may remain. The heterogeneous list is such a case.

The MAL specifies Abstract and Concrete types. An operation may declare an Abstract type in an operation parameter. At runtime, this parameter will be filled with a Concrete type that "derives" from the Abstract type. The "derivation" rules is clear for Composite types, it is less clear for List types. The MAL specification never tells the Abstract or Concrete type of a List. It does define List as a way to define a MAL type from the Abstract or Concrete type of the list elements. Il also defines the concepts of Homogeneous and Heterogeneous lists, both concepts being assigned a Short Form Part. It never formally defines what List types are Abstract or Concrete. My understanding is the following :

  *   a List of Concrete element type is a Concrete type. Its Short Form Part is the negative value of the SFP of the element type.
  *   an Homogeneous List is a global naming for all the lists defined from a Concrete element type
  *   a List of Abstract element type is an Abstract type
  *   an Heterogeneous List is a concrete type. It may hold elements of any MAL type. It is given 0 as SFP.
  *   the statement 4.2.6.1.6 "If the type of the List is Abstract, then the List shall be a Heterogeneous List." is ambiguous. It could be interpreted as : If a service specification specifies a List type from an abstract element type as the parameter of a service operation, then, at runtime, an Heterogeneous List shall be used.

However this interpretation of statement 4.2.6.1.6 is clearly too restrictive. If concrete composite type C extends abstract composite type A, then List<C> can be used in place of a List<A> declared parameter. And list<C> is an Homegeneous List. There are then 2 options matching a List<A> declared parameter :

  1.  Homogeneous List<C>, where C extends A
  2.  Heterogeneous List holding only elements which extend A (runtime check)

In the case 1 above, the parameter will obviously be decoded as a List<C>. The question we have to answer now, which is not considered in the MAL specification, is : in the case 2 above, where an heterogeneous list is provided, should it always be decoded as an heterogeneous list, even if all the enclosed elements are of the same type C? My answer is yes, it should. This is why I reacted to the encoding proposal to allow encoding it as a List<C>. I agree that this optimization is still possible, as long as we agree that the parameter is eventually converted into an heterogeneous list. This is possible as the encoder knows the declared type of the operation.

Serge

(*1) with the split binary, this is not exactly true. It is true for the overall encoding of the MAL message, however the encoding usually refers to the encoding of each individual MAL data type in the MAL message body. The encoding of the MAL header is defined in the transport, with a specific encoding which may not follow the encoding of the body. So the split binary encoding transforms a MAL data into 2 separate arrays of bytes, one for the included booleans, and another for all other included data types.

--

Serge Lacourte

Directeur general

ScalAgent Distributed Technologies SA

tel. +33 4 76 29 79 81

mobile. +33 6 86 47 41 06
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/moims-sc/attachments/20261026/bac9c193/attachment-0001.htm>