[Moims-sc] Binary encoding of lists

Tue Oct 27 06:05:31 EDT 2026

Hi all,

I agree with Serge up to these two points, which I do not agree with:

*	an Heterogeneous List is a concrete type. It may hold elements of any MAL type. It is given 0 as SFP.
*	the statement 4.2.6.1.6 "If the type of the List is Abstract, then the List shall be a Heterogeneous List." is ambiguous. It could be interpreted as : If a service specification specifies a List type from an abstract element type as the parameter of a service operation, then, at runtime, an Heterogeneous List shall be used.

There is no “runtime Heterogeneous List”. The distinction “Homogeneous” and “Heterogeneous” is purely at specification level. As such a “Heterogeneous List” is not a concrete type. It is a way of saying: We need a list here, whose elements may be of different type. What MAL does wrong here in my opinion, is to assign an SFP of 0. We should have left that out, just as for any other abstract type.

Considering that “Homogeneous” and “Heterogeneous” are only valid concepts for declared types, not actual types, I also do not agree with Olivier’s wording “encode it as heterogeneous/homogeneous list” and “so the transport will have changed the MAL object, which is bad”.

Instead of saying that I would make it very clear along the lines similar to the following:

*	A list declared as homogeneous is encoded as “single-type” list.
*	A list declared as heterogeneous is encoded as “multi-type” list.

*	Optimization: If all elements of a “multi-type” list are of same type, the list may be encoded as “single-type” list.
*	(For completeness: There are also “half-way” optimizations if only the area is the same for all list element types – but this does not add anything to the discussion.)

Don’t nail me down on the words “single-type” and “multi-type”, which I just made up. These two words describe the concrete runtime data structures that need to go on the wire, and as such are both concrete. No types are changed: Although a homogeneous list encoded as “single-type” list looks the same as a “heterogeneous list” encoded as “single-type” list on the wire, the decoder can always tell these apart by using the context information. So yes, of course the decoder needs to know what structure to decode from the specification! The binary encoding is not self-describing and never was. This is already true for a simple String: If you receive [2, 72, 73] there is no way to know if you have to decode a List<Integer> with [72, 73] as elements or the String “HI” without having context information (the specification) available.

Now, one thing I believe Serge was hinting at: What does this mean in a concrete programming language mapping?

A decoder might decode single- and multiple-type lists to different programming language types. But this is implementation-dependent. We don’t care if provider and consumer use the same programming language types on both sides. What we care about is that both sides know exactly what MAL types they work with – and this is fixed by specification plus conveying of type information on the wire where needed.

As an example: Suppose my encoder accepts the programming language object “IntegerList [1, 9, 0]“ for a parameter defined as “List<Attribute>”. I have an efficient encoder, so this gets encoded as single-type list on the wire. On the other side, I have a dumb decoder that knows to expect a List<Attribute> and as such instantiates a programming language object “AttributeList [1, 9, 0]”. Now you might say: “But IntegerList is a different type than AttributeList, serialization changed my type!” But did it? On both sides you still know that your MAL type is “List<Attribute> [Integer, Integer, Integer]”. It is not the responsibility of MAL to convey the exact programming language type information. In cases of different programming languages this usually is not even possible. It is enough to simply convey all MAL type information, and this we made sure.

Cheers,

Stefan

From: MOIMS-SC <moims-sc-bounces at mailman.ccsds.org> On Behalf Of Churlaud Olivier via MOIMS-SC
Sent: Montag, 26. Oktober 2026 19:05
To: LACOURTE Serge (contact) <serge.lacourte at scalagent.com>; moims-sc at mailman.ccsds.org
Subject: Re: [Moims-sc] Binary encoding of lists

Hi all,

I’ll try to summarize the issue with an example, and what would be my decision. As usual, feel free to disagree :)

Description of the issue

Entity A exchanges data with Entity B.

Entity A sends a MAL List<Attribute> to Entity B.

In a very particular exchange, it happens that all Attributes are Integers, for instance: [1, 9, 0]

With the optimization that Stefan proposes, Entity A can :

1.	either encode it as an heterogenous list, which has the short form part 0: List<Attribute> [ <int> 1, <int> 9, <int> 0] or a
2.	or encode it as an homogeneous list, which would have the short form part -11 : List<int> [1, 9, 0]

Why it’s an issue

In case 1), Entity B will indeed receive a List<Attribute> (short form part 0), whereas in case 2, Entity B will receive a List<Integer> (short form part -11), so the transport will have changed the MAL object, which is bad.

On the other hand, we said that Entity B knows which operation is being used, and therefore the expected data. So Entity B knows beforehand if the data is a List<int> or a List<Attributes>, so can do the type casting while decoding the message.

The decision

The point here is to decide between 2 options: can we assume that the data type is known before hand, and that the consumer uses this information to do the right type castings or not?

My conclusion

In my opinion, it should be authorized, otherwise we cannot do this kind of optimization.

The consequence is that the encoding of lists must have two decoding paths: 

*	an heterogeneous list which all elements have the same type can be encoded 

*	as an heterogeneous list (short form part = 0, and short form part of each element repeated)
*	or has an heterogeneous list (short form part = 0, and short form part of each element repeated): it’s an optimization and it’s legal

*	An homogeneous list can be encoded 

*	as an homogeneous list (short form part = -11 for integers)
*	or as an heterogeneous list (short form part = 0, and short form part of each element repeated): it’s stupid but legal

*	the decoders should be compliant of that.

Since MAL is ambiguous on this, the transport should be prescriptive: this should be described in the MAL SPP/Binary Binding.  For HTTP/XML, we should either say the same thing, or forbid it explicitly (since we don’t need optimization here, I would go for the simplest in terms of book changes).

I hope the point is clearer now. Let’s discuss it next Monday :)

Cheers

Olivier

De : MOIMS-SC <moims-sc-bounces at mailman.ccsds.org <mailto:moims-sc-bounces at mailman.ccsds.org> > De la part de Serge Lacourte via MOIMS-SC
Envoyé : vendredi 23 octobre 2026 11:17
À : moims-sc at mailman.ccsds.org <mailto:moims-sc at mailman.ccsds.org> 
Objet : [Moims-sc] Binary encoding of lists

EMAIL EXTERNE : Cet e-mail provient d'un expéditeur externe. Faites preuve de vigilance avant de cliquer sur un lien ou d'ouvrir une pièce jointe. Vérifiez que l'expediteur et le contenu du message sont légitimes.

Dear WG members,

we discussed several optimizations proposed by Stefan for the binary encoding. One of them concerns the encoding of lists. I understood the proposal as the possible encoding of a list holding elements of a single concrete type with a single encoding of the type instead of repeating the encoding of this type for all the elements of the list. This bothered me, and I try to discuss it more formally here.

I explain in the following my understanding of the MAL requirements. I would like to have your opinion on this.

There is no formal API of an encoding. An informal API of the encoding could be to convert a MAL data into an array of bytes(*1), and the reverse. I would go one step further, asserting that the decoding of the array of bytes which results from the encoding of a MAL data should be the exact copy of the original MAL data, including its exact MAL type.

With MAL v1 this was clear and obvious. However MALv2 introduces grey cases that have probably not been cleared enough, and I believe that some ambiguity may remain. The heterogeneous list is such a case.

The MAL specifies Abstract and Concrete types. An operation may declare an Abstract type in an operation parameter. At runtime, this parameter will be filled with a Concrete type that "derives" from the Abstract type. The "derivation" rules is clear for Composite types, it is less clear for List types. The MAL specification never tells the Abstract or Concrete type of a List. It does define List as a way to define a MAL type from the Abstract or Concrete type of the list elements. Il also defines the concepts of Homogeneous and Heterogeneous lists, both concepts being assigned a Short Form Part. It never formally defines what List types are Abstract or Concrete. My understanding is the following :

*	a List of Concrete element type is a Concrete type. Its Short Form Part is the negative value of the SFP of the element type.
*	an Homogeneous List is a global naming for all the lists defined from a Concrete element type
*	a List of Abstract element type is an Abstract type
*	an Heterogeneous List is a concrete type. It may hold elements of any MAL type. It is given 0 as SFP.
*	the statement 4.2.6.1.6 "If the type of the List is Abstract, then the List shall be a Heterogeneous List." is ambiguous. It could be interpreted as : If a service specification specifies a List type from an abstract element type as the parameter of a service operation, then, at runtime, an Heterogeneous List shall be used.

However this interpretation of statement 4.2.6.1.6 is clearly too restrictive. If concrete composite type C extends abstract composite type A, then List<C> can be used in place of a List<A> declared parameter. And list<C> is an Homegeneous List. There are then 2 options matching a List<A> declared parameter :

1.	Homogeneous List<C>, where C extends A
2.	Heterogeneous List holding only elements which extend A (runtime check)

In the case 1 above, the parameter will obviously be decoded as a List<C>. The question we have to answer now, which is not considered in the MAL specification, is : in the case 2 above, where an heterogeneous list is provided, should it always be decoded as an heterogeneous list, even if all the enclosed elements are of the same type C? My answer is yes, it should. This is why I reacted to the encoding proposal to allow encoding it as a List<C>. I agree that this optimization is still possible, as long as we agree that the parameter is eventually converted into an heterogeneous list. This is possible as the encoder knows the declared type of the operation.

Serge

(*1) with the split binary, this is not exactly true. It is true for the overall encoding of the MAL message, however the encoding usually refers to the encoding of each individual MAL data type in the MAL message body. The encoding of the MAL header is defined in the transport, with a specific encoding which may not follow the encoding of the body. So the split binary encoding transforms a MAL data into 2 separate arrays of bytes, one for the included booleans, and another for all other included data types.

-- 
Serge Lacourte
Directeur general
ScalAgent Distributed Technologies SA
tel. +33 4 76 29 79 81
mobile. +33 6 86 47 41 06
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/moims-sc/attachments/20261027/12079f34/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5623 bytes
Desc: not available
URL: <http://mailman.ccsds.org/pipermail/moims-sc/attachments/20261027/12079f34/attachment-0001.bin>