[Moims-sc] Use Cases of the ESA's MO stack
Serge Lacourte
serge.lacourte at scalagent.com
Mon Sep 7 11:33:56 EDT 2026
Hi everybody,
following our today's meeting, here is the current status of the
documentation. Any feedback is welcome.
Best regards,
--
Serge Lacourte
Directeur general
ScalAgent Distributed Technologies SA
tel. +33 4 76 29 79 81
mobile. +33 6 86 47 41 06
-------------- next part --------------
# Using the ESA MO stack
This documentation develops five use cases of the Java MO stack from ESA:
1. build an MO application consuming a standard MO service
2. build an MO application providing a standard MO service
3. extend a standard MO service
4. add a new MO service in the ESA's MO stack
5. create a new Transport in the ESA's MO stack
## Build an MO application consuming a standard MO service
### Description
The goal of this use case is to describe how an implementer can use the ESA MO stack to implement an application that consumes a standard MO Service (for providing a service, see UC 2). To illustrate how this is done in an actual use case, we assume that a remote agency deployed a standard Parameter Service from the Monitoring & Control (MC) Area, in order to enable other agencies to monitor the values of its mission's parameters. This documentation describes the implementation of a simple MO consumer application, interoperating with the remote agency's provider.
This is the most basic scenario using MO, which essentially requires Java programming abilities. It can be seen as a good introduction to the main MO concepts.
### Prerequisites
The target Parameter service is already deployed. We collect the out of band agreements related to this provider:
* the type of Transport used by the provider, eg MO standard HTTP in this UC,
* the URL of the provider, eg http://<host>:2026/ParameterProvider in this UC,
* the encoding used by the provider, eg MO standard XML in this UC,
* the ParameterDefinition objects managed by the provider, eg in this UC:
* identity:
* domain: "fr.cnes.mission.sat1"
* key: "ATT_BC_MTQ1VOLTAGE"
* version: 1
* description: "Satellite voltage"
* rawType: DOUBLE
### Overview
We use the Java MO stack from ESA to build the application, which is essentially an MO consumer. The application will connect to the Parameter service provider which is already deployed. The provider may not use the Java MO stack from ESA; interoperability is ensured by compliance with the HTTP transport and XML encoding standards.
The MO application reuses the following artifacts from the ESA's stack:
* `int.esa.ccsds.mo.transport-http`: ESA MO Transport - HTTP
* `int.esa.ccsds.mo.encoding-xml`: ESA MO Encoder - XML
* `int.esa.ccsds.mo.mal-impl`: ESA MO MAL Java Implementation
* `int.esa.ccsds.mo.api-area001-v003-mal`: ESA MO API - MAL v003, including the MAL types
* `int.esa.ccsds.mo.api-area004-v002-mc`: ESA MO API - MC v002, including the Parameter service types and the Parameter service consumer stubs
### Build the project
wait from input from DLR
### Program the application
The ESA's stack includes a code generator which builds a lot of glue code from the XML specification of the services. This includes the consumer's stub code implemented as the `ParameterStub` class in the `org.ccsds.moims.mo.mc.parameter.consumer` package, which can be found in the ESA's stack in the `int.esa.ccsds.mo.api-area004-v002-mc` artifact. This stub translates the MO service operations described in the specification into Java classes and methods, according to what is called a language mapping in the MO terminology.
The Java language mapping is no longer normalized in MO. It is then specific to the ESA's MO stack, and can be approached by mimicking the existing code base. The type mapping is, however, fairly easy to understand[^1]. As the use case aims at retrieving the value of a parameter, we can use the operation `getValue` of the Parameter service. It is naturally translated into the `getValue` method of the consumer stub, which arguments are straightforward:
```java
public ParameterValueList getValue(
IdentifierList domain,
IdentifierList keys)
throws MALInteractionException, MALException
```
In order to get the value of the `ATT_BC_MTQ1VOLTAGE` of the use case, we just have to fill in the proper values to the arguments:
```java
ParameterStub consumer;
IdentifierList domain = new IdentifierList(new ArrayList<> (Arrays.asList(
new Identifier("fr"),
new Identifier("cnes"),
new Identifier("mission"),
new Identifier("sat1"))));
IdentifierList keys = new IdentifierList(new ArrayList<> (Arrays.asList(
new Identifier("ATT_BC_MTQ1VOLTAGE"))));
ParameterValueList values = consumer.getValue(domain, keys);
ParameterValueData data = values.get(0).getValue();
Assert.assertEquals(ValidityState.VALID, data.getValidityState());
Double value = Attribute.attribute2double(data.getRawValue());
```
_Note: this part of the UC remains simple as we use a REQUEST operation. Using a PUBLISH-SUBSCRIBE operation, such as monitorValue, is more complicated. Should we add an example of using monitorValue, in this UC or in a variant?_
### Implement the MO stack side of the consumer API
In the previous section we have implemented the calling code of the consumer, which uses a `ParameterStub` object. We discuss now how this object is created. This section covers the initialization of the MO stack required to execute the consumer, which includes 2 steps:
* configure the transport and encoding layers
* create the consumer endpoint and connect to the provider
In order to interact with the MO stack, we need to specialize the abstract consumer class `ConsumerServiceImpl` from the package `org.ccsds.moims.mo.mal.helpertools.misc`, which may be found in the `int.esa.ccsds.mo.api-area001-v003-mal` artifact. This object is able to connect to the Parameter service provider, whose address is known from a `SingleConnectionDetails` object. Our specialized class `ParameterConsumerServiceImpl` requires the `SingleConnectionDetails` of the provider, and returns the `ParameterStub` object we need for our application with the method `getParameterStub`.
```java
public class ParameterConsumerServiceImpl extends ConsumerServiceImpl {
private ParameterStub parameterService = null;
/**
* Constructor.
*
* @param connectionDetails The connection URIs to the provider.
* @throws MALException If the service could not be started.
*/
public ParameterConsumerServiceImpl(final SingleConnectionDetails connectionDetails) throws MALException {
this(connectionDetails, null, null);
}
/**
* Constructor.
*
* @param connectionDetails The connection URIs to the provider.
* @param authenticationId The authenticationId token.
* @param localNamePrefix The local name prefix.
* @throws MALException If the service could not be started.
*/
public ParameterConsumerServiceImpl(
final SingleConnectionDetails connectionDetails,
final Blob authenticationId,
final String localNamePrefix)
throws MALException {
this.connectionDetails = connectionDetails;
// Close previous connection
if (tmConsumer != null) {
try {
tmConsumer.close();
} catch (MALException exc) {
Logger.getLogger(ParameterConsumerServiceImpl.class.getName()).log(
Level.SEVERE, "The previous connection could not be closed!", exc);
}
}
tmConsumer = connection.startService(
connectionDetails,
ParameterHelper.PARAMETER_SERVICE,
authenticationId,
localNamePrefix);
this.parameterService = new ParameterStub(tmConsumer);
}
@Override
public Object generateServiceStub(MALConsumer tmConsumer) {
return new ParameterStub(tmConsumer);
}
@Override
public Object getStub() {
return this.getParameterStub();
}
/**
* Returns the service stub.
*
* @return The service stub.
*/
public ParameterStub getParameterStub() {
return this.parameterService;
}
}
```
Our next step is to build the `SingleConnectionDetails` object required by the constructor. We know the URL of the provider from the out of band agreement. We can use default values for the other arguments.
```java
SingleConnectionDetails providerDetails = new SingleConnectionDetails(
new URI("http://<host>:2026/ParameterProvider"),
null,
ConfigurationProviderSingleton.getDomain());
```
The execution of the `startService` method automatically creates and initializes the ESA's MO stack layers. It starts with the name of the protocol found in the provider's URL, i.e. `http`, then everything is configured based on the values of properties which include that protocol in their name. The main configuration file is defined by the property `org.ccsds.moims.mo.mal.properties`, which defaults to `org/ccsds/moims/mo/mal.properties`. From this file the ESA's MO stack searches for the properties named `org.ccsds.moims.mo.mal.transport.protocol.http`, to get the actual implementation of the HTTP transport, and `org.ccsds.moims.mo.mal.encoding.protocol.http`, to get the actual implementation of the encoding.
```properties
org.ccsds.moims.mo.mal.transport.protocol.http=esa.mo.mal.transport.http.HTTPTransportFactoryImpl
org.ccsds.moims.mo.mal.encoding.protocol.http=esa.mo.mal.encoder.xml.XMLStreamFactory
```
_Note: All this must be confirmed. My personal experience with the ESA's MO stack is limited to the testbeds, with intra process binding directly using the `SingleConnectionDetails` object returned by the provider._
[^1]: The language mapping is easy to understand, but it would be great to have access to the Javadoc. Retrieving the `Double` value from the returned `ParameterValueList` object is not that trivial. Remember that in this use case we want to prevent the programmer from extracting the entire project of the ESA's MO stack.
### Run the application
## Build an MO application providing a standard MO service
### Description
The goal of this use case is to describe how an implementer can use the ESA MO stack to implement an application that provides a standard MO Service. It is the complement of the first use case, which was describing just the consuming side. Of course, a standard MO application may mix the two use cases, providing some services and consuming other services from possibly remote agencies.
To illustrate how this is done in an actual use case, we assume we want to expose the values of our mission's parameters to other agencies, through a standard Parameter Service from the Monitoring & Control (MC) Area. We assume that our Mission Control System is already deployed, and that the MO application only goal is to enable interoperability with other agencies.
This documentation describes the implementation and deployment of an MO provider application. A dedicated project is created for this provider, importing the ESA stack as any third party library.
### Prerequisites
None.
### Overview
We use the Java MO stack from ESA to build the application, which is essentially an MO provider. The application exposes a standard Parameter service compliant with the HTTP transport and XML encoding standards.
The MO application reuses the following artifacts from the ESA's stack:
* `int.esa.ccsds.mo.transport-http`: ESA MO Transport - HTTP
* `int.esa.ccsds.mo.encoding-xml`: ESA MO Encoder - XML
* `int.esa.ccsds.mo.mal-impl`: ESA MO MAL Java Implementation
* `int.esa.ccsds.mo.api-area001-v003-mal`: ESA MO API - MAL v003, including the MAL types
* `int.esa.ccsds.mo.api-area004-v002-mc`: ESA MO API - MC v002, including the Parameter service types and the Parameter service provider skeleton
### Build the project
wait from input from DLR
### Choose and describe the parameters to share
In this use case, we assume that the parameter values can be fetched from the already running Mission Control system. The first step of our work is to make the list of the parameters we want to expose from our system. This list is used to complete the API of our Parameter service provider, enabling future consumers to use it.
Most MO services standards specify the operations and data structures of the service, but leave some details to be specified on a case by case basis through out of band agreements. The Parameter service from the M&C area says that the actual list of Parameters managed by the provider and accessible for the consumers is to be defined in an out of band agreement. There is no standard operation to get this list dynamically from the provider.
The exposed parameters must be described as ParameterDefinition objects. In the most open way, they are provided as text explicit enough to build each field in the consumer language of choice. It could also be provided as a set of Java objects, which is the chosen implementation language of the provider, leaving the consumers to translate them in their own programming language of choice. The ParameterDefinition composite object is defined in section 4.3.2 of the specification, and it translates as the ParameterDefinition class of the org.ccsds.moims.mo.mc.structures package, which can be found in the ESA's stack in the `int.esa.ccsds.mo.api-area004-v002-mc` artifact. It includes the mandatory fields:
* identity: a unique identification of the Parameter, as a MAL::ObjectIdentity object
* domain: e.g. "fr.cnes.mission.sat1"
* key: e.g. "ATT_BC_MTQ1VOLTAGE"
* version: e.g. 1
* description: e.g. "Satellite voltage"
* rawType: e.g. DOUBLE
### Implement the service side of the provider API
The ESA's stack includes a code generator which builds a lot of glue code from the XML specification of the services. This includes the provider's skeleton code implemented in the `org.ccsds.moims.mo.mc.parameter.provider` package, which can be found in the ESA's stack in the `int.esa.ccsds.mo.api-area004-v002-mc` artifact.
There are an extended version and a simplified version of the provider skeleton[^1]. We use the simplified version, which is represented by the `ParameterInheritanceSkeleton` abstract class. This class implements most of the ESA's MO stack specific stuff, leaving the programmer with mainly the methods of the ParameterHandler interface to implement. There are 7 methods in this interface, matching the 7 operations of the Parameter service. As this interface defines the types of the methods arguments, it is easy to understand the mapping between the MAL types defined in the service specification and the Java types of the methods arguments. We just have to fill in the blank bodies of the methods.
```java
public class MyParameterProvider extends ParameterInheritanceSkeleton {
...
}
```
Let us take the operation getValue as an example. This operation translates in the interface as the method:
```java
/**
* Implements the operation getValue.
*
* @param domain The domain field.
* @param keys The keys field.
* @param interaction The MAL object representing the interaction in the provider.
* @return The return value of the operation
* @throws UnknownException Operation specific.
* @throws AmbiguousException The data or operation is ambiguous, requiring clarification to proceed.
* @throws MALInteractionException if there is a problem during the interaction as defined by the MAL specification.
* @throws MALException if there is an implementation exception
*/
ParameterValueList getValue(IdentifierList domain,
IdentifierList keys,
MALInteraction interaction)
throws UnknownException, AmbiguousException, MALInteractionException, MALException;
```
We must refer to the specification to correctly implement this operation. Several parameters may be targeted by the domain and keys arguments (requirements 1 and 2), the results must be ordered (requirement 4), the exception cases are identified (requirements 2 and 3). The implementation could look like:
```java
// prepare the argument for an UnknownException
UIntegerList unknowns = new UIntegerList();
// build the list of MO parameters references and
// retrieve the parameter identifiers in my M&C system
List<ObjectRef<ParameterDefinition>> moParams = new ArrayList<>();
List<MyMCSParamId> mcsParams = new ArrayList<>();
Iterator<Identifier> iterator = keys.iterator();
for (int i=0; iterator.hasNext(); i++) {
Identifier key = iterator.next();
// find the parameter definition from the out of band agreement
ObjectRef<ParameterDefinition> moParam = findMOParam(domain, key);
// find the parameter id in my M&C system
MyMCSParamId mcsParam = findMyMCSParamIdFromMOId(domain, key);
if (moParam == null || mcsParam == null) {
unknowns.add(new UInteger(i));
} else {
moParams.add(moParam);
mcsParams.add(mcsParam);
}
}
if (! unknowns.isEmpty())
throw new UnknownException(unknowns);
// retrieve the parameter values from my M&C system
ParameterValueList result = new ParameterValueList();
Iterator<ObjectRef<ParameterDefinition>> moParamsIt = moParams.iterator();
Iterator<MyMCSParamId> mcsParamsIt = mcsParams.iterator();
while (mcsParamsIt.hasNext()) {
double value = getParamValueFromMyMCS(mcsParamsIt.next());
// assume there is no converted value to compute
result.add(new ParameterValue(
moParamsIt.next(),
new Time(System.currentTimeMillis()),
null,
new ParameterValueData(
ValidityState.VALID_VALUE,
new Double(value),
null)));
}
return result;
```
[^1]: to be confirmed. The extended version leaves the programmer free to implement the ParameterHandler and ParameterSkeleton interfaces, instead of using the ParameterInheritanceSkeleton. I do not know in which case this would be useful, and maybe we should not talk about this option.
### Implement the MO stack side of the provider API
In the previous section we have implemented the service logic of the provider. This section covers the initialization of the MO stack required to execute the provider, which includes 4 steps:
* configure the transport and encoding layers
* create the provider endpoint
* create and initialize the publishers for the PUBLISH-SUBSCRIBE operations of the service
* expose the provider endpoint reference
There is actually no method to call to initialize the ESA's MO stack. It is statically initialized, and most of the components are defined from configuration files. The main configuration file is defined by the property `org.ccsds.moims.mo.mal.properties`, which defaults to `org/ccsds/moims/mo/mal.properties`. From this file can be found the property `org.ccsds.moims.mo.mal.transport.default.protocol` which configures the transport. This property defaults to `rmi`[^3]. As we want it to be HTTP, we must set the property value to `http`. The actual implementation of the HTTP transport is then specified with the property `org.ccsds.moims.mo.mal.transport.protocol.http`. Then further configuration is possible with other properties, such as the listening port with the property `org.ccsds.moims.mo.mal.transport.http.port`. Details may be found in the source code of the transport. The encoding is configured in the same way, with the property `org.ccsds.moims.mo.mal.encoding.protocol.http`.
```properties
org.ccsds.moims.mo.mal.transport.default.protocol=http
org.ccsds.moims.mo.mal.transport.protocol.http=esa.mo.mal.transport.http.HTTPTransportFactoryImpl
org.ccsds.moims.mo.mal.transport.http.port=2026
org.ccsds.moims.mo.mal.encoding.protocol.http=esa.mo.mal.encoder.xml.XMLStreamFactory
```
We define an `init` method in our provider to create and activate the provider endpoint. This is done with a `startService` call to a `ConnectionProvider`. The class is found in the `int.esa.ccsds.mo.api-area001-v003-mal` artifact, and the `PARAMETER_SERVICE` object as generated code in the `int.esa.ccsds.mo.api-area004-v002-mc` artifact. The statement initiates the deployment and initialization of the MAL, transport and encoding layers, as described above, then creates and starts a standard Parameter service provider. The last parameter of the `startService` call, `this`, makes our provider the target of the upcalls from the MAL, ensuring the execution of the provider API implementation we provided in the previous section:
```java
ConnectionProvider connection;
public void init() throws Exception {
// create the provider endpoint
this.connection = new ConnectionProvider();
MALProvider service = connection.startService(
"ParameterProvider",
ParameterHelper.PARAMETER_SERVICE,
true,
this);
...
}
```
The ESA's MO stack requires a specific initialization for the service operations which follow the PUBLISH-SUBSCRIBE pattern. We need to create and initialize a dedicated Publisher object for each such operation, that is the monitorValue operation for our Parameter service provider. The class of this object is specific to the operation, and can be found as generated code in the `int.esa.ccsds.mo.api-area004-v002-mc` artifact. Our `init` method includes then the following statements:
> public void init() throws Exception {
> ...
> // create the publishers for the PUBLISH-SUBSCRIBE operations
> MonitorValuePublisher publisher = super.createMonitorValuePublisher(null, null, null, null, null, null, null);
> publisher.registerWithDefaultKeys(null);
> }
The provider skeleton defines a function `getConnection` which should[^2] be overridden. It returns a `ConnectionProvider` object, specific to the ESA's MO stack, which holds the provider endpoint reference and can be used by a consumer to connect to the provider. The returned object is the one which was created in our `init` method. This structure refers to active objects of the MAL implementation, so it can only be used in the same process as the provider's one. In order to allow the connection from a consumer in a remote agency, which is our case, we need to extract a URI from this object (cf below).
> @Override
> public ConnectionProvider getConnection() {
> return this.connection;
> }
The final step is to build the main of the program, which is merely creating a `MyParameterProvider` object and call `init`. We symbolically add a code exposing the provider's URI, which must be shared with the service consumer.
> public void main(String args[]) throws Exception {
> MyParameterProvider myProvider = new MyParameterProvider();
> myProvider.init();
> // expose the provider's URIs
> SingleConnectionDetails details = myProvider.getConnection().getConnectionDetails();
> System.out.println("the URI of the provider is: " + details.getProviderURI());
> System.out.println("the URI of the provider's broker is: " + details.getBrokerURI());[^4]
> }
[^2]: This method is actually used by the testbed only. However it is generally necessary for the provider to expose its URI, except when it is well known. Exposing the `ConnectionProvider` object through the `getConnection` method, then extracting the provider's URI from this object, is the proper way to do it.
[^3]: the value is actually `rmi://`, cf class `TransportSingleton`. However it seems that it should be just `rmi`. To be confirmed.
[^4]: there is a question here about the broker's URI. I do not know the convention related to this URI in the ESA's MO stack. I assume that if it is not null, then it must be used by the consumer for all calls to the PUBLISH-SUBSCRIBE operations. I also assume that it is not null if the provider is created with the option of using a shared broker. I do not know what is its value if the shared broker option is not used. Is it null or is it the provider's URI itself? If it is null, then I assume that the provider's URI can be used for the calls to the PUBLISH-SUBSCRIBE operations, and so we could hide this question of a broker URI in this use case.
### Run the application
The ESA's MO stack is implemented in such a way that it allows the choice the actual implementations of the transport and encoding layers at deployment time, i.e. now. The names of the factory classes for those two layers are provided as properties in a configuration file. We have described in the previous section how to set the HTTP transport and the XML encoding, but it could be changed without a single change in the source code of the provider.
Set the content of the `mal.properties` file as described in the previous section, to set the HTTP transport and the XML encoding.
_Note: We could add more input about the question of binding between consumer and provider. In the previous section, I provided the minimum with the implementation of the getConnection method, and with a print of the provider's URI. In the next section I explicitly exposes the provider's URI, which is actually well known in our use case. We could discuss about the use of a name service, notably with the file based implementation included in the ConnectionProvider class of the ESA's MO stack. Then there is the option of using a shared broker, and the consequence on the binding. Should we discuss this in a separate use case, in order to keep this one as simple as we can?_
### Expose the out of band agreements
We need now to collect all the informations required by a consumer of the provider we have just deployed:
* the type of Transport used by the provider, i.e. MO standard HTTP,
* the URL of the provider, which is well known in our use case, i.e. http://<host>:2026/ParameterProvider,
* the encoding used by the provider, i.e. MO standard XML,
* the ParameterDefinition objects managed by the provider, as described in the previous section `Choose and describe the parameters to share`.
## Extend a standard MO service
This use case extends the previous use case by allowing the definition of extensions from the standard service specification.
## Add a new MO service in the ESA's MO stack
This use case describes how to add a new MO service in the ESA's MO stack. This stack is expected to host only standard services, with implementations that are used during the CCSDS prototyping process. The UC will then describe all the steps used to add the M&C services.
The framework actually handles all the services of an area in a single place. The UC actually describes adding all the services of the M&C area in the stack, not just a single service.
### Prerequisites
The XML description of the service is known.
### Overview
Developments are done in a private project, and changes will be pushed to the ESA's repository.
A new standard service area includes the following items:
* a file holding the XML definition of all the services of the area. The file is added to the xml-service-specifications module
* a directory holding the services APIs, to be added as a sub-module of the apis module. This includes specific code to program (in the src subdirectory), and generated code (in the target subdirectory).
* a directory holding the services implementation, to be added as a sub-module of the services-impl module.
* a directory holding the testbed for the added services, to be added as a sub-module of the testbeds module.
When all input files are provided, the whole project may be built from the top level directory, with the command:
```bash
mvn clean install
```
We detail below the successive steps of this command, explaining the various input and intermediate output produced.
### Configure the project repository
Connect to github and fork the project https://github.com/esa/mo-services-java.git. This creates a copy of the repository in your github space.
Clone your repository onto your workstation: git clone https://github.com/<myspace>/mo-services-java.git.
After you implement any changes, you will:
1. commit your changes in your local repository: git commit
2. push your changes to your github repository: git push
3. submit your changes to the ESA's repository by creating a Pull Request from your github repository (https://docs.github.com/fr/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork)
### Provide the service XML file
A lot of code is automatically created from the XML service definition using the code generator included in the repository.
The XML description of the service is known in the file area004-v002-Monitor-and-Control.xml. Put the XML file in the src/main/resources/xml directories of the two modules in the xml-service-specifications module, and build the module artifacts:
```bash
cd xml-service-specifications
cp area004-v002-Monitor-and-Control.xml xml-ccsds-mo-prototypes/src/main/resources/xml/
cp area004-v002-Monitor-and-Control.xml xml-ccsds-mo-standards/src/main/resources/xml/
mvn clean install
```
This creates the artifacts int.esa.ccsds.mo.xml-ccsds-mo-standards and int.esa.ccsds.mo.xml-ccsds-mo-prototypes as jar files, and install them in the local .m2 directory. These artifacts hold a copy of the XML file with the schema files, together with the XML files of all other areas, including the mal area.
_Note: I assume that the xml-ccsds-mo-prototypes module allows for extending the standard specification for the purpose of performing the CCSDS prototyping procedure. However I am not sure of it._
### Build the code generator
The code (and overall process) of the code generator is in the `api-generator` module. It is split in 4 sub-modules, and mainly results in building the `api-generator-maven-plugin` artifact.
```bash
cd api-generator
mvn clean install
```
The `api-generator-maven-plugin` artifact is a maven plugin which is used in `pom.xml` files to trigger the execution of the code generator and produce the service specific source code.
### Generate service specific code
The next module to build is apis. The goal of this module is twofold. In a first pass, the code generator is executed to produce service specific code from the service XML file. In a second pass, this code and other provider code is compiled to build a service specific API artifact.
The apis module defines a sub-module for each area. Create a new directory for the MC area, following the naming convention of the other modules, and populate it with a `pom.xml` file copied from another sub-module:
```bash
cd apis
mkdir api-area0004-v002-mc
cp api-area009-v001-mpd/pom.xml api-area0004-v002-mc/
```
Add `api-area004-v002-mc` to the list of modules of the apis `pom.xml`, then edit `api-area0004-v002-mc/pom.xml` to match the new area. As said before, the code generator execution is triggered by the `api-generator-maven-plugin` maven plugin. It is configured by a set of `ccsds.specification.download.*` properties, notably the filter property which identifies the file name of the XML definition to process:
```xml
<ccsds.specification.download.artifact>xml-ccsds-mo-standards</ccsds.specification.download.artifact>
<ccsds.specification.download.filter>**/area004-v002-Monitor-and-Control.xml</ccsds.specification.download.filter>
```
The file will be found from the artifact `xml-ccsds-mo-standards`, that was built in the first phase of the build process.
_Note: the actual referenced artifact in the pom files is `xml-ccsds-mo-prototypes` instead of `xml-ccsds-mo-standards`. I am not sure it is good practice, as it will build and export an artifact based on a possibly non standard version of the service. This could be an issue if the artifact is to be used in another context than service prototyping._
The code generator is executed during the generate-sources phase of the maven build.
```bash
mvn clean generate-sources
```
The generated code can be found in the `target/generated-sources/stub` directory, in the service specific package `org.ccsds.moims.mo.mc`. It includes:
* the area helper class `MCHelper`, holding a number of useful constants;
* exception classes mapping the area errors;
* classes mapping the area types in the structures sub-directory;
* service helper classes `<service>Helper` in `<service>` sub-directory, holding a number of useful constants;
* consumer stub classes in `<service>/consumer` sub-directory;
* provider skeleton classes in `<service>/provider` sub-directory.
Note also that the first sub-module of the apis module is the mal itself. A large part of the MAL implementation (types and interfaces) may be found in the src and target/generated-source of the api-area001-v003-mal sub-module.
### Define the provider backend APIs
The provider backend is not a MAL concept; it is specific to the ESA stack. The rationale for the backend concept is closely related to the testbed architecture in the ESA stack which targets the CCSDS prototyping procedure.
The usual architecture used in the prototyping of a MAL service is to provide two independent implementations of the service consumer and provider from two agencies, then to perform interoperability tests between the consumer from one agency and the provider from the other agency. However, in order to perform those tests, some shared assumptions and data may be required and shared on the provider implementations. This is ensured by a shared backend, used by both providers implementations.
An important aspect of the backend is shared data used in the tests. This is why backend implementations are called Datasets. However keep in mind that neither backends nor datasets are mandatory. This all depends on architectural choices for realizing the service prototyping. The backend/dataset architecture is used for the M&C services, and has been copied from the MPD testbed.
The backend may also provide tests specific functions, such as a time management API which is required by the tests of the M&C Parameter service. This function is defined in the BackendTimer API. An implementation specific to the testbed is provided in the testbed code, but a standard implementation based on the OS may be built and used in an operational deployment.
### Implement the backend
The backend API and base implementation, and other related classes, are the second main part of the apis module. They are found in the src subdirectory, while the generated code was in the target directory (cf section above). In more details, `apis/api-area0004-v002-mc/src/main/java/org/ccsds/moims/mo/mc` contains:
* backend APIs for all M&C services as interfaces (in the backend subdirectory);
* base datasets for all M&C services as abstract classes implementing the backend APIs;
* the `MCServicesFactory` class, which is the API of the entry point used by the testbed to instanciate consumers and providers implementations from the proper agency. Its content is simple, and can be created by adapting the same file from other directories. Each agency implementation must provide an implementation of this abstract class.
```bash
cd apis/api-area0004-v002-mc
mvn clean install
```
The resulting artifact (`api-<service>`) includes the backends APIs and base implementations, and the service specific code generated by the code generator from the service XML file.
### Implement the provider
As described above, the backend itself is shared code. This includes backend implementations (Datasets) which are described in the "Implement tests" section below. On the contrary, the provider implementation is specific to each agency participating in the service prototyping. The provider implementation must use the shared backend, at least when used in the prototyping process.
Let us state again that the provider implementation which can be found in the ESA stack is primarily designed and used in the service prototyping process as the provider implementation from ESA. It may also be used in a real deployment context, hopefully by defining a specific Dataset, but it could also require some bigger changes. Beyond this prototyping usage, it is expected to eventually be used as a reference implementation for checking that another provider implementation complies to the specification.
The provider implementation may be found in an area specific directory in the services-impl module. Create a new directory for the MC area, following the naming convention of the other modules, and populate it with a `pom.xml` file copied from another sub-module:
```bash
cd services-impl
mkdir services-area0004-v002-mc
cp services-area009-v001-mpd/pom.xml services-area0004-v002-mc/
```
Add `services-area004-v002-mc` to the list of modules of the `services-imp` pom.xml, then edit `services-area0004-v002-mc/pom.xml` to match the new area.
A part of the provider implementation already exists as skeletons automatically generated from the XML specification. The skeleton provides the glue between the provider's business logic and the MAL framework, reducing the implementation work to fill in predefined abstract functions and ignoring any MAL specific technology.
The skeleton framework of a service is produced by the code generator of the ESA stack as the class `<service>InheritanceSkeleton`. It can be found in `apis/<service>/target/generated-sources/stub/.../provider`, and must be retrieved from the `api-<service>` artifact to be declared as a dependency in the `pom.xml` file of the provider implementation.
_Note: The class `<service>InheritanceSkeleton` is the simplest skeleton pattern provided by the code generator. Another pattern exists which requires a better understanding of the ESA stack._
A service provider may be implemented by extending the `<service>InheritanceSkeleton` class, and implementing the abstract methods of the class and some other key methods.
The abstract methods all derive from the operations declared in the service specification. The provided abstract methods definitions respect the Java language mapping defined by the ESA stack. They may be implemented freely within the constraints defined by the requirements in the specification. Most of the out of band agreements listed in the specification should be covered by the backend API previously defined. This includes notably the list of Definition objects defined in the service specification.
Another key method must be overloaded to integrate to the ESA stack: getConnection. It returns a `ConnectionProvider` structure which can be used by a consumer to connect to the provider via the MAL. This structure is initialized by a MAL call to `startService`, which must be executed when the provider is started.
Another important method is the creation and initialization of the provider. This method, among other things, initializes the `ConnectionProvider` structure returned by `getConnection`. There is no standard interface to this method. However, in the testbed framework, the provider is created and initialized via the `MCServicesFactory` class which is described in section "Implement the backend" above. The implementation of `MCServicesFactory` must call this initialization function of the provider.
_Note: the M&C service providers also define a close method. However this method does not seem to ever be called._
### Implement the consumer
The consumer implementation may be found in the same sub-module as the provider. The source code architecture may be replicated from other services.
The consumer implementation is much more simpler than the provider one, as its code comes mostly from the `ConsumerServiceImpl` class and from stubs which are automatically generated from the XML specification. Just copy the `<Service>ConsumerServiceImpl` class from another service, then edit it to match the new service.
The client code directly uses the stub which can be retrieved from the `<Service>ConsumerServiceImpl` object.
### Implement the MCServicesFactory class
The final step of the service implementation is to provide a class implementing the `MCServicesFactory` API defined in section "Implement the backend" above. This class provides methods to instanciate and initialize consumers and providers from this implementation.
The name of this class will eventually be provided to the testbed as a property, so that the testbed can use the implementations from the proper agencies.
```bash
cd services-impl/services-area0004-v002-mc
mvn clean install
```
The resulting artifact (`services-<service>`) includes the implementations of the consumers and providers of all the services in the area, and the implementation of the MCServicesFactory API to be used by the testbed.
### Design and implement tests
The latest prototyping procedures have discarded the old FitNesse based architecture to a more simple one. The tests are specified as simple text in a spreadsheet document, and are implemented as JUnit tests.
All tests for the area are collected in a sub-module of the testbeds module. Create a new directory for the MC area, following the naming convention of the other modules, and populate it with a `pom.xml` file copied from another sub-module:
```bash
cd testbeds
mkdir testbed-mc
cp testbed-mpd/pom.xml testbed-mc/
```
Add testbed-mc to the list of modules of the testbeds `pom.xml`, then edit `testbed-mc/pom.xml` to match the new area.
The testbed framework uses JUnit. All tests are defined in the `src/test` directory, with naming conventions which respect the JUnit rules, i.e. class names ending with *Test*, and method names beginning with test with an `@Test` annotation.
The framework also includes code in the `src/main` directory. In a standard architecture, the `src/test` directory holds unit tests for code in the `src/main` directory. The testbed architecture is somehow different, as the `src/test` directory holds unit tests for another module. The code in the `src/main` directory is then just another part of the test code. The architecture of the testbed code is as follows:
* `src/main/java/<service>/testbed`: code used by the provider
* `Constants.java`: shared constants used by the provider and the consumer
* `backends/*`: definition of the Datasets used in the tests
* `SetUpProvidersAndConsumers.java`: entry point for deploying the proper implementations of consumers and providers
* `src/test/java/<service>/testbed`: code used only by the consumer part of the test
* tests scenarios and instances
The architecture of the tests is actually free. There is no real constraint, and the architecture of the existing testbeds can be used as a starting point only.
`SetUpProvidersAndConsumers` is a key class. It deploys the proper implementations of the consumers and providers, and it links consumers to their matching providers trough the MAL.
Choosing the proper implementations goes through environment variables. `SetUpProvidersAndConsumers` looks up for the values of "testbed.provider" and "testbed.consumer", to get the names of the Factory classes used to build the providers and consumers respectively. The factories are those built in the "Implement the MCServicesFactory class" section above.
Linking consumers to their matching providers makes use of the `getConnection` method described in the "Implement the provider" section above. A provider is created by the provider factory, then its connection details are retrieved through the `getConnection` method. A consumer may then be created by the consumer factory, and it can be connected to that provider by providing the provider connection details as parameter of the consumer constructor.
```java
// create the proper provider factory
String factoryClassForProvider = System.getProperty("testbed.provider");
Class factoryClassProvider = Class.forName(factoryClassForProvider);
MCServicesFactory factoryProvider = (MCServicesFactory) factoryClassProvider.newInstance();
// create and initialize the service with the test backend
ActionInheritanceSkeleton actionProviderService = factoryProvider.createProviderAction(actionBackend);
// retrieve the provider connection details
SingleConnectionDetails details = actionProviderService.getConnection().getConnectionDetails();
// create the proper consumer factory
String factoryClassForConsumer = System.getProperty("testbed.consumer");
Class factoryClassConsumer = Class.forName(factoryClassForConsumer);
MCServicesFactory factoryConsumer = (MCServicesFactory) factoryClassConsumer.newInstance();
// create and initialize the consumer and connect it to the provider
ActionStub actionConsumerStub = factoryConsumer.createConsumerStubAction(details);
```
### Run tests
Tests must be configured before being executed. This includes the choice of the MAL transport and encoding, and the choice of consumers and producers implementations.
The environment variables used to retrieve the consumers and producers factories are defined in the pom.xml file in the profiles section. In the case of the M&C testbed, where the two prototyping agencies are ESA and CNES, four profiles are defined. The profile is chosen by the -P option of the maven command:
* ESA: ESA consumer to ESA provider
* CNES: CNES consumer to CNES provider
* ESA-CNES: ESA consumer to CNES provider
* CNES-ESA: CNES consumer to ESA provider
How to choose the MAL transport and encoding is not confirmed. There are a lot of instructions loading properties file in the MAL code, automatically executed from static initializers. It is hard to tell exactly which files are actually loaded or not. It is likely that the consumer.properties, provider.properties, and transport.properties files existing in the root directory of the service testbed are loaded. In those files, for the M&C testbed, we can see the following properties defined:
* `org.ccsds.moims.mo.mal.factory.class`
* defines the MAL implementation, points to the ESA stack
* `org.ccsds.moims.mo.mal.transport.default.protocol`
* defines the transport, value is `rmi://`
* `org.ccsds.moims.mo.mal.transport.protocol.rmi`
* defines the transport implementation, points to the RMI transport in ESA stack
* `org.ccsds.moims.mo.mal.encoding.protocol.rmi`
* defines the encoding implementation, points to the FixedBinary encoding in ESA stack
Tests are executed with the test target in the `testbed-<service>` sub-module. It is possible to execute all tests in one command, or a single test with the proper JUnit properties.
```bash
cd testbeds/testbed-mc
mvn -P ESA-CNES clean test
mvn -P ESA-CNES test -Dtest=AC_1_Basic_Execution_Test#testCase_01
```
## Create a new Transport in the ESA's MO stack
This use case describes how to add a new transport in the ESA's MO stack.
More information about the MOIMS-SC
mailing list