[Css-csts] [CSTS] Notification of Eventevents

[Martin Götzelmann]

Members of the CSTSWG,

Due to the discussions on the last teleconference and some issues raised by John via e-mail, I have tried to collect all issues associated with notification of events and to assess options for solving them. I have not been able to come to a final conclusion or recommendation, but I nevertheless provide the analysis as input to the teleconference scheduled for Wednesday. Any comments would be very much appreciated.

In the following, I refer to SLE services rather than to the more abstract procedures as the SLE services are probably better suited for the analysis of real world problems. The arguments for return services should apply to the Buffered Data Delivery procedure and those related to forward services should apply to the Data Processing procedure respectively.


1. Issues related to notifications:
-----------------------------------
As far as I remember, we decided to define a generic notification procedure in Heppenheim primarily because we felt the monitoring service should support cyclic monitoring, polling, and event monitoring and each of these aspects was a good candidate for an individual generic procedure that would be useful elsewhere as well. I cannot recall any more specific reasons and the minutes do not identify any either.

In the meantime, further issues related to use of notifications within (SLE) services have come up:

[ISSUE 1]
Notifications may be issued as soon as the service instance is bound, i.e. before start of the procedure using the notification. For instance the operational scenarios described for CLTU/FSP assume the user will bind and then wait for the provider to send the notification 'production operational'. Only when this notification has been received, the user will invoke the START operation. As we constrain invocation of operations that belong to a procedure (other than START) to the active state, this will obviously no longer work - the user must invoke START before he can receive notifications.

The approach described in the "operational scenario" is not mandatory - the user could invoke START and then wait for then wait for the notification 'production operational'. This will work if the production status is configured but according to the current specifications START will be rejected if the production status is interrupted or halted. The user may of course start the cyclic report procedure and monitor the production status there ... but that was probably not the original intention.

[Issue 2]
Notifications might be issued after the STOP operation is completed. This does not only refer to notifications related to the production status - CLTU and FSP allow notifications related to processing progress to be generated after the STOP operation completed. As discussed previously, such notifications could be suppressed but one needs to consider the fact that the only way to abort processing of already transmitted commands - for whatever reason - is to invoke STOP and this case the user might want to know what command was the last one transmitted. If transmission is aborted due to a fault on the provider side or expiry of a command, the last command that successfully terminated processing is notified to the user but that is not the case if the user stops processing. Again, one could argue that the user could eventually obtain this information from the status report, but that was probably not the original intention. If one wants to constrain notifications to the period in which the main procedure is active, and nevertheless provide the capability to abort processing of commands and receive a notification of the last processing steps, a new means to abort processing might be necessary (either a new operation "reset" or by means of EXECUTE-DIRECTIVE).

In addition to these issues, John has pointed out the following that originate from the introduction of the notification procedure:

[Issue 3]
As currently defined the basic NOTIFY operation specifies notifications of the production state transitions. The notification procedure extends this by adding a generic "notifiable event" the definition of which is left to the service using the procedure. The user may select the notifiable events that he wants to be reported. John has raised the question whether the production status events should also be selectable or should always be reported.

[Issue 4]
If a service uses two procedure instances that make use of notifications (either one instance of the notification procedure and one instance of buffered data delivery or data processing or two instances of the notification procedure) the user may receive more than one notification of the same event. Duplicate notifications might be quite difficult to handle especially if an (automated) action is required in response.


2. Discussion of potential options
----------------------------------

[OPTION 1]
Remove NOTIFY from the prime procedure (derived from Buffered Data Delivery or Data Processing) and use a (derived) notification procedure to notify all events.

On a first sight, this option seems to solve issues 1 and 2 as it can be started immediately after BIND and can remain in the ACTIVE state when the prime procedure is stopped. The approach would imply that events occurring within the prime procedure itself would be notified by the notification procedure and that all information transmitted with the notification would have to be passed from the prime procedure to the notification procedure. The responsibility of the notification procedure would actually be limited to sending a notification actually generated by the prime procedure.

A further, maybe more serious problem I see is that the rules of when the notifications shall be issued, how they shall be transmitted, and what their effect should be are very tightly coupled with service production and with the provision of the prime procedure. Examples of issues that would need to be addressed and solved include the following:

Buffered Data Delivery / SLE return services:
- All notifications are synchronous; if this shall be maintained, tight synchronisation between the prime procedure and notification procedure would be required;
- All notifications also need to be recorded such that they can be correctly inserted into the data stream transmitted in complete mode;

Data Processing / SLE forward services
- The user selects notifications to be issued in individual TRANSFER-DATA invocation and not in a global START operation
- The notification "production interrupted" may be deferred, i.e. should only be issued if the user is actually affected.
- Certain events (production status transition from interrupted to operational) should only be notified if other events (production interrupted) had previously been notified.

Obviously both SLE return and forward services depend on the notifications to be enabled and the service specification would therefore have to make selection of these notifications mandatory. I do not consider that a major drawback, however. 

A very tight synchronisation of the prime procedure (Buffered Data Delivery or Data Processing) with the notification procedure might also impede the specification and development of reusable software components because the developer of the notification procedure SW might have to make too many assumptions about the prime procedure with which it needs to cooperate. 


[OPTION 2]
Notify "service specific" events related to reception, delivery, and processing of data as part of the prime procedure and notify production status changes using the notification procedure. 

This option would solve ISSUE 1 but not ISSUE 2 as the events causing difficulties after STOP are service specific. It would avoid some (most?) of the problems associated with OPTION 1. However, I think some of the issues remain, for instance:
a) SLE return services require that all notifications including those related to the production status are sent synchronously and are recorded for later playback in complete mode;
b) SLE forward services require support for deferred notification of the status 'interrupted' and conditional notification of the status 'operational'.
c) The effects that production status changes have on the behaviour of forward services require close synchronisations of service state changes and transmission of the notification.


[OPTION 3] 
For procedures for which notification of events is an essential part of the protocol always notify all relevant events as part of the procedure itself. Use the notification procedure only for services where notification of events is nice to have and therefore optional (or for notification of additional events "of interest").

This option does not solve ISSUE 1 and ISSUE 2 for which alternative solutions would need to be identified. However, it would avoid the interdependencies between the prime procedure (derived from Buffered Data Delivery or Data Processing) and the notification procedure. Production status events would have to be defined already for the basic operation in a similar fashion as this is done currently (see however comment 4.1 below). Duplicate notification of the same event could be avoided simply by not including the notification procedure into the service.

Options to solve (or maybe mitigate) ISSUE 1 and ISSUE 2 in this case might be:
- for ISSUE 1 allow START to always succeed independent of the production status (implications would require further detailed study);
- for ISSUE 2 provide means to reset the service without stopping the prime procedure.


3. Some preliminary conclusions
-------------------------------
For the first time I have the feeling that we hit limits of the specification approach selected for the 'toolkit'. Specification of individual procedures from which various different services can be composed seems to work well only when the procedures are reasonably independent from each other. That is the case for instance for the Association Control, Data Processing, Information Query, and Cyclic Report from which an SLE forward service could be built (after derivation). Although the Information Query and the Cyclic Report procedures report the values of parameters related to and manipulated by the Data Processing procedure the execution of these procedures is more or less independent. This is obviously different if one moves all or parts of the notifications from the Data Processing (or the Buffered Data Delivery) procedure to a notification procedure.

Because we are hitting a limitation of the approach, we will probably only be able to solve the issues with some sort of more or less unpleasant compromise. I see the following basic options:

(A) Accept the complications in the specification (and possibly the implementation) of procedures that have a high need of synchronisation. Issues that need to be addressed in this case include for instance:
- What is the best place to specify the synchronisation (e.g. in the Buffered Data Deliver / Data Processing Procedure, in the service using the procedure?)
- The notification procedure would have to provide an interface to other procedures enabling them to .. request transmission of an invocation and to supply the data to be transmitted with it; .. be informed on success or failure of such transmission; .. be informed on the reception of a notification (on the user side) Obviously, such interfaces between procedures would add a new specification element that would further complicate the books.

(B) Accept that notifications must be integrated with the prime procedure in some cases and accept the duplication of notifications by the notification procedure. In this the remaining issues with notifications before and after START / STOP must be solved by other means.

Intermediate solutions between these basic options might well be possible.


4. A few additional comments
----------------------------
What follows are a few additional comments/questions concerning aspects not addressed by the discussion so far (partially these are actually comments on the notification procedure).

4.1 Why handle production standard events differently?

Currently the NOTIFY operation specifies the production status related events in a manner that allows derived operations to specify additional information to be transmitted with the notification. It further allows addition of new notification type. The notification procedure defines a single additional notification type "notifiable event", which itself consists of an event name and an event qualifier. I find the difference in the specification approach disturbing as I do not really see a justification. I do not have a strong preference, but I feel all notifications should be handled the same way. This implies that the user of the notification procedure - in my opinion - should be able to select the production status changes as well unless this is specified differently by the service using the procedure.


4.2 What production status do notifications refer to?

On second thoughts I feel that it might not always be obvious what production status the events actually refer to. For a service such as an SLE service, the production status obviously refers to the production of the SLE service in question. But what does it refer to for the monitoring procedure? The production of the monitoring service? Would a user not be rather interested in the production of other services being used concurrently such as SLE forward, SLE return, radiometric measurement, etc? Is there a need to qualify what production is meant?


4.3 Use of NEDB and NELNDB

The notification procedure uses the model of a virtual Notifiable Events Data Base (NEDB) and a virtual Notifiable Events List Name Data Base (NELNDB) to specify how events may be specified by a service using the procedure (or by derived procedures) and how they can be selected by a user. The Cyclic Report procedure and the GET operation specify similar things just addressing parameters instead of events but do not make use of similar models. I feel the specification approach should be unified to either use the concept of virtual data bases or not to use them (I personally do not see a compulsory reason to use such a model.)

Kind Regards,
Martin