[Moims-mp] Test Case specification V19

Guillermo Buenadicha Guillermo.Buenadicha at esa.int
Mon Feb 10 16:46:56 UTC 2025 

Dear Marvin:

Thanks as usual. Please, see my comments to your comments, with color codes (green accepted and implemented, red discrepancy, orange for discussion, if you wish on Monday or through email before, as your wish).

Regards!




PDS
T_PDS_007_PQ7: Expect to return P6 (correct in notes).
PES
T_PES_005_3: Expect error code to be 1.  (disagree. Error code is 1 (INVALID) and the extra info is the field number (1) and the reason (0, unkown)
T_PES_005_4: Expect error code to be 2. (disagree. Error code is 1 (INVALID) and the extra info is the field number (2) and the reason (0, unkown)

T_PES_006_1: Currently throws invalid error because IED1 does not exist on P5 and input validation is performed before plan status checks. • Event instance EI1 added to P5 as well. This also changes T_PDS_007_PQ11, now P1 and P5 returned.
T_PES_006_3: Expect error code to be 1.  (disagree. Error code is 1 (INVALID) and the extra info is the field number (1) and the reason (0, unkown)
T_PES_006_4: Expect error code to be 2. (disagree. Error code is 1 (INVALID) and the extra info is the field number (2) and the reason (0, unkown)
T_PES_008_2: Currently throws an invalid error because AU1 does not exist on P5 and input validation is performed before plan status checks. • Disagree. The operation fails with UPDATE_FAILED since the P5 is terminated. AI1 exists in P5, so an activity update should not be rejected as invalid or unkown.  Added information on the failure error codes.
T_PES_009_2: Currently throws an invalid error because EU1 does not exist on P5 and input validation is performed before plan status checks. • Since EU1 is now added to P5, it works.  Added information on the failure error codes.
T_PES_011_2: Currently throws an invalid error because RU1 does not exist on P5 and input validation is performed before plan status checks. • RI1 added to P5, so it works. I believe this does not have impact on any other test (i.e. Plan Filters)
T_PES_011_3: I’d expect additionally error path 2.1 since the resource is unknown as well. • Added, it is 2,0. I use the comma to separate the Extra info index and secondary code. The dot is used in the index to refer to complex structures. Question, P0 now exists as a plan, though with erroneous elements as validity fields. Shall we flag it as invalid due to its own errors?
T_PES_012_2: Currently throws an invalid error because RU1 does not exist on P5 and input validation is performed before plan status checks. • Should work now, RI1 is added to P5.
T_PES_012_3: I’d expect additionally error path 2.1 since the resource profile is unknown as well. • Added, it is 2,0. Question, P0 now exists as a plan, though with erroneous elements as validity fields. Shall we flag it as invalid due to its own errors?
PECS
T_PECS_001_P0:
MW: I’d expect the error path to be “1.10” with error code “UNDEFINED” (?) and path “1.5.6” with “BAD_TIME” because the validity end is before the start. Should only the end date be invalid, or also the start?
GB: I have updated to 1.9,4 and 1.10,4.
MW: Maybe I got something wrong here. Could you explain why you would expect 1.9 and 1.10 with BAD_TIME as an error code? Field nine of a plan is “isAlternate” and field ten is “status”. I’d expect field 1.5.6 to be BAD_TIME or INCONSISTENT since the end time of the plan information is before the start time. Additionally, the status is invalid since TERMINATED plans may not be accepted by the execution system according to the plan-state diagram. I’m not quite sure which secondary error fits bets here. Maybe UNDEFINED or OUT_OF_RANGE? Otherwise one could also say that the input validation passes and the new SUBMIT_FAILED error is thrown later. Should be the same behavior as P2, however a validation error should be thrown in any case due to the invalid time.
• This is an interesting point. Indeed, the validityStart and validityEnd are the 5th and 6th fields of the Plan information, what is the 5th field of the Plan. I guess the error is on validityEnd, that is the 6th element, being the one not being after validityStart.
T_PECS_003_1:
MW: My understanding is that P2 is rejected initially when submitted because it is TERMINATED and should therefore be unknown to PECS. In that case, P2 should be removed from input and expected output.
GB: Not fully sure. This is getPlanStatus. A terminated plan, even if rejected initially on the submission, may be kept in the provider DB showing it was a terminated plan. It is an interesting question for the group.
MW: As I understand it, the plan was never in the dataset of the execution system since it was rejected due to an invalid status. The getPlanStatus operation of PECS should only return the status of known plans if I understand it correctly.
• The P2 is changed to Terminated in T_PDS_005, so it is assumed that it is a plan available to the system. I have no problem to follow your logic, but I cannot see within the PECS test where the plan is rejected. And since plans are distributed with PDS, this sets the test criteria. We agree to validate the different service areas uncorrelated, so we should not make assumptions either way,

T_PECS_003_P0: Expect the error path to be 1.1 since it is the first element in the list.
• My former question applies: Should we use the error code 1 with extra info 1.5.6, 4 in any instance of P0?
T_PECS_004_P0: Expect the error path to be 1.1 since it is the first element in the list.
• My former question applies: Should we use the error code 1 with extra info 1.5.6, 4 in any instance of P0?

T_PECS_005_3: Expect the error path to be 2.1 for the first element of the second parameter.
• My former question applies: Should we use the error code 1 with extra info 2.5.6, 4 in any instance of P0?
T_PECS_008_P0: Expect the error path to be 1.1 since it is the first element in the list.
• My former question applies: Should we use the error code 1 with extra info 1.5.6, 4 in any instance of P0?

T_PECS_011_2:
MW: If I understand the BB correctly, then all activities were unloaded in T_PECS_008 since an activity was already executed in T_PECS_006. This results in no monitor notification since the plan has no activity instances.
GB: I do not agree. I believe a subplan is a collection of Activity Instances, and they may belong to different plans. The fact that one is terminated does not mean that a subplan cannot be called and activated. Still a valid question for the WG. There is no connection of a subPlan to a Plan, nor a way to submit a subPlan.
MW: The subplan activation is not the problem but rather the expected monitor notification. My understanding is that the plans, which are under monitoring, do not contain any activity instance anymore due to T_PECS_006 and therefore do not trigger a notification.
• I believe you mean T_PECS_008. Still, there only P1 is deactivated. However, the subPlan1 contains AI7 that belongs to P6, that at the time is released. So my assumption is that the subPlan1 is activated with this operation.  Please, see page 3-44 section 3.7.5.13:

NOTE      –     The status of SubPlans is independent to that of Plans and a change in their activation status has no impact on the status of the Plan.  It may impact the status of individual ActivityInstances, which can be observed using the monitorPlanExecutionDetail operation.


T_PECS_011_4: Expect the error path to be 1.1 since it is the first element in the list.
Not sure what is 011_4, but if this for the subPlan0, this indeed does not exist. So it is correctly 1 and Extra info 1,0. Unless you assume that a subplan id is the first element of its list, then would be 1 with ExtraInfo 1.1, 0. Is a subplan a plan structure? If so, then it is the later. Even 1.1.2, 0 (see later) if we assume the second element of identity to be the affected one.
T_PECS_012_2:
MW: If I understand the BB correctly, then all activities were unloaded in T_PECS_008 since an activity was already executed in T_PECS_006. This results in no monitor notification since the plan has no activity instances.
GB: I do not agree. I believe a subplan is a collection of Activity Instances, and they may belong to different plans. The fact that one is terminated does not mean that a subplan cannot be called and activated. Still a valid question for the WG. There is no connection of a subPlan to a Plan, nor a way to submit a subPlan.
MW: The subplan activation is not the problem but rather the expected monitor notification. My understanding is that the plans, which are under monitoring, do not contain any activity instance anymore due to T_PECS_006 and therefore do not trigger a notification.
• See my comments above. Let’s discuss Monday 17th.
T_PECS_012_subPlan0: Expect the error path to be 1.1 since it is the first element in the list.
See comment before, agree if considered a plan as composite structure. In this case the passed element should be the identify (domain, key, version) and the failure shall be on key, so should be 1.1.2
T_PECS_013_subPlan0: Expect the error path to be 1.1 since it is the first element in the list.

See comment before, agree if considered a plan as composite structure. In this case the passed element should be the identify (domain, key, version) and the failure shall be on key, so should be 1.1.2

T_PECS_016_P0:
MW: Expect error codes (1.5.6, BAD_TIME) and (“1.10”, UNDEFINED) since status is TERMINATED
GB: See my question in T_PECS_008. If P0 passed on a service is an UNKNOWN we should be consistent. Which plan is Terminated? I apply same logic as there, and use UNKNOWN. However, if we assume that the execution engine knows about P0, then indeed we need to enter into inner error codes.
MW: The status of P0 is TERMINATED. This is indeed the same as P_PECS_001_2. My understanding is that the execution system only accepts plans that are RELEASED. I take this from the plan-state ULM diagram. Not sure which Secondary error fits best here.
• Let’s address this on the 17ht teleconf.


From: marvin.wittschen at dlr.de <marvin.wittschen at dlr.de>
Date: Monday, 3 February 2025 at 18:29
To: Guillermo Buenadicha <Guillermo.Buenadicha at esa.int>, Dominik Marszk <Dominik.Marszk at esa.int>, Peter Van Der Plas <Peter.van.der.Plas at esa.int>, Cesar Coelho <Cesar.Coelho at ext.esa.int>, Christoph.Lenzen at dlr.de <Christoph.Lenzen at dlr.de>, Maria.Woerle at dlr.de <Maria.Woerle at dlr.de>, luke.berry at gmv.com <luke.berry at gmv.com>, Rachel Jenkins <rjenkins at gmv.com>, eric.w.ferguson at jpl.nasa.gov <eric.w.ferguson at jpl.nasa.gov>, geoff.lochmaier at nasa.gov <geoff.lochmaier at nasa.gov>, David Frew <David.Frew at esa.int>, olly.page at cgi.com <olly.page at cgi.com>, clement.hubin-andrieu at cnes.fr <clement.hubin-andrieu at cnes.fr>
Cc: moims-mp at mailman.ccsds.org <moims-mp at mailman.ccsds.org>
Subject: AW: Test Case specification V19
Hello Guillermo,

below the notes on what I found when implementing the changes in the testbed and the prototype. In addition to the open topics, the points are mainly two different types:

  *   Error paths of list input fields. Our prototype previously didn’t implement this correctly. I noticed it because you corrected it for some tests with the last version.
  *   Input validation order. In some tests, you try to raise an operation error (e.g. update failed) with an invalid status of the plan. But because the input is invalid as well, an invalid error is thrown before the operation error is reached. I didn’t notice this earlier because I accidentally threw an operation error when the ref was invalid.

Best regards,
Marvin
PDS

  *   T_PDS_007_PQ7: Expect to return P6 (correct in notes).
PES

  *   T_PES_005_3: Expect error code to be 1.
  *   T_PES_005_4: Expect error code to be 2.
  *   T_PES_006_1: Currently throws invalid error because IED1 does not exist on P5 and input validation is performed before plan status checks.
  *   T_PES_006_3: Expect error code to be 1.
  *   T_PES_006_4: Expect error code to be 2.
  *   T_PES_008_2: Currently throws an invalid error because AU1 does not exist on P5 and input validation is performed before plan status checks.
  *   T_PES_009_2: Currently throws an invalid error because EU1 does not exist on P5 and input validation is performed before plan status checks.
  *   T_PES_011_2: Currently throws an invalid error because RU1 does not exist on P5 and input validation is performed before plan status checks.
  *   T_PES_011_3: I’d expect additionally error path 2.1 since the resource is unknown as well.
  *   T_PES_012_2: Currently throws an invalid error because RU1 does not exist on P5 and input validation is performed before plan status checks.
  *   T_PES_012_3: I’d expect additionally error path 2.1 since the resource profile is unknown as well.
PECS

  *   T_PECS_001_P0:

     *   MW: I’d expect the error path to be “1.10” with error code “UNDEFINED” (?) and path “1.5.6” with “BAD_TIME” because the validity end is before the start. Should only the end date be invalid, or also the start?
     *   GB: I have updated to 1.9,4 and 1.10,4.
     *   MW: Maybe I got something wrong here. Could you explain why you would expect 1.9 and 1.10 with BAD_TIME as an error code? Field nine of a plan is “isAlternate” and field ten is “status”. I’d expect field 1.5.6 to be BAD_TIME or INCONSISTENT since the end time of the plan information is before the start time. Additionally, the status is invalid since TERMINATED plans may not be accepted by the execution system according to the plan-state diagram. I’m not quite sure which secondary error fits bets here. Maybe UNDEFINED or OUT_OF_RANGE? Otherwise one could also say that the input validation passes and the new SUBMIT_FAILED error is thrown later. Should be the same behavior as P2, however a validation error should be thrown in any case due to the invalid time.

  *   T_PECS_003_1:

     *   MW: My understanding is that P2 is rejected initially when submitted because it is TERMINATED and should therefore be unknown to PECS. In that case, P2 should be removed from input and expected output.
     *   GB: Not fully sure. This is getPlanStatus. A terminated plan, even if rejected initially on the submission, may be kept in the provider DB showing it was a terminated plan. It is an interesting question for the group.
     *   MW: As I understand it, the plan was never in the dataset of the execution system since it was rejected due to an invalid status. The getPlanStatus operation of PECS should only return the status of known plans if I understand it correctly.

  *   T_PECS_003_P0: Expect the error path to be 1.1 since it is the first element in the list.
  *   T_PECS_004_P0: Expect the error path to be 1.1 since it is the first element in the list.
  *   T_PECS_005_3: Expect the error path to be 2.1 for the first element of the second parameter.
  *   T_PECS_008_P0: Expect the error path to be 1.1 since it is the first element in the list.
  *   T_PECS_011_2:

     *   MW: If I understand the BB correctly, then all activities were unloaded in T_PECS_008 since an activity was already executed in T_PECS_006. This results in no monitor notification since the plan has no activity instances.
     *   GB: I do not agree. I believe a subplan is a collection of Activity Instances, and they may belong to different plans. The fact that one is terminated does not mean that a subplan cannot be called and activated. Still a valid question for the WG. There is no connection of a subPlan to a Plan, nor a way to submit a subPlan.
     *   MW: The subplan activation is not the problem but rather the expected monitor notification. My understanding is that the plans, which are under monitoring, do not contain any activity instance anymore due to T_PECS_006 and therefore do not trigger a notification.

  *   T_PECS_011_4: Expect the error path to be 1.1 since it is the first element in the list.
  *   T_PECS_012_2:

     *   MW: If I understand the BB correctly, then all activities were unloaded in T_PECS_008 since an activity was already executed in T_PECS_006. This results in no monitor notification since the plan has no activity instances.
     *   GB: I do not agree. I believe a subplan is a collection of Activity Instances, and they may belong to different plans. The fact that one is terminated does not mean that a subplan cannot be called and activated. Still a valid question for the WG. There is no connection of a subPlan to a Plan, nor a way to submit a subPlan.
     *   MW: The subplan activation is not the problem but rather the expected monitor notification. My understanding is that the plans, which are under monitoring, do not contain any activity instance anymore due to T_PECS_006 and therefore do not trigger a notification.

  *   T_PECS_012_subPlan0: Expect the error path to be 1.1 since it is the first element in the list.
  *   T_PECS_013_subPlan0: Expect the error path to be 1.1 since it is the first element in the list.
  *   T_PECS_016_P0:

     *   MW: Expect error codes (1.5.6, BAD_TIME) and (“1.10”, UNDEFINED) since status is TERMINATED
     *   GB: See my question in T_PECS_008. If P0 passed on a service is an UNKNOWN we should be consistent. Which plan is Terminated? I apply same logic as there, and use UNKNOWN. However, if we assume that the execution engine knows about P0, then indeed we need to enter into inner error codes.
     *   MW: The status of P0 is TERMINATED. This is indeed the same as P_PECS_001_2. My understanding is that the execution system only accepts plans that are RELEASED. I take this from the plan-state ULM diagram. Not sure which Secondary error fits best here.






Von: Guillermo Buenadicha <Guillermo.Buenadicha at esa.int>
Gesendet: Montag, 13. Januar 2025 21:51
An: Wittschen, Marvin <marvin.wittschen at dlr.de>; Dominik Marszk <Dominik.Marszk at esa.int>; Peter Van Der Plas <Peter.van.der.Plas at esa.int>; Cesar Coelho <Cesar.Coelho at ext.esa.int>; Lenzen, Christoph <Christoph.Lenzen at dlr.de>; Wörle, Maria Theresia <Maria.Woerle at dlr.de>; luke.berry at gmv.com; Rachel Jenkins <rjenkins at gmv.com>; Ferguson, Eric W (US 3970) <eric.w.ferguson at jpl.nasa.gov>; geoff.lochmaier at nasa.gov; David Frew <David.Frew at esa.int>; olly.page at cgi.com; clement.hubin-andrieu at cnes.fr
Cc: Guillermo Buenadicha via MOIMS-MP <moims-mp at mailman.ccsds.org>
Betreff: MPS: Test Case specification V19

Dear Marvin:

Thanks for your inputs to the V18, that have resulted in a V19, available in the GDrive and also attached here.

In orange you may see some questions/doubts I have on some of the tests, I would like to raise these to the attention of the WG. See that in the Notes section.

As reported in the meeting today, I will proceed to a version 20 implementing the changes of the BB post Agency review, kindly compiled by Peter (thanks!!!). Since this will be a heavy work, if you find something in between or we could get clarification to my points that would indeed help. Some of the existing test steps will be waived by the BB, and some service implementation or data elements will change, but at least I would like to have the logic of the services clear.

Regards!

G

-----------------------------------------------------------------------------
Guillermo Buenadicha, SCI-SOE
Euclid SOC Operations Coordinator
SCO-02 Technical Responsible
European Space Astronomy Centre (ESAC) ESA
Villanueva de la Cañada, Madrid, SPAIN
Phone  (34) 686 50 52 68
guillermo.buenadicha at esa.int<mailto:guillermo.buenadicha at esa.int>
----------------------------------------------------------------------------


This message is intended only for the recipient(s) named above. It may contain proprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo at esa.int<mailto:dpo at esa.int>).
This message is intended only for the recipient(s) named above. It may contain proprietary information and/or protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo at esa.int).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/moims-mp/attachments/20250210/c1e41f8f/attachment-0001.htm>

More information about the MOIMS-MP mailing list