[CESG] CESG-P-2017-06-003 Approval to release CCSDS 522.2-R-1, Mission Operations—Mission Data Product Distribution Services (Red Book, Issue 1) for CCSDS Agency review

Thu Oct 18 14:15:34 UTC 2018

Dear CESG Members,

Conditions for approval of CESG-P-2017-06-003 Approval to release CCSDS 522.2-R-1, Mission Operations—Mission Data Product Distribution Services (Red Book, Issue 1) for CCSDS Agency review have been disposed to the satisfaction of the AD(s) who voted to approve with conditions. The Secretariat will now proceed with CMC polling to authorize release for CCSDS Agency review.

On 18/10/2018 15:32, Barkley, Erik J (3970) wrote:

Sam,

Thanks for taking the time to chat during the Berlin meetings.  I now consider my conditions to be retired.

Best regards,

-Erik 

From: Sam Cooper  <mailto:sam at brightascension.com> <sam at brightascension.com> 
Sent: Thursday, October 11, 2018 00:38
To: Barkley, Erik J (3970)  <mailto:Erik.J.Barkley at jpl.nasa.gov> <Erik.J.Barkley at jpl.nasa.gov>; Thomas Gannett  <mailto:thomas.gannett at tgannett.net> <thomas.gannett at tgannett.net>; Burleigh, Scott C (312B)  <mailto:Scott.C.Burleigh at jpl.nasa.gov> <Scott.C.Burleigh at jpl.nasa.gov>
Cc: Mario.Merri at esa.int <mailto:Mario.Merri at esa.int> ; 'Smith, Danford S. (GSFC-5800)'  <mailto:danford.s.smith at nasa.gov> <danford.s.smith at nasa.gov>; Shames, Peter M (312B)  <mailto:Peter.M.Shames at jpl.nasa.gov> <Peter.M.Shames at jpl.nasa.gov>
Subject: Re: CCSDS MO MDPD Service specification

Hi Erik,

We now have a fuller example given in section 2.3.1. For the COM archive usage, these relationships are detailed in section 3 (specifically 3.2.(4, 5, 6) and 3.3.(4, 5, 6)).

Best regards,
Sam.

From: Burleigh, Scott C (312B) [mailto:Scott.C.Burleigh at jpl.nasa.gov] 
Sent: Wednesday, October 10, 2018 1:33 PM
To: Sam Cooper <sam at brightascension.com>; Thomas Gannett <thomas.gannett at tgannett.net>; Barkley, Erik J (3970) <Erik.J.Barkley at jpl.nasa.gov>
Cc: Mario.Merri at esa.int; 'Smith, Danford S. (GSFC-5800)' <danford.s.smith at nasa.gov>; Shames, Peter M (312B) <Peter.M.Shames at jpl.nasa.gov>
Subject: RE: FW: CCSDS MO MDPD Service specification

Sorry, guys, I didn’t realize that a response to my stated condition on this book had been sent; I must have misfiled it.  Assuming that the response was the same explanation that was given for 522x0, I agree that my condition is resolved in exactly the same way that my condition on the earlier document was resolved.  My reservations are undiminished.

Scott

From: Sam Cooper <sam at brightascension.com <mailto:sam at brightascension.com> > 
Sent: Wednesday, October 10, 2018 9:00 AM
To: Thomas Gannett <thomas.gannett at tgannett.net <mailto:thomas.gannett at tgannett.net> >; Barkley, Erik J (3970) <Erik.J.Barkley at jpl.nasa.gov <mailto:Erik.J.Barkley at jpl.nasa.gov> >; Burleigh, Scott C (312B) <Scott.C.Burleigh at jpl.nasa.gov <mailto:Scott.C.Burleigh at jpl.nasa.gov> >
Cc: Mario.Merri at esa.int <mailto:Mario.Merri at esa.int> ; 'Smith, Danford S. (GSFC-5800)' <danford.s.smith at nasa.gov <mailto:danford.s.smith at nasa.gov> >; Shames, Peter M (312B) <Peter.M.Shames at jpl.nasa.gov <mailto:Peter.M.Shames at jpl.nasa.gov> >
Subject: Re: FW: CCSDS MO MDPD Service specification

Sure, no problem, just trying to keep things moving.

On 10/10/2018 16:56, Thomas Gannett wrote:

Thanks, Sam.

I am hoping to get an explicit OK from Scott for notification to the CESG; failing to receive that I will use what you just sent.

Tom

Logothete, L.L.C.

thomas.gannett at tgannett.net <mailto:thomas.gannett at tgannett.net> 

+1 443 472 0805

From: Sam Cooper [mailto:sam at brightascension.com] 
Sent: Wednesday, October 10, 2018 11:49 AM
To: Thomas Gannett; Erik.J.Barkley at jpl.nasa.gov <mailto:Erik.J.Barkley at jpl.nasa.gov> ; Scott C Burleigh
Cc: Mario.Merri at esa.int <mailto:Mario.Merri at esa.int> ; 'Smith, Danford S. (GSFC-5800)'; 'Shames, Peter M (312B)'
Subject: Re: FW: CCSDS MO MDPD Service specification

Hi Tom,

Scott raised an identical comment on our specification for Common services (522x0):

Scott Burleigh (Approve with Conditions): This seems to be excellent work, but I don't see how it can be released as a draft Recommended Standard. According to section 6261 of the A02.1-Y-4 Procedures book, a document can be released as a Recommended Standard only after "two independent and interoperable prototypes or implementations ... have been developed and demonstrated in an operationally relevant environment". Because this specification relies wholly on the definitions of MAL and COM, which are abstract and therefore cannot be implemented, I don't see how two interoperable prototypes of this specification can ever be independently developed and demonstrated. I think this book is probably an excellent statement of *requirements* for a new Recommended Standard, hence an excellent Green Book. But I don't know how it can be released as a Red Book. My approval is conditional upon an explanation.

Response:

Our services are defined in an abstract notation that becomes concrete when one of our technology mappings is used. Currently we have one published (524.1-B-1) and three in the CCSDS pipeline. So, to test these service specifications you just need to select a mapping and test using that. It is what we have done for the services defined in the COM specification.

The reality is that all these parts form aspects of the concrete protocol, you need to select a book from each aspect to be able to implement it all. So you need the MAL and COM, and you need a service specification, and you need a technology mapping. You probably also might as well choose one of our APIs (Magenta books) but that is not needed for interoperability. None of these are Green Book material (informational) they are all specification. We separate them into separate books as they are not specific to any one combination, i.e. you can use out SPP mapping for any service, in the same way you can use this service specification over any technology mapping.

The issue is that currently CCSDS book colouring does not really cover this approach if taken literally, I suspect Blue Book were originally intended for something very fixed and very focussed such as time format encodings. Now that we, and other working groups, are looking to create suites of standards that can be combined and recombined in different ways (but are still formal specification in their own right but only part of the whole) the current book colourings get a bit confusing.

So, TL;DR, these are formal specifications and therefore should be Blue. Yes, you must combine these with other existing SM&C Blue books to be implementable/testable, but that approach has already been agreed as long as we clearly identify this “stack” in the yellow book produced during testing. 

Scott's response to this was (email dated 14/09/2017):

Sam, I recognize that your response to the condition I stated reflects what has become the reality of CCSDS specification approval thinking, and on that basis I guess I have to say that you have met my condition.

This is by no means to say that I find this current reality satisfactory.  If the Blue Book definitions and procedures in the Yellow Book cannot accommodate the work that is being done for SM&C, then either the Yellow Book must be updated to reflect reality or else the SM&C books cannot be Blue.  If we are doing otherwise then we are kidding ourselves, and in the long run that compromise will damage CCSDS.

So whilst I cannot comment on higher regards of Yellow book procedures, I would say that we have met Scott's identical condition on this.

For Erik's comments, if these are not addressed directly then I would suggest we move these to Agency review RIDs to keep the moment up on this specification.

Best regards,
Sam.

On 10/10/2018 16:21, Thomas Gannett wrote:

Erik, Scott:

I have received indication that Peter has signed off on resolution of his poll conditions for the poll below, but I have so far failed to find responses from you two. 

Can you either forward correspondence related to resolution of your conditions or signify by return email your acquiescence?

The current version is attached.

* * * * * * * * * * * * * * * * * * * * * * * *

CESG E-Poll Identifier: CESG-P-2017-06-003 Approval to release CCSDS 522.2-R-1, Mission Operations—Mission Data Product Distribution Services (Red Book, Issue 1) for CCSDS Agency review

Results of CESG poll beginning 30 June 2017 and ending 14 July 2017:

                 Abstain:  1 (16.67%) (Calzolari)

Approve Unconditionally:  2 (33.33%) (Merri, Behal)

Approve with Conditions:  3 (50%) (Barkley, Shames, Burleigh)

Disapprove with Comment:  0 (0%)

CONDITIONS/COMMENTS:

Erik Barkley (Approve with Conditions): Two conditions, and a general observation, question.

1) Pg 2-2 , at the top, RE: "...The product type represents a category of mission data to which the product belongs

(e.g., parameter value evolution in a given time period, actions history)..."

How is a "parameter value evolution in a given time period.." a product type? I suspect the type, as in classification, that is being attempted here is engineering telemetry. If I think of controlling, via product type, I doubt I'd be selecting access control just for evething that is on a time-series basis -- rather I think it would be types of telemetry/particular instrument readings, etc. They all have a time-series nature, but that is not the fundamental type. Can a better example be provided?

2) Pg 2-7 states that the catalog entry is created and stored in the "COM Archive". Please include a mapping that ensures the "the product type, the product source", and the "product format" conform with COM Archive service needs (its not clear how this maps, to, for example, "area service", "object instance identifier", etc)

General observation/question (not a condition): in reviewing this, I could not but help to think about the fact that some CCSDS member space agencies already have various data distribution capabilities in place. For example, ESA has the Planetary Science Archive (PSA) and NASA has the Planetary Data System (PDS). To the best of my knowledge these organizations have already cooperated in developing standards related to distribution of various mission science products, particularly, through the IPDA (for example see https://planetarydata.org/standards/IPDA_PDAP_v1.0.pdf) I can't help but wonder with regard to this recommendation if in fact CCSDS is not overlapping with standards that have already been established. Perhaps there is a need for a liaison to be established? At the least, it seems that perhaps there could be supporting standards to this recommendation and therefore an overall stronger body of work.

Peter Shames (Approve with Conditions): This document is too immature to be esnt out for agency review.See attached PID form and document with mark-ups for details.

Scott Burleigh (Approve with Conditions): This seems to be excellent work, as best I can tell, but I don't see how I can in good conscience approve its release as a draft Recommended Standard. According to section 6261 of the A02.1-Y-4 Procedures book, a document can be released as a Recommended Standard only after "two independent and interoperable prototypes or implementations ... have been developed and demonstrated in an operationally relevant environment". Because this specification is entirely abstract, I don't see how two interoperable prototypes of the specification can ever be independently developed and demonstrated. I think this book is probably an excellent statement of *requirements* for a new Recommended Standard, hence an excellent Green Book. But I don't know how it can be released as a Red Book. My approval is conditional upon an explanation.

Total Respondents: 6

No response was received from the following Area(s):

SOIS

SECRETARIAT INTERPRETATION OF RESULTS:  Approved with Conditions

PROPOSED SECRETARIAT ACTION:            Generate CMC poll after conditions have been addressed

Logothete, L.L.C.

thomas.gannett at tgannett.net <mailto:thomas.gannett at tgannett.net> 

+1 443 472 0805

From: Mario.Merri at esa.int <mailto:Mario.Merri at esa.int>  [mailto:Mario.Merri at esa.int] 
Sent: Friday, October 05, 2018 8:59 AM
To: thomas.gannett at tgannett.net <mailto:thomas.gannett at tgannett.net> 
Cc: Sam Cooper; Dan Smith
Subject: Re: CCSDS MO MDPD Service specification

Dear Tom, 

I think that now the document can be put finally under Agency Review. Sam will provide you with the agreed version of the document. 

Thanks, 

__Mario 
----- Forwarded by Mario Merri/esoc/ESA on 05/10/2018 14:56 ----- 

From:        "Shames, Peter M (312B)"  <mailto:Peter.M.Shames at jpl.nasa.gov> <Peter.M.Shames at jpl.nasa.gov> 
To:        Sam Cooper  <mailto:sam at brightascension.com> <sam at brightascension.com> 
Cc:        Maciej Józef Prokopczyk  <mailto:mprokopczyk at gmv.com> <mprokopczyk at gmv.com>, "Mario Merri"  <mailto:Mario.Merri at esa.int> <Mario.Merri at esa.int>, Mehran Sarkarati  <mailto:Mehran.Sarkarati at esa.int> <Mehran.Sarkarati at esa.int>, "Dan Smith"  <mailto:danford.s.smith at nasa.gov> <danford.s.smith at nasa.gov> 
Date:        05/10/2018 01:03 
Subject:        Re: CCSDS MO MDPD Service specification 

  _____  

Ok.  I (obviously) just do not have the spare cycles at this point to deal with this.  Agency review will have to suffice.

Cheers, Peter

From: Sam Cooper  <mailto:sam at brightascension.com> <sam at brightascension.com>
Date: Thursday, October 4, 2018 at 1:10 AM
To: Peter Shames  <mailto:Peter.M.Shames at jpl.nasa.gov> <Peter.M.Shames at jpl.nasa.gov>
Cc: Maciej Józef Prokopczyk  <mailto:mprokopczyk at gmv.com> <mprokopczyk at gmv.com>, Mario Merri  <mailto:Mario.Merri at esa.int> <Mario.Merri at esa.int>, Mehran Sarkarati  <mailto:Mehran.Sarkarati at esa.int> <Mehran.Sarkarati at esa.int>, Dan Smith  <mailto:danford.s.smith at nasa.gov> <danford.s.smith at nasa.gov>
Subject: Re: CCSDS MO MDPD Service specification

HI Peter,

Just to let you know I'm assuming Maciej's update is satisfactory for you. I'm going to apply a bit of polish to it and pass it to Tom for further processing. 

My opinion is that anything left outstanding should be delegated to the Agency review at this point.

Best regards,
Sam.

On 20/09/2018 08:40, Sam Cooper wrote:

Hi Peter,

Just a gentle reminder to cast your eyes over Maciej's updated document. Just to keep things moving, if I've not heard anything by the end of September I'll assume nothing major is wrong and update it and pass it to Tom.

Cheers,
Sam.

On 14/08/2018 09:24, Sam Cooper wrote:

Peter, could you check you are happy with the updates as made?

Once that's done I'll do a pass over it to check it before we pass it back to Tom for putting out for Agency review.

Sam.

On 13/08/2018 16:33, Maciej Józef Prokopczyk wrote:

Dear All,

Here is a document with (I hope) all updates requested in the Excel file (attached) and also updates Sam asked for (examples of type, source, format).

Please let me know if it is OK and how should we proceed.

Best regards,

Maciej

  _____  

Maciej Prokopczyk
Kierownik projektów, Inżynier oprogramowania/
Project manager, Software engineer 

GMV 
Ul. Hrubieszowska 2
01-209 Warszawa
Tel. +48 22 395 51 65
Fax +48 22 395 51 67 
 <http://www.gmv.com/> www.gmv.com 

From: Sam Cooper [ <mailto:sam at brightascension.com> mailto:sam at brightascension.com] 
Sent: Thursday, August 2, 2018 12:26 PM
To: Maciej Józef Prokopczyk  <mailto:mprokopczyk at gmv.com> <mprokopczyk at gmv.com>
Cc: Mario Merri  <mailto:Mario.Merri at esa.int> <Mario.Merri at esa.int>; Mehran Sarkarati  <mailto:Mehran.Sarkarati at esa.int> <Mehran.Sarkarati at esa.int>; Dan Smith  <mailto:danford.s.smith at nasa.gov> <danford.s.smith at nasa.gov>
Subject: Re: CCSDS MO MDPD Service specification

Attached.

I was wondering if we could add the following modifications to the specification to help users understand what kind of data this service can be used for. At the moment it is not obvious that it can be used for data other than existing MO services (which is not the case as I understand it).

So in section 1.6 (Definitions) expand "Product Type" to something like:
product type: A distinct category of mission data, e.g., CCSDS space packet, navigation orbit files, image files, parameter time series, action history.

Then add to section 2 of the specification an expansion of those examples to really explain the concept that it can hold anything but also to give real examples of the product type, source, format. I think it would be good to add a section 2.2.1 that expands the following examples:
 CCSDS Space packet
 Navigation orbit files
 Mission specific images files (for example JPEGs)
 Parameter time series
 Action history

If you could add in a first revision of these examples I'm happy to help you polish the text in that section.

Cheers,
Sam.

On 02/08/2018 10:31, Maciej Józef Prokopczyk wrote:

Hello Sam,

I will try to do it at the end of the next week. Could you provide me the last version of the specification in the Word format?

Best regards,

Maciej

  _____  

Maciej Prokopczyk
Kierownik projektów, Inżynier oprogramowania/
Project manager, Software engineer 

GMV 
Ul. Hrubieszowska 2
01-209 Warszawa
Tel. +48 22 395 51 65
Fax +48 22 395 51 67 
 <http://www.gmv.com/> www.gmv.com 

From: Sam Cooper [ <mailto:sam at brightascension.com> mailto:sam at brightascension.com] 
Sent: Thursday, August 2, 2018 11:14 AM
To: Maciej Józef Prokopczyk  <mailto:mprokopczyk at gmv.com> <mprokopczyk at gmv.com>
Cc: Mario Merri  <mailto:Mario.Merri at esa.int> <Mario.Merri at esa.int>; Mehran Sarkarati  <mailto:Mehran.Sarkarati at esa.int> <Mehran.Sarkarati at esa.int>; Dan Smith  <mailto:danford.s.smith at nasa.gov> <danford.s.smith at nasa.gov>
Subject: Re: CCSDS MO MDPD Service specification

Hi Maciej,

Ok so the next step is to update the document taking into account the changes listed in that Excel sheet, don't forget to include the changes requested by Peter in the last set of discussions as the sheet still has those marked down as Reject.

When do you think you will be able to do this update by?

Sam.

On 01/08/2018 16:18, Shames, Peter M (312B) wrote:

Hi Sam,

Ok.  I can accept those changes.

Thanks, Peter

From: Sam Cooper  <mailto:sam at brightascension.com> <sam at brightascension.com>
Date: Wednesday, August 1, 2018 at 8:07 AM
To: Peter Shames  <mailto:peter.m.shames at jpl.nasa.gov> <peter.m.shames at jpl.nasa.gov>
Cc: Mario Merri  <mailto:Mario.Merri at esa.int> <Mario.Merri at esa.int>, Mehran Sarkarati  <mailto:Mehran.Sarkarati at esa.int> <Mehran.Sarkarati at esa.int>, Dan Smith  <mailto:danford.s.smith at nasa.gov> <danford.s.smith at nasa.gov>, Maciej Józef Prokopczyk  <mailto:mprokopczyk at gmv.com> <mprokopczyk at gmv.com>
Subject: Re: CCSDS MO MDPD Service specification

Hi Peter,

Sorry if you got caught up in internal discussions, not my intension!

I think we've finally hammered out the last few updates in the excel sheet for you to review, we've only looked at your yellow commented lines and have proposed next to it a change or clarification (ignore the other columns where we say REJECTED etc, we just haven't updated those yet).

Could you check that you are happy with these proposed changes please and then we can move this forward.

Best regards,
Sam.

On 09/07/2018 16:27, Shames, Peter M (312B) wrote:

Ah, and now the attachment …

-p

From: Sam Cooper  <mailto:sam at brightascension.com> <sam at brightascension.com>
Date: Thursday, June 14, 2018 at 7:32 AM
To: Peter Shames  <mailto:peter.m.shames at jpl.nasa.gov> <peter.m.shames at jpl.nasa.gov>
Cc: Mario Merri  <mailto:Mario.Merri at esa.int> <Mario.Merri at esa.int>, Mehran Sarkarati  <mailto:Mehran.Sarkarati at esa.int> <Mehran.Sarkarati at esa.int>, Dan Smith  <mailto:danford.s.smith at nasa.gov> <danford.s.smith at nasa.gov>, Maciej Józef Prokopczyk  <mailto:mprokopczyk at gmv.com> <mprokopczyk at gmv.com>
Subject: CCSDS MO MDPD Service specification

Hi Peter,

Apologies in advance here, this may take some recollection on your part but hopefully you'll be able to work it out!

You provided some comments on one of our draft specification as part of CESG review before Agency review almost exactly one year ago. After a lot of going round the houses we have finally managed to collection a set of responses to your comments. 

I have attached them in a spreadsheet and also include your original PDF review document with your comments in for your reference. Please could you cast your eye over the responses and let me know what you think.

Again, sorry for the long delay, hopefully you can get to grips with it again and we can get this specification moving!

Best regards,
Sam.

P Please consider the environment before printing this e-mail. 

This message is intended only for the recipient(s) named above. It may contain proprietary information and/or
protected content. Any unauthorised disclosure, use, retention or dissemination is prohibited. If you have received
this e-mail in error, please notify the sender immediately. ESA applies appropriate organisational measures to protect
personal data, in case of data privacy queries, please contact the ESA Data Protection Officer (dpo at esa.int <mailto:dpo at esa.int> ).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/cesg/attachments/20181018/04465567/attachment.html>