[CESG] CESG review of CCSDS MO M&C Services book
Mario.Merri at esa.int
Mario.Merri at esa.int
Fri Mar 3 11:27:12 UTC 2017
Dear Peter,
OK, then we move on following the below approach.
As for the YB, I recommend you give a second read. In fact in section
6.2.5.2 it is stated:
a) if technical issues are identified in the course of a review, those
issues must be
resolved and the review must be repeated before approval can be sought for
a change
of document status;
There is no concept of "substantive changes … to a document". What I read
is: if there are technical issues (and there is a flag in the CCSDS form
to identify "Technical Facts"), the review must be repeated. Period.
The "substantive changes" come only in the clause
c) if substantive changes are made to a document that has completed review
without
technical comment, the Secretariat shall conduct a final review in which
Agencies
can approve or reject the document but may not suggest additional changes;
In the above (which, by the way, I do not fully understand), technical
comments are explicitly excluded. I might have completely misunderstood
the text, but this could be already an indication that it is not that
clear.
Again, if everyone is reasonable I do not see a problem to handle
imprecise and confusing statements in the YB, but this is not always the
case.
Regards,
__Mario
From: "Shames, Peter M (312B)" <peter.m.shames at jpl.nasa.gov>
To: "Mario.Merri at esa.int" <Mario.Merri at esa.int>
Cc: "cesg at mailman.ccsds.org" <cesg at mailman.ccsds.org>, Dan Smith
<danford.s.smith at nasa.gov>, "Barkley, Erik J (3970)"
<erik.j.barkley at jpl.nasa.gov>, Nestor Peccia <Nestor.Peccia at esa.int>, "Sam
Cooper" <sam at brightascension.com>, "Secretariat at mailman.ccsds.org"
<Secretariat at mailman.ccsds.org>
Date: 01/03/2017 20:19
Subject: Re: CESG review of CCSDS MO M&C Services book
Dear Mario,
Thank you for being reasonable and recognizing that the magnitude of the
changes made to the document do, in fact, require another Agency Review.
The steps you identified seem reasonable and I believe that they are in
line with our agreed processes.
I cannot speak for the CCSDS Chief Tech Editor, he will utilize whatever
level of review he judges is required to get the document into final form.
That is his responsibility. The intent, of course, is to have the
document in a form where if it does pass Agency Review without any
substantive changes it can be offered for approved by the CESG and CMC,
and then published, according to the established procedures.
I will point out that the operative words re what triggers another review
are "substantive changes … to a document". I grant you that these words
(most words, in fact) are subject to interpretation. I believe that these
words were used because they are commonly understood. It would be
possible, but unlikely, for 159 RIDs to all result in minor editorial
changes were not substantive (except in number) and would not trigger
another review. As you have agreed, this was not the case here.
I will agree that CCSDS has not precisely defined the meaning of
"substantive", some organizations do. Good dictionary definitions, from
Merriam Webster and others are these:
"having substance : involving matters of major or practical importance to
all concerned"
"considerable in amount or numbers"
or, from Vocabulary.com
"having a firm basis in reality and being therefore important, meaningful,
or considerable"
or, from Dictionary.com
"possessing substance; having practical importance, value, or effect"
I think any of these accurately convey the intent. I do not think that we
need to make substantive changes to our procedures in order to address
this. The procedures and their intent have been understood and adhered to
for years by most of the Areas and WG and adding more rules is likely only
to confuse. That said, perhaps adding a definition like one of these
would help clarify?
Best regards, Peter
From: Mario Merri <Mario.Merri at esa.int>
Date: Wednesday, March 1, 2017 at 10:02 AM
To: Peter Shames <peter.m.shames at jpl.nasa.gov>
Cc: CCSDS Engineering Steering Group - CESG Exec <cesg at mailman.ccsds.org>,
Dan Smith <danford.s.smith at nasa.gov>, Erik Barkley
<erik.j.barkley at jpl.nasa.gov>, Nestor Peccia <Nestor.Peccia at esa.int>, Sam
Cooper <sam at brightascension.com>, CCSDS Secretariat
<Secretariat at mailman.ccsds.org>
Subject: Re: CESG review of CCSDS MO M&C Services book
Dear Peter,
sorry for the late answer, but I have been one week on holiday.
Please note that the document is largely auto-generated and MS Word has
trouble with matching text when its in tables, and we have a lot of
tables. Hence a pure MS-Word change track checking is misleading. Also,
the CCSDS guidelines are quite vague as they call for a new AR "if
technical issues are identified in the course of a review ..." without
qualifying the type of technical issues. Indeed, the CCSDS MO M&C Services
book R3 AR had 159 RIDs of which 129 resulted in some change to the
document. Should we check if other documents under review had "technical
issues" and call for a re-review? For instance, did the CSS Monitoring
data BB had no technical issue identified out of the RIDs raised? I have
checked and it is certainly not the case.
What I want to say is that if there are rules, they should be clear and
applied fairly to everyone. I suggest that the CESG looks into these rules
in view of making them clearer.
Having said that, I propose to move forward and go for the 4th Agency
Review. The document to be reviewed will be the final version submitted
for publication after the 3rd AR amended with the changes coming from the
CESG review by you and Erik. You and Erik will be provided with a version
with change bars so that you can see how your comments have been
implemented. Considering that this would be the 4th AR, I have discussed
the matter with the CESG chair and we have agreed to proceed with an
expedite 4th AR. This implies:
· No need for CESG/CMC poll for AR (allowed by the YB)
· No/minimal re-checking of the draft document by Tom before
putting it out for review (being the 4th review, the document should be
already in the proper CCSDS format). This implies that this document will
be put on top of Tom's queue.
· 1 month review time (allowed by the YB). Please note that the
review community of this document is largely within the SM&C WG and
therefore they are fully aware and in agreement of the changes
implemented.
I trust the CESG ADs will use this additional opportunity to raise any
outstanding issue so that we can have a smooth CESG poll after the 4th AR.
Regards,
__Mario
From: "Shames, Peter M (312B)" <peter.m.shames at jpl.nasa.gov>
To: "Mario.Merri at esa.int" <Mario.Merri at esa.int>
Cc: "Dan Smith" <danford.s.smith at nasa.gov>, "Barkley, Erik J
(3970)" <erik.j.barkley at jpl.nasa.gov>, "Nestor Peccia"
<Nestor.Peccia at esa.int>, "Sam Cooper" <sam at brightascension.com>,
"Secretariat at mailman.ccsds.org" <Secretariat at mailman.ccsds.org>,
"cesg at mailman.ccsds.org" <cesg at mailman.ccsds.org>
Date: 16/02/2017 19:17
Subject: Re: CESG review of CCSDS MO M&C Services book
Mario,
For some reason you persist in trying to convolve the role that the Area
Directors and the whole CESG is supposed to play in ensuring the quality
of CCSDS documents with that of an Agency Review participant. I have
quoted this section of the CCSDS Organization and Processes, A02.1-Y-4,
before, but you seem to keep ignoring it.
2.3.2.4.3 An Area Director is responsible for the work done in his or her
WGs, BOFs, and SIGs and is specifically responsible for the following:
d) ensuring that CCSDS documents are properly categorized and that they
embody the content and quality expected of documents of their type;
There is no part of the CCSDS Processes that requires an AD to only submit
inputs as a part of an agency review. That is something you have made up.
On the contrary, the requirement, as clearly stated, is for ADs, and the
CESG as a whole, for "identifying “red flag” items where technical work in
a proposed CCSDS document is not of the required quality or nature". I
take this responsibility seriously and try my hardest to carry out the
role that has been assigned. Given that situation, and the effort that is
involved, I do not make the choice to also participate in Agency Reviews
and to read through every document twice, or three times, I just do not
have the time.
You should be aware that the intended flow of Red Book processing is that
by the time they get to Agency Review they are mature documents. It
should be the case that following an Agency Review that the only minor
changes to the document are made and that the document is then easily
prepared for final CESG and CMC review and approval. If a document goes
through major revisions after an Agency Review it is supposed to be sent
out for another Agency Review.
6.2.5 Formal Agency Review
a) if technical issues are identified in the course of a review,
those issues must be resolved and the review must be repeated before
approval can be sought for a change of document status;
c) if substantive changes are made to a document that has completed
review without technical comment, the Secretariat shall conduct a final
review in which Agencies can approve or reject the document but may not
suggest additional changes;
I must admit to being somewhat puzzled as to why there were so many issues
identified in this document that was supposed to have been reviewed by the
usual Agency Review process. I had the feeling that there was a lot of
new material and so I asked the Tech Editor to confirm that the document
sent for Agency Review Red-3 was, in fact, a minor revision to the one
sent to the CESG. That comparison is attached. I think that if you take
the time to review this comparison that you will find that there were, in
fact, many substantive changes made to the document between what was sent
to Agency Review Red-3 and what was later sent to the CESG for final
review.
Based on this I can only conclude that you, as AD, and the SM&C WG, did
not follow the CCSDS Procedures and that this document, because of these
substantive changes, should have been sent out for another Agency Review.
I recommend that you and the WG review the inputs Erik and I provided and
apply as many of them as seem acceptable before sending the document out
for the next Agency Review.
The "Show Stopper" PID that you requested is attached.
Regards, Peter Shames
________________________________________________________
Peter Shames
CCSDS System Engineering Area Director
Jet Propulsion Laboratory, MS 301-490
California Institute of Technology
Pasadena, CA 91109 USA
Telephone: +1 818 354-5740, Fax: +1 818 393-6871
Internet: Peter.M.Shames at jpl.nasa.gov
________________________________________________________
We must recognize the strong and undeniable influence that our language
exerts on our ways of thinking and, in fact, delimits the abstract space
in which we can formulate - give form to - our thoughts.
Niklaus Wirth
From: Mario Merri <Mario.Merri at esa.int>
Date: Thursday, February 16, 2017 at 1:04 AM
To: Peter Shames <peter.m.shames at jpl.nasa.gov>
Cc: Dan Smith <danford.s.smith at nasa.gov>, Erik Barkley
<erik.j.barkley at jpl.nasa.gov>, Nestor Peccia <Nestor.Peccia at esa.int>, Sam
Cooper <sam at brightascension.com>, CCSDS Secretariat
<Secretariat at mailman.ccsds.org>, CCSDS Engineering Steering Group - CESG
Exec <cesg at mailman.ccsds.org>
Subject: Re: CESG review of CCSDS MO M&C Services book
Peter,
could you please kindly reply to my note and provide the PIDs?
Thanks
__Mario
----- Forwarded by Mario Merri/esoc/ESA on 16/02/2017 10:03 -----
From: Mario Merri/esoc/ESA
To: "Shames, Peter M (312B)" <peter.m.shames at jpl.nasa.gov>
Cc: "Dan Smith" <danford.s.smith at nasa.gov>, "Barkley, Erik J
(3970)" <erik.j.barkley at jpl.nasa.gov>, "Nestor Peccia"
<Nestor.Peccia at esa.int>, "Sam Cooper" <sam at brightascension.com>,
"Secretariat at mailman.ccsds.org" <Secretariat at mailman.ccsds.org>,
cesg at mailman.ccsds.org
Date: 10/02/2017 23:13
Subject: Re: CESG review of CCSDS MO M&C Services book
Dear Peter,
Sam will proceed as you recommended, i.e. Sam will update the document so
that you can re-re-review it, but only for the showstopper issues.
As done already at the last CESG in Rome, Brigitte and I would like to
voice again our discontent on your approach that repeated also on this
CESG poll. What we had agreed in Rome and agreed by the CMC was:
Scope of conditions prior to Publication (including AD/DAD participation
in Agency Reviews)
- CESG suggests that AD/DAD minimize their conditions at time of CESG
book publication polls. CESG members have the opportunity to raise
technical issues prior / during Agency Review(s)
...
- CESG recommends to raise PIDs (Poll Item Discrepancy) at least for
showstopper conditions during polls and identify conditions that are non
showstopper as such.
We believe that your 91 points raised at CESG poll for publication and
your insistence to have the document re-updated (after the already made
update that followed the Agency Review) for your own personal review are
not in line with the above agreement. We repeat once more: your detailed
comments are very welcome, but they must be channelled through the Agency
Review. If this is not done, the already scarce agency resources will be
wasted with work duplication and with frustrated WG members.
In order to speed up the work and respect the agreement in Rome, could you
please generate asap the PIDs for those points that you consider
showstoppers. For those, Sam will provide you with the red-lined version
of the document. Please provide the PIDs by eob 14Feb17 (they should have
been provided at the closure of the CESG poll). We think that this is an
acceptable compromise at this stage. In the future, ideally this should
not happen again and your detailed comment must be raised during the
agency review period.
Regards,
__Mario
From: "Shames, Peter M (312B)" <peter.m.shames at jpl.nasa.gov>
To: "Sam Cooper" <sam at brightascension.com>, "Barkley, Erik J
(3970)" <erik.j.barkley at jpl.nasa.gov>
Cc: "Secretariat at mailman.ccsds.org"
<Secretariat at mailman.ccsds.org>, "Mario Merri" <Mario.Merri at esa.int>, "Dan
Smith" <danford.s.smith at nasa.gov>, "Nestor Peccia" <Nestor.Peccia at esa.int>
Date: 09/02/2017 22:41
Subject: Re: CESG review of CCSDS MO M&C Services book
Hi Sam,
In my viewer I see your comments stacked on top of mine and have to
separate them and view them separately. I did that for quite a number, so
I saw what you had replied. In some cases your notes said "Accepted" as
in this case. In others there was some other sort of a reply that
indicated agreement, or issues. Some of the conditions asked for new
figures or clarifications. And with all of that there are really a
significant number of changes requested for clarifications and re-wording.
If there were just a few issues using the PDF document could be made to
work. But in this case, until I see the document that is the result of
all these changes I could not possibly provide approval. I do not think
that trying to do this as an exchange of annotations and notes in the PDF
file is going to be a good way to accomplish this, so I suggest resorting
to actually editing the document.
Regards, Peter
From: Sam Cooper <sam at brightascension.com>
Date: Thursday, February 9, 2017 at 12:39 AM
To: Peter Shames <peter.m.shames at jpl.nasa.gov>, Erik Barkley
<erik.j.barkley at jpl.nasa.gov>
Cc: CCSDS Secretariat <Secretariat at mailman.ccsds.org>, Mario Merri
<Mario.Merri at esa.int>, Dan Smith <danford.s.smith at nasa.gov>, Nestor Peccia
<Nestor.Peccia at esa.int>
Subject: Re: CESG review of CCSDS MO M&C Services book
Hi Peter,
I would really like to acoid that situation as I would like to have
consensus on each point before making the changes, this would be in the
hope of avoiding multiple revisions (and update/review cycles).
I think the problem might be the use of PDF as a review medium, although I
wonder if the PDF viewer you are using is missing some feature as I do not
see what you see:
As you can see, my comment is located in your comment as a reply. The
version of Acrobat that I am using is:
Shall we try again to use the PDF tools? It should be noted that the
version you just sent me has had the replies split, so you would have to
go back to the original I sent out before.
Regards,
Sam.
On 09/02/2017 04:42, Shames, Peter M (312B) wrote:
Hi Sam,
In trying to follow what you propose to change using this "my note covers
your note" approach I cannot get any kind of clear picture of what the
final document is going to look like. I suggest that you attempt to
implement the requested changes (at least those that you agree with) and
re-submit the integrated result. I commit to reviewing that and either
concurring, or not, as seems appropriate.
Thanks, Peter
From: Sam Cooper <sam at brightascension.com>
Date: Thursday, January 19, 2017 at 1:21 AM
To: Peter Shames <peter.m.shames at jpl.nasa.gov>, Erik Barkley
<erik.j.barkley at jpl.nasa.gov>
Cc: CCSDS Secretariat <Secretariat at mailman.ccsds.org>, Mario Merri
<Mario.Merri at esa.int>, Dan Smith <danford.s.smith at nasa.gov>, Nestor Peccia
<Nestor.Peccia at esa.int>
Subject: CESG review of CCSDS MO M&C Services book
Dear Erik and Peter,
Thank you for taking the time to review the specification and provide your
comments. Please find attached the WG response to your comments including
the detailed marked up copy from Peter (I have responded in the PDF like
you did Peter).
Please could you let us have your response by Friday the 3rd of Feb.
Best regards,
Sam.
This message and any attachments are intended for the use of the addressee
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the
sender.
Please consider the environment before printing this email.
[attachment "MO M&C Services Input for Blue Compared to Red 3.doc"
deleted by Mario Merri/esoc/ESA] [attachment "PID SEA-522x1b0-001.txt"
deleted by Mario Merri/esoc/ESA]
This message and any attachments are intended for the use of the addressee
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the
sender.
Please consider the environment before printing this email.
This message and any attachments are intended for the use of the addressee
or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in
whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete
it from your system.
Emails can be altered and their integrity cannot be guaranteed by the
sender.
Please consider the environment before printing this email.
This message and any attachments are intended for the use of the addressee or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete it from your system.
Emails can be altered and their integrity cannot be guaranteed by the sender.
Please consider the environment before printing this email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/cesg/attachments/20170303/e346a1bd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 154643 bytes
Desc: not available
URL: <http://mailman.ccsds.org/pipermail/cesg/attachments/20170303/e346a1bd/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 27852 bytes
Desc: not available
URL: <http://mailman.ccsds.org/pipermail/cesg/attachments/20170303/e346a1bd/attachment-0001.png>
More information about the CESG
mailing list