[CESG] RE: MORE COMMENTS on Document Review System ? Coordinator page in your Cleveland presentation (but not only)

Kearney, Mike W. (MSFC-EO01) mike.kearney at nasa.gov
Wed Dec 5 13:39:35 EST 2012 

Very good responses, Gian-Paolo, thanks much.

Your suggestions on more specific "Disposition settings" are all very good, except "Reconditioned".  You also say "Rediscussed".  I think you're trying to distinguish between "Not Yet Reviewed" and "Reviewed once, but still undecided".  The implication is that once a coordinator looks at a RID he can't set it back to "Not Yet Reviewed".  Is that correct?

If so, we can allow "Not Yet Reviewed" to be set only by the system (not the coordinator).  Once the coordinator looks at a RID, instead of "To Be Reconditioned" perhaps he can set it to something like "Under Review"?

If I understand what you're asking, I'm sure we can do that.  We will take the rest of your list of "Disposition Settings" as our main guidance to Brian right now.

I think we understand your desire for "holding back RIDs until released".  One problem is that it is difficult to implement within the Sharepoint platform.  But we will have Brian have a look at it.  I will also mention this during the CMC meeting as one of the open design questions.

Having more than one Agency Review Coordinator at a given time in the system (for different books) is not an issue...   but we are assuming there is only one Agency Review coordinator for one book.  I trust you aren't expecting more than one coordinator per book.

Yes, Gian-Paolo, I think we are very well converged.   Only a few open questions.

   -=- Mike

Mike Kearney
Lead Technology Manager
Mission Operations Laboratory
NASA MSFC EO-01
+1-256-544-2029

From: Gian.Paolo.Calzolari at esa.int [mailto:Gian.Paolo.Calzolari at esa.int]
Sent: Tuesday, December 04, 2012 12:19 PM
To: Kearney, Mike W. (MSFC-EO01)
Cc: CCSDS Tech Support; CESG -- CCSDS-Engineering Steering Group; Mario.Merri at esa.int
Subject: RE: MORE COMMENTS on Document Review System ? Coordinator page in your Cleveland presentation (but not only)

Mike,
        no apologies needed.
I will try to go in order.

It looks OK changing "Status" to "Disposition"  and changing "Disposition" to say "Rationale for Disposition".
However I think there a couple of remarks for the values allowed for  "Disposition"  wrt to which coordinator is doing that.

Therefore I would distinguish the "Disposition" settings allowed to each coordinator e.g.
a) WG Coordinator should be the only one allowed to set APPROVE / APPROVE WITH MODIFICATIONS
b) Agency Coordinator should be the only one allowed to set FORWARDED TO CCSDS
c) Center  Coordinator should be the only one allowed to set FORWARDED TO AGENCY
d) All  Coordinators should be the only ones allowed to set RETURN TO ORIGINATOR (i.e. go back to original author)
e) All  Coordinators should be allowed to set  REJECTED
f) All  Coordinators should be allowed to set  WITHDRAWN
g)  Agency+ Center Coordinators should be allowed to set  MODIFIED AND FORWARDED TO CCSDS/MODIFIED AND FORWARDED TO AGENCY
h) All  Coordinators should be allowed to set  TO BE RECONDITIONED

I would insist on having also option g to allow minor editing to be possible without triggering a ping-pong with the author, but informing the author about the changes.
Note that during a real WG review may not be time for RID bouncing and then the straight editing may be a useful alternative.

For option h I think that "NOT YET REVIEWED should be a starting state when a RID arrives to be distinguished from a RID switching from an approved/rejected status back into a not yet decided status (you may find something better than TO BE RECONDITIONED or TO BE REDISCUSSED :o) .

For the term "International Coordinator" opposite to "WG Chair Coordinator" or "CCSDS Coordinator" I do not have particular problems.

OK for your way of handling attached files. Attaching files is essential for e.g. drawings, etc.

Opposite to you,  we do see benefit in holding back" RIDs until a given later date to allow proper consolidation within a given Agency also for proper processing at next level.
As well the possibility for a "SEND ALL" option is very important e.g. in cases where there several RIDs. Clicking individually 200 RIDs can be a nightmare!

For the Agency Review Coordinator is OK getting it set by Secretariat , on request from the agency members in the CESG or CMC.
However I would make clear that there could be more than one Agency Review Coordinator at a given time; e.g. assume you have under review a book from SLS and one from MOIMS, then I could be the ESA coordinator the the SLS Book and Mario could be the ESA coordinator for the MOIMS book.

Creating an HTML report (eventually printable as pdf :o) looks satisfactory.

I hope we are converging.

Best regards

Gian Paolo





From:

"Kearney, Mike W. (MSFC-EO01)" <mike.kearney at nasa.gov>

To:

"Gian.Paolo.Calzolari at esa.int" <Gian.Paolo.Calzolari at esa.int>, CCSDS Tech Support <ccsds_techsupport at aiaa.org>

Cc:

CESG -- CCSDS-Engineering Steering Group <cesg at mailman.ccsds.org>

Date:

30/11/2012 22:06

Subject:

RE: MORE COMMENTS on Document Review System ? Coordinator page in your Cleveland presentation (but not only)


________________________________



Gian Paolo:  I'm sorry in advance for a long email, but you had some very good questions and observations, and I want to make sure they are addressed.  Here we go...

******************************

>  Rejected......
- Returned to Originator.......
First of all I am puzzled by the last two items looking (almost) identical. This shall be clarified

Rejected has the usual meaning... disapproved and will not be forwarded.
Returned to originator is for when the coordinator wants the originator to make some changes before he can accept/forward the RID.

But first... based on your and Mario's comments, on that screen on Page 11, we are changing "Status" to "Disposition" because when the coordinator selects that drop-down box, he is deciding on the disposition at his level (may not be "approve" but it is whatever the disposition is at his level).  And the text box that is below it that currently says "Disposition" will now say "Rationale for Disposition".  We think that will answer some of your confusion on the term "Disposition".

When the coordinator wants to return a RID to the originator to make some changes (to make it acceptable), he selects "Return to Originator" as the disposition and types the rationale for why he returned it in the text box.  When he is finished and clicks "Save" (at the top of the screen), then an email will be sent to the originator.  The originator then makes some changes (as requested by the coordinator) and then resubmits it ("Saves" it again).  If he doesn't resubmit for forwarding/approval, the RID will not be processed.

>  The reference to the Disposition is also puzzling me as it looks as the same term is used for the real WG Review as well as for internal Agency review and this is confusing
>  In this respect I think the operations performed by the last one are different from those performed by the other two guys.
I think you are saying that only the last level ("International" or "WG Chair") actually does a disposition, and the first two levels, Center or Agency, only forward them without a disposition.  You are correct.

Brian was simplifying the screens for his presentation on Page 11.  We did intend to have a different screen for center and agency level coordinators than we have for international (WG Chair) coordinators.  However, your discussion of "Accept" and "Approve" has exposed that we need to make things more clear.

I think that the choices for Center and Agency (lower level) coordinators should be:
*         REJECT (see explanation in "Rationale")
*         RETURN TO ORIGINATOR (for modification... see explanation in "Rationale")
*         FORWARD to next level

Meanwhile the choices for the International Coordinator will be:
*         REJECT (see explanation in "Rationale")
*         RETURN TO ORIGINATOR (for modification... see explanation in "Rationale")
*         APPROVE
*         APPROVE WITH MODIFICATION (see explanation in "Rationale")

Also, they will all be able to select "Not Yet Reviewed" to switch it from an approved/rejected status back into a not yet decided status.

Does all that make sense?    (Note... your suggestion for "TO BE FORWARDED" and "ALREADY FORWARDED" is addressed by the responses below)

>  I would assume that the last one is actually the WG Chair that will report the final WG decision.

The current system refers to the last level as the "International Coordinator".  Probably that is because a large WG might see that the chairman delegates to someone else in the WG.  If you think we need to call the last level "WG Chair" rather than "International Coordinator", we can consider that, but we should have some other CESG concurrence, since the current system uses the "International Coordinator" term.

>  If we want to allow capturing internal Center/Agency discussions you should have a mechanism for commenting back and forth (e.g. to ask modifications to the originator or to allow modifications to the originator).

The field (which was called "Disposition" on page 11) is now "Rationale for Disposition".  The coordinator selects "Return to Originator" as his disposition, and writes his explanation in "Rationale for Disposition".  When he clicks "Save" then an email alert will go to the originator with the text of the "Rationale for Disposition".   It's then up to the originator if he wants to revise as he has been asked.   Also...
*         When the International Coordinator does this, the email goes to the Agency and Center level as well as the Originator.
*         When the Agency level coordinator does this, the email goes to the Center level as well as the Originator.

>  In general I have the impression that the Review Coordinator will be able to modify the entered RID as he likes. This is practical but it may be give issues for changes not approved by the originator.

The system can be designed either way.  We can "lock out" anyone but the originator from making changes.  Or we can allow the Center/Agency/International coordinators to make changes.

I have no idea, with the current system, if it is common practice for a coordinator to "change" the RIDs from their originators.  Is that done routinely?  In that case, I think we should keep it as a capability.  With that philosophy, we are trusting the coordinators to make only minor or editorial changes, not changes to the originator's intent.  I think this would be faster, and prevent delays as coordinators go back-and-forth discussing changes to the RIDs.  This was the "default" assumption that Brian was using for the system he is developing.

However, if that is a problem and we want the originator *only* to revise his RIDs as requested by the "Return to Originator" function, then we can do that also.  Your opinions on this are welcome.

>  Page 11 - A Review Coordinator should be able to see all the data entered by the Originator but the snapshot shows nothing about attached files.

The attached files will be visible, and will have icons on the page if there is an attached file.  And they will also be on the view of all submitted RIDs.  However, you will see the icon on the page, not the actual document.  To see the document you have to click on the icon to open the attachment (like any email system functions).  This is provided by the normal SharePoint functions.  If the CESG thinks that attached files are not a good idea (because it adds workload or complexity to the coordinators) that can be easily "turned off".

>  Page 8 - Privacy Scheme. It is not clear to me whether a given Coordinator would be able to see a RID as soon as it is approved at previous level.

Yes he would.  The concept is that a RID is forwarded as soon as the lower level coordinator agrees.  This way it will give the next level, or the international coordinator, an earlier access to forwarded RIDs, allowing more time to work any issues, and speeding up the RID process.   No benefit is seen from "holding back" RIDs until the final deadline, or until some later date.

>  In general I think that APPROVED RIDs should remain within a Center?agency and to be forwarded only explicitly (e.g. Button at Center/Agency level to forward all approved RIDs).
This would change the concept.  If the CESG wants this delay mechanism, we can try to add it, but we want a clear consensus from the CESG that it prioritizes "synchronized delay" over "rapid resolution of RIDs".   To clarify...
When an originator submits to a center, nobody can see that RID except the originator and the Center coordinator.  Until the Center coordinator selects "FORWARD".  Then the next level will see it (immediately).   If the Center level coordinator changes his mind, and revises that RID to "REJECT", then it will disappear from the list of the Agency Coordinator.  The same process applies between the Agency coordinator and the International coordinator.  Nobody from other agencies can see an agency's RIDs until the Agency Coordinator clicks "Forward" and then "Save" for that RID.

If the CESG wants this delay mechanism, it may be difficult to implement in SharePoint.  But we are asked, Brian will look at it and try.  As I said, though, we need a clear consensus from the CESG on that.

>  We assume that the name of the  Agency Review Coordinator is configurable case by case and that delegation/addition of  an additional Review Coordinator should be easily possible within an agency

Yes.  The plan is that it can only be configured by the Secretariat, on request from the agency members in the CESG or CMC.  Does the CESG need to have the capability for you guys to configure the names of the coordinators?

>  Page 11 says "Review Coordinator will click on each RID and review...." but this can be very cumbersome. A possibility for doing <send "group/all"?

As we discussed above, this capability is difficult in SharePoint, and is not in the current plan.

>  On page 11 the Reviewer data are not visible and they should be visible.

I think you're asking for the Originator's name to be displayed.  In the existing screen, it is listed at the bottom of the screen (but it was cut off at Brian's page 11).  But we will look at making it more prominent at the top of the screen by the RID title.

>  Tools to show/print all RIDs (e.g. generate pdf Report) should be available to facilitate the task of the review coordinator and distribution.

PDFs are a particular problem, but we may be able to create an HTML page that can be printed to provide the same "hardcopy report" capability.  We will look into that.  Also, remember that the whole set can be exported to Excel to be viewed/worked when offline.  (But of course, as with all other systems, we cannot import that Excel file back into the system).

******************************
I hope those responses clarify what Brian presented.  And thanks again for such a careful review.

   -=- Mike

Mike Kearney
Lead Technology Manager
Mission Operations Laboratory
NASA MSFC EO-01
+1-256-544-2029

From: Kearney, Mike W. (MSFC-EO01)
Sent: Friday, November 30, 2012 11:30 AM
To: CCSDS Tech Support
Subject: FW: MORE COMMENTS on Document Review System ? Coordinator page in your Cleveland presentation (but not only)

   -=- Mike

Mike Kearney
Lead Technology Manager
Mission Operations Laboratory
NASA MSFC EO-01
+1-256-544-2029

From: Gian.Paolo.Calzolari at esa.int [mailto:Gian.Paolo.Calzolari at esa.int]
Sent: Friday, November 30, 2012 8:54 AM
To: Brian Oliver
Cc: cesg-bounces at mailman.ccsds.org; Kearney, Mike W. (MSFC-EO01)
Subject: MORE COMMENTS on Document Review System ? Coordinator page in your Cleveland presentation (but not only)

Brian,
       a few more comments thanks to Mario Merri.

We assume that the name of the  Agency Review Coordinator is configurable case by case and that delegation/addition of  an additional Review Coordinator should be easily possible within an agency

Page 11 says "Review Coordinator will click on each RID and review...." but this can be very cumbersome. A possibility for doing <send "group/all"? should be provided in an high level view (BTW see also my previous comment about decoupling the Agency Approval of a RID from the official forwarding of the RID to CCSDS).

On page 11 the Reviewer data are not visible and they should be visible.

Tools to show/print all RIDs (e.g. generate pdf Report) should be available to facilitate the task of the review coordinator and distribution.

In need Mario and I could support a telecon with you on this matter.

Regards

Gian Paolo

----- Forwarded by Gian Paolo Calzolari/esoc/ESA on 30/11/2012 15:35 -----
From:

Gian Paolo Calzolari/esoc/ESA

To:

"CCSDS Tech Support" <ccsds_techsupport at aiaa.org>

Cc:

cesg-bounces at mailman.ccsds.org, "Kearney, Mike W. (MSFC-EO01)" <Mike.Kearney at nasa.gov>

Date:

30/11/2012 12:14

Subject:

Document Review System - Coordinator page in your Cleveland presentation (but not only)




________________________________



Brian,
       I was just looking once more at your  Cleveland presentation and I ave a comment about the "Document Review System - Coordinator" page 11 in your presentation.

The Review Coordinator can see/set the Status to four values:
- Not Yet Reviewed
- Approved
- Rejected......
- Returned to Originator.......

First of all I am puzzled by the last two items looking (almost) identical. This shall be clarified
The reference to the Disposition is also puzzling me as it looks as the same term is used for the real WG Review as well as for internal Agency review and this is confusing.

You mention 3 types of coordinators
- Center Coordinators
- Agency Coordinators
- International Coordinator
I would assume that the last one is actually the WG Chair that will report the final WG decision.
In this respect I think the operations performed by the last one are different from those performed by the other two guys.

By experience, for me the  International Coordinator  will normally be able to start from a RID in state "Not Yet Reviewed" and - after WG discussion and consensus - set it to one of the following "processed" states.
- RID ACCEPTED as is  (Disposition field can be empty)
- RID ACCEPTED WITH MODIFICATIONS  (Disposition field can NOT be empty)
- RID REJECTED  (Disposition field can NOT be empty)
Normally if clarifications are needed they are discussed at WG (face2face/telecon) meetings etc.

Conversely,  the  Center/Agency Coordinator  will normally be able to start from a RID in state "Not Yet Reviewed" and - after Center/Agency discussion and consensus - set it to one of the following "processed" states.
- RID APPROVED (and TO BE sent as is to next level)
- RID APPROVED WITH MODIFICATIONS  (and TO BE sent MODIFIED to next level)
- RID REJECTED (and NOT FORWARDED  to next level)
I understand that "my" difference between accepted and approved is somehow artificial  :o)
If we want to allow capturing internal Center/Agency discussions you should have a mechanism for commenting back and forth (e.g. to ask modifications to the originator or to allow modifications to the originator). This could allow an additional state as e.g. RID UNDER DISCUSSION (was it your "Returned to Originator"?). In such a state there could be the need of exchanging also comments. I think this is more important for  Center/Agency Coordinator  and for sure such discussion should not be visible to the next level.
In general I have the impression that the Review Coordinator will be able to modify the entered RID as he likes. This is practical but it may be give issues for changes not approved by the originator. I am just puzzled, not proposing/rejecting anything, but it could be wise to give to the Coordinator "a read only view" with an edit button that would allow notifying changes to the Originator. This approach could also work for   for commenting back and forth before approving a RID.

MORE COMMENTS
Page 11 - A Review Coordinator should be able to see all the data entered by the Originator but the snapshot shows nothing about attached files.
Page 8 - Privacy Scheme. It is not clear to me whether a given Coordinator would be able to see a RID as soon as it is approved at previous level. I think that sometimes this may not be so good as it can generate some entropy. In other word I see difference between  APPROVING and FORWARDING. The approval of RIDs within a given Center/Agency can be sequential and some new RIDs could invalidate modify some previous ones. In general I think that APPROVED RIDs should remain within a Center?agency and to be forwarded only explicitly (e.g. Button at Center/Agency level to forward all approved RIDs).

I hope this input is not causing too much confusion  ;o)

Regards

Gian Paolo

This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender.

Please consider the environment before printing this email.

This message and any attachments are intended for the use of the addressee or addressees only. The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content is not permitted. If you received this message in error, please notify the sender and delete it from your system. Emails can be altered and their integrity cannot be guaranteed by the sender.



Please consider the environment before printing this email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ccsds.org/pipermail/cesg/attachments/20121205/3678c74f/attachment-0001.html

More information about the CESG mailing list