[CESG] CESG-P-2019-12-008 Approval to release CCSDS 650.0-P-2.1, Reference Model for an Open Archival Information System (OAIS) (Pink Book, Issue 2.1) for CCSDS Agency review

Tue Sep 8 19:07:10 UTC 2020

Dear CESG Members,

Conditions for approval of CCSDS 650.0-P-2.1, Reference Model for an 
Open Archival Information System (OAIS) (Pink Book, Issue 2.1) have 
been disposed to the satisfaction of the AD(s) who voted to approve 
with conditions. The Secretariat will now proceed with CMC polling to 
authorize release for Agency review.
-------------- next part --------------
From:	MOIMS-DAI <moims-dai-bounces at mailman.ccsds.org> on behalf of Shames, 
Peter M (US 312B) via MOIMS-DAI <moims-dai at mailman.ccsds.org>
Sent:	Tuesday, September 01, 2020 7:34 PM
To:	david at giaretta.org
Cc:	Shames, Peter M (US 312B); 'MOIMS-Data Archive Interoperability'
Subject:	Re: [Moims-dai] [EXTERNAL] RE: CESG conditions on your OAIS Reference 
Model, CCSDS 650.0-P-2.0
Attachments:	Untitled attachment 00007.txt

Hi David,

You are quite welcome.  I am really glad we have come to a mutually satisfactory conclusion and that 
you feel that this process resulted in significant improvements to the document.  That is precisely the 
outcome I was hoping for.

Best regards, Peter

From: David Giaretta <david at giaretta.org> 
Date: Tuesday, September 1, 2020 at 2:23 PM 
To: Peter Shames <peter.m.shames at jpl.nasa.gov> 
Cc: 'MOIMS-Data Archive Interoperability' <moims-dai at mailman.ccsds.org> 
Subject: RE: [Moims-dai] [EXTERNAL] RE: CESG conditions on your OAIS Reference 
Model, CCSDS 650.0-P-2.0

Thanks Peter, some significant improvements have been made to the document.

Ive updated the definition in the version with all the other changes you made and will send it to 
the CCSDS Editor to fix the formatting and then on to the CCSDS Management Council.

Regards

..David

From: MOIMS-DAI <moims-dai-bounces at mailman.ccsds.org> On Behalf Of Shames, Peter M 
(US 312B) via MOIMS-DAI 
Sent: 01 September 2020 17:02 
To: david at giaretta.org 
Cc: Shames, Peter M (US 312B) <peter.m.shames at jpl.nasa.gov>; 'MOIMS-Data Archive 
Interoperability' <moims-dai at mailman.ccsds.org> 
Subject: Re: [Moims-dai] [EXTERNAL] RE: CESG conditions on your OAIS Reference Model, CCSDS 
650.0-P-2.0

Dear David,

Thanks for taking the time to carefully analyze this and for bringing your definition of an OAIS 
into line with best current practice.

I can accept your revised definition.  It appears to be well aligned with what your intent is and 
also with common practice.

Proceed.

Thanks, Peter

From: David Giaretta <david at giaretta.org> 
Date: Tuesday, September 1, 2020 at 7:45 AM 
To: Peter Shames <peter.m.shames at jpl.nasa.gov> 
Cc: 'MOIMS-Data Archive Interoperability' <moims-dai at mailman.ccsds.org> 
Subject: RE: [Moims-dai] [EXTERNAL] RE: CESG conditions on your OAIS 
Reference Model, CCSDS 650.0-P-2.0

Dear Peter

Thank you for agreeing, although reluctantly, to forward the OAIS document to CMC 
leading to formal agency review.
Im providing a few clarifications that hopefully will ease your reluctance.

Yes I can confirm that DAI has accepted your other suggested changes. 

The only point of contention is the definition of an OAIS. I think the issue is that as a 
Magenta Book, OAIS provides 
       normative, controlling guidance rather than purely descriptive 
material  (CCSDS A02.1-Y-4)
and so it is not simply describing an IT system. Thanks to your other comments we have 
made the normative parts of the document much clearer.

You say that 
*         policies and procedures are strictly ancillary to the use of the system, but 
useful to the organization, 
o   But OAIS, as a Magenta Book, provides normative controlling guidance, 
and is not just a description of an IT system. The policies and procedures 
could be viewed as the equivalent of the software of an IT system.
*         Systems may store, transfer, and manipulate information, but they are not 
that information.  
o   But the system needs information (as a raw material) to help to decide 
what to do with the information it is storing, transferring, manipulating 
etc. This is surely true of all systems, whether that information is in 
database tables or configuration files etc  which are clearly separate 
from hardware and software. I will edit the Wikipedia definitions to 
include this.
*         Systems are systems 
o   But there are different types of systems and in particular not all systems 
are purely IT systems. We are defining an OAIS. Wikipedia defines many 
types of systems and we can easily add to that list. Of course there is 
already a Wikipedia article about OAIS which uses the current 
definition. We can update this in due course so that Wikipedia 
continues to be accurate.

We can eliminate the Wikipedia definition you propose of an information and computer 
science system since OAIS is not just an IT system.
The third definition you provide, from CCSDS 910.2-G-1, is for what is termed a real 
system, rather than a general definition. That GB does not provide a general definition 
of system. 

The second definition, which originated in one of your SEA documents, is much closer to 
what we intended.  I believe that our definition comes very close to that definition as 
can be seen in the analysis below, comparing to the proposed definition of an OAIS 
which is below the table:

CCSDS 311.0-M-1
OAIS definition below
A set of elements
yes
(people, 
yes  staff of the organisation
products [hardware and software],
yes- hardware and software
facilities, 
Implicit in hardware and software
equipment, 
Implicit in hardware
material, and 
yes - Information as a raw material
processes [automated as well as manual procedures])
yes  processes and procedures
that are related and 
yes-  since we are describing an 
OAIS
whose behavior satisfies customer and/or operational 
needs.
yes  to preserve information

Going back to the definition you proposed on 18 August and making updates in red to 
be
1)      consistent with CCSDS 311.0-M-1  see table above  to include people, 
processes, procedures and material (in the form of information)
2)      consistent with the definition you provide of Policy  A policy is a statement of 
intent, and is implemented as a procedure
3)      correct the reference to the mandatory responsibilities and make it a NOTE

we have

Open Archival Information System (OAIS): An Archive system consisting of 
hardware, software, information and policy based processes and procedures, 
developed put in place and operated by an organization and its staff.  The 
organization has accepted the responsibility to preserve information and make 
it available for a Designated Community. The organisation may be part of a 
larger organisation. The system meets a set of mandatory responsibilities, as 
defined in section 4, that allows an OAIS Archive to be distinguished from other 
uses of the term archive. The term Open in OAIS is used to imply that this 
Recommended Practice and future related Recommended Practices and 
standards are developed in open forums, and it does not imply that access to 
the Archive is unrestricted.
NOTE -  The set of mandatory responsibilities an OAIS Archive must 
perform are defined in section 3.2

The role of the organisation, procedures etc are prominent because they are important, 
although, as the table shows, this is entirely consistent with the CCSDS Glossary 
definition you propose. 

If this is OK with you then we will update the MB.

..David

From: MOIMS-DAI <moims-dai-bounces at mailman.ccsds.org> On Behalf Of Shames, 
Peter M (US 312B) via MOIMS-DAI 
Sent: 27 August 2020 00:34 
To: david at giaretta.org 
Cc: Shames, Peter M (US 312B) <peter.m.shames at jpl.nasa.gov>; 'MOIMS-Data Archive 
Interoperability' <moims-dai at mailman.ccsds.org> 
Subject: Re: [Moims-dai] [EXTERNAL] RE: CESG conditions on your OAIS Reference 
Model, CCSDS 650.0-P-2.0

Dear David, et al,

In general, while I could accept what you propose I would do it rather reluctantly, and 
for the following reasons:

1)      In Wikipedia, that font of all knowledge, system, in information and computer 
science, is define like this:  "system is a hardware system, software system, or 
combination, which has components as its structure and observable inter-
process communications as its behavior."
2)      In the CCSDS glossary, that font of our corporate knowledge, system is defined 
like this: (CCSDS 311.0-M-1) " A set of elements (people, products [hardware 
and software], facilities, equipment, material, and processes [automated as well 
as manual procedures]) that are related and whose behavior satisfies customer 
and/or operational needs".
3)      Or like this: (CCSDS 910.2-G-1) " A set of one or more computers, the associated 
software, peripherals, terminals, human operators, physical processes, 
information transfer means, etc., that forms an autonomous whole capable of 
performing information processing and/or information transfer."

It is only these OAIS definitions that stray so far from the norm as to attempt to define 
this particular kind of "system" as being first and foremost about "organizations", as in 
the original definition, or now, about "policies, procedures, hardware, software, and 
information."  Systems are systems.  Policies are organizational guidance about how 
organizations carry out their tasks.  Procedures are instructions about the steps 
organizations are to follow in carrying out their tasks.  Neither of these are the primary 
elements of a system, they are strictly ancillary to the use of the system, but useful to 
the organization.  Likewise information, for that matter.  Systems may store, transfer, 
and manipulate information, but they are not that information.

Policy (Wikpedia):  A policy is a deliberate system of principles to guide 
decisions and achieve rational outcomes. A policy is a statement of intent, and is 
implemented as a procedure or protocol. Policies are generally adopted by a 
governance body within an organization.

Procedure (Wikipedia):   a set of step-by-step instructions compiled by an 
organization to help workers carry out complex routine operations. 

As an instance of a particular kind of SYSTEM, please adopt a straightforward definition 
of OAIS that does not convolve these other, separate, terms into the definition nor 
make them the prominent features of the definition.  

It's a quibble, but the addition of that phrase "but is not limited to" seems vague and to 
add no value.  It's rather like sticking ", etc" at the end of a sentence when you cannot 
summon up another concrete item to add to a list.  But in fact, placed as it is, it seems to 
take away meaning from the very concrete definition that should follow it.  Please 
remove it.

I do not have any issue with the use of "put in place" (or "acquired", or "employed") nor 
the reference to "staff", in the last sentence.

Is it safe to assume that the other changes I proposed have been adopted as proposed 
or are there changes to these as well?  Your note was unclear.

Thanks, Peter

From: David Giaretta <david at giaretta.org> 
Date: Wednesday, August 26, 2020 at 4:22 AM 
To: Peter Shames <peter.m.shames at jpl.nasa.gov> 
Cc: 'MOIMS-Data Archive Interoperability' <moims-
dai at mailman.ccsds.org> 
Subject: RE: [Moims-dai] [EXTERNAL] RE: CESG conditions on your OAIS 
Reference Model, CCSDS 650.0-P-2.0

Dear Peter

The DAI WG reached consensus at yesterdays meeting to accept your proposals 
but with a few changes to your proposed definition of OAIS Archive which we 
think is a bit clearer.

The following shows our changes, with your text as the basis. A few words of 
your text have been deleted (double strikeout) and additional words inserted 
(red text).

Some points to explain the changes:
1)      Add policies and information
2)      To make it clear that the Archive organisation does not have to develop 
the system itself we use the words put in place
3)      In order to clean up the Glossary definition we move the the reference 
to the section which specifies the mandatory responsibilities to a NOTE. 

Open Archival Information System (OAIS): An Archive system consistsing of, but 
is not limited to, policies, procedures, hardware, software, and information. 
procedures The Archive system is developed put in place and operated by an 
organization and its staff.  The organization has accepted the responsibility to 
preserve information and make it available for a Designated Community. The 
organisation may be part of a larger organisation. The system An OAIS Archive 
meets a set of mandatory responsibilities, as defined in section 4, that allows an 
OAIS Archive to be distinguished from other uses of the term archive. The term 
Open in OAIS is used to imply that this Recommended Practice and future 
related Recommended Practices and standards are developed in open forums, 
and it does not imply that access to the Archive is unrestricted.
NOTE -  The set of mandatory responsibilities an OAIS Archive must 
perform are defined in section 3.2

I hope you concur with this. We can then go forward with this improved version 
of OAIS. Once again I want to thank you for suggesting many improvements.

Regards

..David

From: MOIMS-DAI <moims-dai-bounces at mailman.ccsds.org> On Behalf Of 
Shames, Peter M (US 312B) via MOIMS-DAI 
Sent: 18 August 2020 18:56 
To: david at giaretta.org 
Cc: Shames, Peter M (US 312B) <peter.m.shames at jpl.nasa.gov>; MOIMS-DAI 
List <moims-dai at mailman.ccsds.org> 
Subject: Re: [Moims-dai] [EXTERNAL] RE: CESG conditions on your OAIS 
Reference Model, CCSDS 650.0-P-2.0

Dear David, et al,

I'd like to thank you and your team for taking my suggestions seriously and for 
working diligently to improve the clarity of the document.  I am really glad that 
you took this opportunity to make these long overdue changes.  I suspect that 
your community will also appreciate the changes.

I have read the document (a quick skim this time), and find that I agree with all 
of the changes that you have made.  The added clarity in use of terms and 
diagrams helps a lot.  I did find a few relatively minor items that I still wish to ask 
that you fix.  Instead of doing the usual "change this to that" PID format I just 
made the proposed changes directly in the text, which I have attached with 
Track Changes turned on.  For transparency, I will tell you where to look:

Sec 1.1: fixed the definition of an OAIS to align with the usual notion of what a 
"system" is:

An OAIS is an Archive system consisting of hardware, software, and 
procedures developed and operated by an organization.  The 
organization has accepted the responsibility to preserve information 
and make it available for a Designated Community. The organization 
may be part of a larger organization.

Made the same change in sec 1.6.2 Terminology.

In other sections and sub-sections I added a clarifying sentences in the intro 
paragraphs, of the general form: "This sub-section (or this entire section) is non-
normative.".  I am concerned that without this sort of clarity the mix of 
normative and non-normative materials in various sections can cause 
confusion.  I did not want it to be too intrusive, but I think we must be clear. 

To take the "sting" out of this, in some sections I added an explanatory 
sentence, like this one in Sec 4.2:

This entire subsection is informative, but provides useful abstractions 
for understanding how an OAIS might be designed.

I hope that you and your team can accept these changes.  If the answer is yes 
you have my concurrence to go forward.

Thanks, Peter

From: David Giaretta <david at giaretta.org> 
Date: Tuesday, August 18, 2020 at 9:07 AM 
To: Peter Shames <peter.m.shames at jpl.nasa.gov> 
Cc: MOIMS-DAI List <moims-dai at mailman.ccsds.org> 
Subject: Re: [Moims-dai] [EXTERNAL] RE: CESG conditions on your 
OAIS Reference Model, CCSDS 650.0-P-2.0

Dear Peter

Steve Hughes provided us with feedback following your discussion with 
him. As a result we have taken an in-depth look at the conformance 
sections, the definitions and discussion of Representation Information, 
and in particular Semantic Information, and the discussion of 
Representation Information Networks. I believe we have made 
significant improvements to OAIS as a result. Some of the  text dates 
back to the original version of OAIS; it is only thanks to your insistence 
that we were forced to recognize a number of weaknesses and sources 
of confusion for readers.

The updated document is available at 
https://cwe.ccsds.org/moims/docs/MOIMS-
DAI/Draft%20Documents/OAIS%20v3/650x0020_CESG_Aproval_Mod-
20200818.doc?Web=0

A short list of the changes are:
-          The conformance section 1.4 has been clarified as you 
suggested and the individual sections are clearly marked as 
normative
-          We checked the use of shall/must. One instance of must was 
missing from a section marked as normative. We found we did 
have some uses of must in informative sections, which we re-
phrased. 
-          Changed the name of Semantic Information to Semantic 
Representation Information to avoid confusion with Semantic 
networks etc.  The definition has been clarified to 
o   Semantic Representation Information: The 
Representation Information that further describes the 
meaning of the Data Object, and its parts or elements, 
beyond that provided by the Structure Representation 
Information.
-          There is a similar change for Structure Information to 
Structure Representation Information, which is now defined 
as 
o   Structure Representation Information: The 
Representation Information that imparts information 
about the arrangement of and the organization of the 
parts or elements of the Data Object
-          The explanation of Representation Information Networks has 
been updated, with some new diagrams which we hope make 
the concept very clear. 

We hope that clears the way for progressing OAIS to the next stage.

Regards

..David