[Moims-dai] FW: LTDP document

Rosemarie.Leone at esa.int Rosemarie.Leone at esa.int
Fri Jan 15 08:44:31 UTC 2016

Dear all,

sorry for the late reply, I got confused by the spam too. Hope you receive 
this e-mail. 

In reply to John, you will find at the following address all the 
guidelines elaborated by the LTDP working group during the last  years and 
approved by CEOS. Please let me know if you have issue accessing the 
documents. I am available to present the guidelines and best practices and 
scope at the next webex. 


Regards, Rosemarie

ESA - European Space Agency 
Rosemarie Leone | ESA Information Technology Department
ESOC | Robert-Bosch-Strasse 5, 64293 Darmstadt, Germany
Tel +49 6151 902854 | M. +49 (0) 152 01801844
rosemarie.leone at esa.int | www.esa.int

From:   "John Garrett" <garrett at his.com>
To:     "'MOIMS-Data Archive Ingestion'" <moims-dai at mailman.ccsds.org>
Date:   14/01/2016 06:34
Subject:        [Moims-dai] FW: LTDP document
Sent by:        moims-dai-bounces at mailman.ccsds.org

Thanks to Brian  who has taken a look at why our messages aren?t making it 
through the DAI mailing list.  Apparently I?m sending spam ;^).
Still don?t know why our messages are getting flagged, but it does explain 
why we often don?t get responses.
Hopefully adding all of us to the whitelist will help.
And hopefully we won?t end up getting spam that is made to appear as if it 
was coming from one of us.
For those of you who didn?t get my spam, let me know and I?ll send it 
directly to you.
Wishing you Prosperity and Peace,
From: Brian Oliver [mailto:BrianO at aiaa.org] 
Sent: Tuesday, January 12, 2016 4:36 PM
To: John Garrett <garrett at his.com>
Subject: Re: LTDP document
By looking in the logs on the SPAM filter it appears that some of these 
messages sent to the mailing list were flagged as SPAM.  In order to 
mitigate this from happening again?I have taken the liberty to add all 60+ 
e-mail addresses that are subscribed to the MOIMS-DAI list to the SPAM 
Filter ?WhiteList?.  This is probably ?overkill?, but it should correct 
any issues going forward.
On Jan 12, 2016, at 2:05 PM, John Garrett <garrett at his.com> wrote:
Hi Brian,
We?re having problems with the MOIMS-DAI list again.
Messages are not going through and the sender is not being told that the 
messages are not going through.   There were several messages, including 
ones from David Giaretta and from me, that did not make it through.  I?ve 
forwarded one that I sent.  Please let us know if there is some problem 
with the mailing lists.
Wishing You Prosperity and Peace,
From: John Garrett [mailto:garrett at his.com] 
Sent: Tuesday, January 12, 2016 12:09 AM
To: 'MOIMS-Data Archive Ingestion' <moims-dai at mailman.ccsds.org>; 'David 
Giaretta' <david at giaretta.org>; 'Boucon Daniele' <Daniele.Boucon at cnes.fr>; 
'Mike Martin' <tahoe_mike at sbcglobal.net>; 'D or C Sawyer' <Sawyer at acm.org
>; 'Mark Conrad' <mark.conrad at NARA.GOV>; 'Robert Downs' <
rdowns at ciesin.columbia.edu>
Subject: LTDP document
Yes, I think the current version is still very project oriented.   The 
intention of at least some of the participants was to make it more 
general, but the source document was from EU Earth Observation projects. 
I?ll send what I think is the most current version of their document 
(Rosemarie can let us know if there is a more recent document).  I believe 
this version was also adopted by one of the working groups from the 
international Committee on Earth Observation (CEOS).
The approved CCSDS Project was standardize that document (see project 
description below).
The purpose of this recommendation is to provide a standard method 
structured as a complete process to formally define the steps and 
associated activities required to preserve digital information objects. 
The process thus defined along with the activities, is linked with the 
data lifecycle. This activity will work to standardize materials fed into 
the process by the EU Project - Long Term Digital Preservation (for Earth 
Science Data). It is likely that participants in EU Project will also 
participate in the CCSDS efforts on behalf of their Agencies.
But as you?ve seen our current document has changed quite a bit.  We need 
to decide where we are going with the document going forward.
Wishing you Prosperity and Peace,
From: moims-dai-bounces at mailman.ccsds.org [
mailto:moims-dai-bounces at mailman.ccsds.org] On Behalf Of Mark Conrad
Sent: Thursday, January 7, 2016 5:02 PM
To: MOIMS-Data Archive Ingestion <moims-dai at mailman.ccsds.org>; David 
Giaretta <david at giaretta.org>
Subject: Re: [Moims-dai] NASA Guidance (Records Schedule) for 
Project/Program Files
Hi Mike,
I think it would actually be more useful (and a whole lot easier) to 
simply make it clear that the information lifecycle described in the 
document is for a specific context (i.e., project/experiment). If you try 
to re-write the document for a more generic lifecycle it would be very 
difficult. The current document is far too prescriptive in terms of 
workflow/responsibilities for all of the different contexts that 
records/information/data are created under. 
As I said before, I think the document would be very useful for the 
specific context. I just think the document scope should be qualified to 
indicate the context in which it can be applied.

Mark Conrad
NARA Information Services/Applied Research
The National Archives and Records Administration
Erma Ora Byrd Conference and Learning Center
Building 494 Second Floor
610 State Route 956
Rocket Center, WV  26726

Phone: 304-726-7820
Fax: 304-726-7802
Email: mark.conrad at nara.gov 
Twitter: @lmc1990
On Thu, Jan 7, 2016 at 4:12 PM, Mike Martin <tahoe_mike at sbcglobal.net> 
Hi Mark

Thanks for your comments.  In the "much more generic lifecycle framework" 
would all the topics still apply?  If so, then maybe the paper can be 
worded to be more inclusive and to make sure that an individual could see 
that he/she was the "project", and that sometimes the "sponsor" would be 
one's boss or oneself.

Thanks, Mike

On 1/7/2016 9:58 AM, Mark Conrad wrote:
Hi Mike,

I am also the one responsible for generating the action item from the
December 22nd meeting as well. As an archivist I am used to a much more
generic information lifecycle framework. Archivists and records managers
use more generic frameworks because we have to deal with
records/information/data that are created in many different contexts.
For example, records/information/data are created in many organizations
on a daily basis in contexts that don't have someone in a formal role of
sponsor. Records/information/data are also generated outside the context
of a particular project.

I guess my main objection was that the title of Information Lifecycle
Framework was not sufficiently qualified to distinguish it from more
generic frameworks like those used by archivists and records managers.
The document as it currently exists could be entitled something like,
Information Lifecycle Framework for Major Projects/Experiments.

I think the document would be very useful in this qualified context.
Many archivist or records managers can tell you horror stories about
receiving calls like, "We have shut down this experiment/project/system,
do you want any of the information." The archivist ends up doing "data
archaeology" trying to see what can be salvaged. Having information
reuse considered from the initiation of a project would make our lives
so much easier - not to mention making the results of the work
accessible and usable to a much wider audience.

Hope this helps explain where my comments come from.


Mark Conrad
NARA Information Services/Applied Research
The National Archives and Records Administration
Erma Ora Byrd Conference and Learning Center
Building 494 Second Floor
610 State Route 956
Rocket Center, WV  26726

Phone: 304-726-7820
Fax: 304-726-7802
Email: mark.conrad at nara.gov <mailto:mark.conrad at nara.gov>
Twitter: @lmc1990

On Tue, Jan 5, 2016 at 6:01 PM, Mike Martin <tahoe_mike at sbcglobal.net
<mailto:tahoe_mike at sbcglobal.net>> wrote:

    Hi Mark and others

    On 1/5/2016 11:52 AM, Mark Conrad wrote:

        Second, the schedule identifies 8 stages of a project - 
        Approval, Design Development, Manufacture, Fabrication and
        Assembly, Pre-launch System Integration and Verification,
        and Operations, Observational Data, and Evaluation and 

    Related to this, there was an action item from the meeting on the
    22nd of Dec:

    Action: clarify why we need another lifecycle

    I spent many hours going through all the lifecycles in:


    and looking at other archiving documents provided a summary in late
    2014 for the DAI group which is included below.

    Most lifecycles don't really consider the interactions of the three
    participants (sponsor/project/archive).  I wanted our lifecycle to
    point out the importance of the sponsor and archive being involved
    in the initiation of the project and then to point out the need for
    bringing in requirements and tools to the specify and design
    stages.  The Exploitation activities aren't covered in most
    lifecycles.  I didn't think that all the themes in the LTDP (PDSC
    definition and appraisal, archive operation and organization,
    security, ingestion, maintenance, access and interoperability,
    exploitation and reprocessing, purge prevention) were applicable to
    this document so came up with a shorter list of activities.

    Another thing to mention, the topics/issues came from a list David
    provided from his work on the Active Data Management Plan, plus
    evaluation of all the LTDP Common Guidelines, plus evaluation of all
    the activities in the PAIMAS standard, plus looking at the ESDIS
    Earth Science Content Specification, plus other issues that group
    members raised.

    Nov 20, 2014
    Hi Everyone

    I've gone through all the reference documents we have seen and the
    articles in our bibliography and tried to summarize the unique life
    cycles that are presented.  Here are some summaries with more
    details below:

    David's:  Planning and Creation Stage->Consolidation Stage->Long
    Term Preservation Stage->Adding Value, Re-Use and Sustainability

    LTDP:    Consolidation->Implementation->Operations

    OAIS+:  Planning->Collection->Analysis->Packaging->Ingest->Data
    Management->Archival Storage->Access->Preservation Planning

    DCC:     Conceptualize->create or receive->appraise and
    select->ingest->preservation action-> store->access->use and

    USGS:   Plan->Acquire->Process->Analyze->Preserve->Publish/Share

    SDMW:  Plan->Collect->Integrate and transform->Publish->Discover and
    inform->Archive or discard


    DMF:     Planning and Production->Data Management Activities
    ->Dissemination->Usage Activities

    Can we come up with an optimal set of categories based on all these
    various views?

    Thanks, Mike

    More detail from the various documents:

    1.  The LTDP preservation workflow includes:
    Initialization (appraisal, define designated community,
    specification of preservation/curation requirements, consolidation
    procedure, tailoring content, consult with community, cost and risk
    Consolidation (implement consolidation, gather missing content and
    update), Implementation (data ingestion and catalog generation,
    Operations (operations and maintenance,  curation and stewardship -
    adding value).

    2.  The OAIS model includes Ingest, Data Management, Archival
    Storage, Access, Management and Preservation Planning.   It is
    missing Planning (meaning enterprise planning), Collecting (Mission
    Operations, building and running the enterprise), Analyzing
    (producing knowledge) and maybe Packaging.  All these occur prior to
    OAIS, but OAIS should be involved.  Consolidation could be part of
    Ingest or possibly an separate activity outside the OAIS.  Adding
    Value could be part of or a combination of Preservation Planning or
    Access.   This model syncs up with RASIM which builds advanced
    information management objects in terms of five services which
    correlate with OAIS components, archive service (ingest), repository
    service (archival storage), registry service (data management),
    product service (access plus archival storage), and query service
    (access plus data management).

    3.  The Data Curation Centre life cycle includes conceptualize,
    create or receive, appraise and select (with potential to dispose),
    ingest, preservation action (migrate or reappraise), store, access,
    use and reuse, transform (with potential to migrate).

    4.  The NOAA Environmental Data Life Cycle Functions include
    planning new systems, then stewardship which includes observing
    operations, archive,  access, use.   Overarching themes are
    governance, requirements management, architecture management,
    security; developing rich metadata; and mechanisms for user and
    requirements and feedback.   Each of the major categories has many

    5.  The Global Change Science Requirements for Long-Term Archiving
    Workshop (USGCRP) identified the following components:  User
    Involvement, Data Administration, Documentation, Data Ingest and
    Data Preservation and Maintenance, Data Processing/Reprocessing,
    Data Access and User Support.

    6.  The USGS Life Cycle includes Plan, Acquire, Process, Analyze,
    Preserve, Publish/Share with three activities running through all
    phases: Describe (Metadata and Documentation), Manage Quality,
    Backup and Secure.

    7.  The ESA Heterogenous Missions Accessibility Report really
    focuses on data access and not the other phases.

    8.  The Harnessing the Power of Digital Data: Taking the Next Step,
    Science Data Management Workshop report provides a number of models:

    FGDC life cycle:  Define, Inventory/Evaluate, Obtain, Access,
    Maintain, Use/Evaluate, Archive.
    Linear data lifecycle: Plan, Collect, Integrate and Transform,
    Publish, Discovery with two activities running through all phases,
    Governance and Stewardship and Communications.
    Basic science model: plan, collect, integrate and transform,
    publish, discover and inform, archive or discard.

    The topics that are identified in the report include:  data
    governance, stewardship, sharing, access, security, version control,
    metadata management, content and format, document and content
    management, preservation, transfer of responsibility, data
    architecture, database operations management, reference and master
    data management, data warehousing and business intelligence, data
    quality management, provenance, usability, value added services,
    workflow systems.

    9.  The LPDAAC Lifecycle Plan identifies the phases:  Inception,
    Active Archive, Long-Term Archive which each have four elements,
    characterization, critical data and information, applicable
    standards, transition.

    The WBS is broken into phases, inception-planning (embed in producer
    team, provide data management plan), inception-production (laison to
    science stakeholders, collection inception checklist, support
    production, repeat experiment, determine approach to tools/services,
    authorize to migrate, provide NASA data template), active archive
    transition from producer (obtain authorization to migrate, plan
    migration, install new product line, migrate, advertise new
    products, assume primary access and discovery role), active archive
    transition to long-term (obtain authorization to migrate, plan
    migration), long term archive transition to long-term (enable
    migration, execute migration, advertise new products, transfer
    primary access and discovery role, obtain authorization for
    certification, sunset products).

    10.  DataOne includes Collect, Assure, Describe, Deposit, Preserve,
    Discover, Integrate, Analyze

    11. Jeff de La Beaujardičre's Data Management Framework
    Planning and Production (Requirements Definition, Planning,
    Development, Deployment, Operations);
    Data Management Activities (Collection, Processing, Quality Control,
    Documentation, Dissemination, Cataloging, Preservation, Stewardship,
    Usage Tracking, Final Disposition);
    Usage Activities (Discovery, Reception, Analysis, Product
    Generation, User Feedback, Citation, Tagging, Gap Analysis).

    Moims-dai mailing list
    Moims-dai at mailman.ccsds.org <mailto:Moims-dai at mailman.ccsds.org>

Moims-dai mailing list
Moims-dai at mailman.ccsds.org

Moims-dai mailing list
Moims-dai at mailman.ccsds.org
<Preservation Workflow_v1.0.pdf>
Moims-dai mailing list
Moims-dai at mailman.ccsds.org

This message and any attachments are intended for the use of the addressee or addressees only.
The unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its
content is not permitted.
If you received this message in error, please notify the sender and delete it from your system.
Emails can be altered and their integrity cannot be guaranteed by the sender.

Please consider the environment before printing this email.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/moims-dai/attachments/20160115/1edb2285/attachment.html>

More information about the MOIMS-DAI mailing list