[Moims-dai] FW: LTDP document
John Garrett
garrett at his.com
Thu Jan 14 05:29:21 UTC 2016
Hi,
Thanks to Brian who has taken a look at why our messages aren’t making it through the DAI mailing list. Apparently I’m sending spam ;^).
Still don’t know why our messages are getting flagged, but it does explain why we often don’t get responses.
Hopefully adding all of us to the whitelist will help.
And hopefully we won’t end up getting spam that is made to appear as if it was coming from one of us.
For those of you who didn’t get my spam, let me know and I’ll send it directly to you.
Wishing you Prosperity and Peace,
-JOhn
From: Brian Oliver [mailto:BrianO at aiaa.org]
Sent: Tuesday, January 12, 2016 4:36 PM
To: John Garrett <garrett at his.com>
Subject: Re: LTDP document
John,
By looking in the logs on the SPAM filter it appears that some of these messages sent to the mailing list were flagged as SPAM. In order to mitigate this from happening again…I have taken the liberty to add all 60+ e-mail addresses that are subscribed to the MOIMS-DAI list to the SPAM Filter “WhiteList”. This is probably “overkill”, but it should correct any issues going forward.
Thanks,
Brian
On Jan 12, 2016, at 2:05 PM, John Garrett <garrett at his.com <mailto:garrett at his.com> > wrote:
Hi Brian,
We’re having problems with the MOIMS-DAI list again.
Messages are not going through and the sender is not being told that the messages are not going through. There were several messages, including ones from David Giaretta and from me, that did not make it through. I’ve forwarded one that I sent. Please let us know if there is some problem with the mailing lists.
Wishing You Prosperity and Peace,
-JOhn
From: John Garrett [ <mailto:garrett at his.com> mailto:garrett at his.com]
Sent: Tuesday, January 12, 2016 12:09 AM
To: 'MOIMS-Data Archive Ingestion' < <mailto:moims-dai at mailman.ccsds.org> moims-dai at mailman.ccsds.org>; 'David Giaretta' < <mailto:david at giaretta.org> david at giaretta.org>; 'Boucon Daniele' < <mailto:Daniele.Boucon at cnes.fr> Daniele.Boucon at cnes.fr>; 'Mike Martin' < <mailto:tahoe_mike at sbcglobal.net> tahoe_mike at sbcglobal.net>; 'D or C Sawyer' < <mailto:Sawyer at acm.org> Sawyer at acm.org>; 'Mark Conrad' < <mailto:mark.conrad at nara.gov> mark.conrad at NARA.GOV>; 'Robert Downs' < <mailto:rdowns at ciesin.columbia.edu> rdowns at ciesin.columbia.edu>
Subject: LTDP document
Hi,
Yes, I think the current version is still very project oriented. The intention of at least some of the participants was to make it more general, but the source document was from EU Earth Observation projects. I’ll send what I think is the most current version of their document (Rosemarie can let us know if there is a more recent document). I believe this version was also adopted by one of the working groups from the international Committee on Earth Observation (CEOS).
The approved CCSDS Project was standardize that document (see project description below).
The purpose of this recommendation is to provide a standard method structured as a complete process to formally define the steps and associated activities required to preserve digital information objects. The process thus defined along with the activities, is linked with the data lifecycle. This activity will work to standardize materials fed into the process by the EU Project - Long Term Digital Preservation (for Earth Science Data). It is likely that participants in EU Project will also participate in the CCSDS efforts on behalf of their Agencies.
But as you’ve seen our current document has changed quite a bit. We need to decide where we are going with the document going forward.
Wishing you Prosperity and Peace,
-JOhn
From: <mailto:moims-dai-bounces at mailman.ccsds.org> moims-dai-bounces at mailman.ccsds.org [ <mailto:moims-dai-bounces at mailman.ccsds.org> mailto:moims-dai-bounces at mailman.ccsds.org] On Behalf Of Mark Conrad
Sent: Thursday, January 7, 2016 5:02 PM
To: MOIMS-Data Archive Ingestion < <mailto:moims-dai at mailman.ccsds.org> moims-dai at mailman.ccsds.org>; David Giaretta < <mailto:david at giaretta.org> david at giaretta.org>
Subject: Re: [Moims-dai] NASA Guidance (Records Schedule) for Project/Program Files
Hi Mike,
I think it would actually be more useful (and a whole lot easier) to simply make it clear that the information lifecycle described in the document is for a specific context (i.e., project/experiment). If you try to re-write the document for a more generic lifecycle it would be very difficult. The current document is far too prescriptive in terms of workflow/responsibilities for all of the different contexts that records/information/data are created under.
As I said before, I think the document would be very useful for the specific context. I just think the document scope should be qualified to indicate the context in which it can be applied.
Mark
Mark Conrad
NARA Information Services/Applied Research
IXA
The National Archives and Records Administration
Erma Ora Byrd Conference and Learning Center
Building 494 Second Floor
610 State Route 956
Rocket Center, WV 26726
Phone: 304-726-7820
Fax: 304-726-7802
Email: <mailto:mark.conrad at nara.gov> mark.conrad at nara.gov
<http://www.facebook.com/NARACAST> http://www.facebook.com/NARACAST
<http://www.archives.gov/applied-research/> http://www.archives.gov/applied-research/
Twitter: @lmc1990
On Thu, Jan 7, 2016 at 4:12 PM, Mike Martin < <mailto:tahoe_mike at sbcglobal.net> tahoe_mike at sbcglobal.net> wrote:
Hi Mark
Thanks for your comments. In the "much more generic lifecycle framework" would all the topics still apply? If so, then maybe the paper can be worded to be more inclusive and to make sure that an individual could see that he/she was the "project", and that sometimes the "sponsor" would be one's boss or oneself.
Thanks, Mike
On 1/7/2016 9:58 AM, Mark Conrad wrote:
Hi Mike,
I am also the one responsible for generating the action item from the
December 22nd meeting as well. As an archivist I am used to a much more
generic information lifecycle framework. Archivists and records managers
use more generic frameworks because we have to deal with
records/information/data that are created in many different contexts.
For example, records/information/data are created in many organizations
on a daily basis in contexts that don't have someone in a formal role of
sponsor. Records/information/data are also generated outside the context
of a particular project.
I guess my main objection was that the title of Information Lifecycle
Framework was not sufficiently qualified to distinguish it from more
generic frameworks like those used by archivists and records managers.
The document as it currently exists could be entitled something like,
Information Lifecycle Framework for Major Projects/Experiments.
I think the document would be very useful in this qualified context.
Many archivist or records managers can tell you horror stories about
receiving calls like, "We have shut down this experiment/project/system,
do you want any of the information." The archivist ends up doing "data
archaeology" trying to see what can be salvaged. Having information
reuse considered from the initiation of a project would make our lives
so much easier - not to mention making the results of the work
accessible and usable to a much wider audience.
Hope this helps explain where my comments come from.
Mark
Mark Conrad
NARA Information Services/Applied Research
IXA
The National Archives and Records Administration
Erma Ora Byrd Conference and Learning Center
Building 494 Second Floor
610 State Route 956
Rocket Center, WV 26726
Phone: <tel:304-726-7820> 304-726-7820
Fax: <tel:304-726-7802> 304-726-7802
Email: <mailto:mark.conrad at nara.gov> mark.conrad at nara.gov <mailto: <mailto:mark.conrad at nara.gov> mark.conrad at nara.gov>
<http://www.facebook.com/NARACAST> http://www.facebook.com/NARACAST
<http://www.archives.gov/applied-research/> http://www.archives.gov/applied-research/< <http://www.archives.gov/ncast/> http://www.archives.gov/ncast/>
Twitter: @lmc1990
On Tue, Jan 5, 2016 at 6:01 PM, Mike Martin < <mailto:tahoe_mike at sbcglobal.net> tahoe_mike at sbcglobal.net
<mailto: <mailto:tahoe_mike at sbcglobal.net> tahoe_mike at sbcglobal.net>> wrote:
Hi Mark and others
On 1/5/2016 11:52 AM, Mark Conrad wrote:
Second, the schedule identifies 8 stages of a project - Formulation,
Approval, Design Development, Manufacture, Fabrication and
Assembly, Pre-launch System Integration and Verification,
Implementation
and Operations, Observational Data, and Evaluation and Termination.
Related to this, there was an action item from the meeting on the
22nd of Dec:
Action: clarify why we need another lifecycle
I spent many hours going through all the lifecycles in:
<http://www.pnamp.org/sites/default/files/data_life_cycle_models_and_concepts.pdf> http://www.pnamp.org/sites/default/files/data_life_cycle_models_and_concepts.pdf
and looking at other archiving documents provided a summary in late
2014 for the DAI group which is included below.
Most lifecycles don't really consider the interactions of the three
participants (sponsor/project/archive). I wanted our lifecycle to
point out the importance of the sponsor and archive being involved
in the initiation of the project and then to point out the need for
bringing in requirements and tools to the specify and design
stages. The Exploitation activities aren't covered in most
lifecycles. I didn't think that all the themes in the LTDP (PDSC
definition and appraisal, archive operation and organization,
security, ingestion, maintenance, access and interoperability,
exploitation and reprocessing, purge prevention) were applicable to
this document so came up with a shorter list of activities.
Another thing to mention, the topics/issues came from a list David
provided from his work on the Active Data Management Plan, plus
evaluation of all the LTDP Common Guidelines, plus evaluation of all
the activities in the PAIMAS standard, plus looking at the ESDIS
Earth Science Content Specification, plus other issues that group
members raised.
-------------------------------------------------------------------
Nov 20, 2014
Hi Everyone
I've gone through all the reference documents we have seen and the
articles in our bibliography and tried to summarize the unique life
cycles that are presented. Here are some summaries with more
details below:
David's: Planning and Creation Stage->Consolidation Stage->Long
Term Preservation Stage->Adding Value, Re-Use and Sustainability
LTDP: Consolidation->Implementation->Operations
OAIS+: Planning->Collection->Analysis->Packaging->Ingest->Data
Management->Archival Storage->Access->Preservation Planning
DCC: Conceptualize->create or receive->appraise and
select->ingest->preservation action-> store->access->use and
reuse->transform
USGS: Plan->Acquire->Process->Analyze->Preserve->Publish/Share
SDMW: Plan->Collect->Integrate and transform->Publish->Discover and
inform->Archive or discard
DataOne;
Collect->Assure->Describe->Deposit->Preserve->Discover->Integrate
->Analyze
DMF: Planning and Production->Data Management Activities
->Dissemination->Usage Activities
Can we come up with an optimal set of categories based on all these
various views?
Thanks, Mike
More detail from the various documents:
1. The LTDP preservation workflow includes:
Initialization (appraisal, define designated community,
specification of preservation/curation requirements, consolidation
procedure, tailoring content, consult with community, cost and risk
assessment),
Consolidation (implement consolidation, gather missing content and
update), Implementation (data ingestion and catalog generation,
dissemination),
Operations (operations and maintenance, curation and stewardship -
adding value).
2. The OAIS model includes Ingest, Data Management, Archival
Storage, Access, Management and Preservation Planning. It is
missing Planning (meaning enterprise planning), Collecting (Mission
Operations, building and running the enterprise), Analyzing
(producing knowledge) and maybe Packaging. All these occur prior to
OAIS, but OAIS should be involved. Consolidation could be part of
Ingest or possibly an separate activity outside the OAIS. Adding
Value could be part of or a combination of Preservation Planning or
Access. This model syncs up with RASIM which builds advanced
information management objects in terms of five services which
correlate with OAIS components, archive service (ingest), repository
service (archival storage), registry service (data management),
product service (access plus archival storage), and query service
(access plus data management).
3. The Data Curation Centre life cycle includes conceptualize,
create or receive, appraise and select (with potential to dispose),
ingest, preservation action (migrate or reappraise), store, access,
use and reuse, transform (with potential to migrate).
4. The NOAA Environmental Data Life Cycle Functions include
planning new systems, then stewardship which includes observing
operations, archive, access, use. Overarching themes are
governance, requirements management, architecture management,
security; developing rich metadata; and mechanisms for user and
requirements and feedback. Each of the major categories has many
sub-activities.
5. The Global Change Science Requirements for Long-Term Archiving
Workshop (USGCRP) identified the following components: User
Involvement, Data Administration, Documentation, Data Ingest and
Verification
Data Preservation and Maintenance, Data Processing/Reprocessing,
Data Access and User Support.
6. The USGS Life Cycle includes Plan, Acquire, Process, Analyze,
Preserve, Publish/Share with three activities running through all
phases: Describe (Metadata and Documentation), Manage Quality,
Backup and Secure.
7. The ESA Heterogenous Missions Accessibility Report really
focuses on data access and not the other phases.
8. The Harnessing the Power of Digital Data: Taking the Next Step,
Science Data Management Workshop report provides a number of models:
FGDC life cycle: Define, Inventory/Evaluate, Obtain, Access,
Maintain, Use/Evaluate, Archive.
Linear data lifecycle: Plan, Collect, Integrate and Transform,
Publish, Discovery with two activities running through all phases,
Governance and Stewardship and Communications.
Basic science model: plan, collect, integrate and transform,
publish, discover and inform, archive or discard.
The topics that are identified in the report include: data
governance, stewardship, sharing, access, security, version control,
metadata management, content and format, document and content
management, preservation, transfer of responsibility, data
architecture, database operations management, reference and master
data management, data warehousing and business intelligence, data
quality management, provenance, usability, value added services,
workflow systems.
9. The LPDAAC Lifecycle Plan identifies the phases: Inception,
Active Archive, Long-Term Archive which each have four elements,
characterization, critical data and information, applicable
standards, transition.
The WBS is broken into phases, inception-planning (embed in producer
team, provide data management plan), inception-production (laison to
science stakeholders, collection inception checklist, support
production, repeat experiment, determine approach to tools/services,
authorize to migrate, provide NASA data template), active archive
transition from producer (obtain authorization to migrate, plan
migration, install new product line, migrate, advertise new
products, assume primary access and discovery role), active archive
transition to long-term (obtain authorization to migrate, plan
migration), long term archive transition to long-term (enable
migration, execute migration, advertise new products, transfer
primary access and discovery role, obtain authorization for
certification, sunset products).
10. DataOne includes Collect, Assure, Describe, Deposit, Preserve,
Discover, Integrate, Analyze
11. Jeff de La Beaujardičre's Data Management Framework
Planning and Production (Requirements Definition, Planning,
Development, Deployment, Operations);
Data Management Activities (Collection, Processing, Quality Control,
Documentation, Dissemination, Cataloging, Preservation, Stewardship,
Usage Tracking, Final Disposition);
Usage Activities (Discovery, Reception, Analysis, Product
Generation, User Feedback, Citation, Tagging, Gap Analysis).
_______________________________________________
Moims-dai mailing list
<mailto:Moims-dai at mailman.ccsds.org> Moims-dai at mailman.ccsds.org <mailto: <mailto:Moims-dai at mailman.ccsds.org> Moims-dai at mailman.ccsds.org>
<http://mailman.ccsds.org/mailman/listinfo/moims-dai> http://mailman.ccsds.org/mailman/listinfo/moims-dai
_______________________________________________
Moims-dai mailing list
<mailto:Moims-dai at mailman.ccsds.org> Moims-dai at mailman.ccsds.org
<http://mailman.ccsds.org/mailman/listinfo/moims-dai> http://mailman.ccsds.org/mailman/listinfo/moims-dai
_______________________________________________
Moims-dai mailing list
<mailto:Moims-dai at mailman.ccsds.org> Moims-dai at mailman.ccsds.org
<http://mailman.ccsds.org/mailman/listinfo/moims-dai> http://mailman.ccsds.org/mailman/listinfo/moims-dai
<Preservation Workflow_v1.0.pdf>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/moims-dai/attachments/20160114/dc40da1d/attachment.html>
More information about the MOIMS-DAI
mailing list