[Moims-dai] FW: [wg-all] New document published: Data Format Description Language (DFDL) v1.0

Sun Dec 7 19:53:47 UTC 2014

Hi Mike

Yes I agree with Stephane and you that it would be useful to make that survey as part of the work. One important concern for me is that we try to get some idea of which data description tool/technique can be used for which sort of data. For example neither DRB nor EAST is probably is useful for describing a Word document or PDF (see http://www.alliancepermanentaccess.org/index.php/tools/tools-for-preservation/ ).

..David

-----Original Message-----
From: Mike Martin [mailto:tahoe_mike at sbcglobal.net] 
Sent: 07 December 2014 18:30
To: MOIMS-Data Archive Ingestion
Cc: Giaretta, David (STFC,RAL,RALSP); stephane.mbaye at gael.fr
Subject: Re: [Moims-dai] FW: [wg-all] New document published: Data Format Description Language (DFDL) v1.0

Hi David

This is very appropriate as Stephane just brought DFDL up in a side conversation responding to my inquiry on his thoughts about future work for DAI.  I include his email to me below:

Stephane:  "I believe that data description is of a major interest in computer science although it is still not widely used, and not only in the space area. Up to now the development of "standard" format is considered satisfactory by a large number of communities but I don't believe it is a good technical solution, at least because each new "universal" format is yet another one, which is a bit contradictory with the intent to stop a proliferation. Worse, if the new universal format is successful and widely implemented it may stick the community in a bottleneck and avoid interesting evolution. XML is an example, where the massive success of the 1.0 denied the jump to the 1.1, however useful.

I am more convinced by systems that would consider any format changes as a natural. In that area, data description is in interesting step that makes applications less dependent from the physical characteristics but more at higher levels e.g. logical, semantic.

I don't know if it is up to the CCSDS to develop the solution but it should at least identify or sustain developments in that direction. EAST is probably used by the CNES only, but it may be for reasons different from the need of data description. Apart political reasons, my feeling is that the ADA base syntax is a big part of the problem. ADA is so specific that it is difficult to find a reusable parser/compiler for all platforms. Users of EAST have to develop their own solution prior to any use or reuse some "native" components that tie them down to a specific hardware or OS version.

As you said for DEDSL, before thinking about scrapping EAST or developing an XML version of EAST, I think it would be interesting for the CCSDS to make a survey of the current practices and solutions. There is for example the quite new DFDL from the OpenGrid that could suffice for space applications."

Back to Mike:  The purpose of my spreadsheet itemizing the usage and disposition of CCSDS DAI standards was to assess the utility of current standards.  If these standards are not being adopted outside of the organizations that created them, then I think we need to ask why, and what can be done to either improve them or look for new alternatives.  I agree with Stephane's recommendation that we make a survey of the current practices and solutions and where they are being used.  What standards are new curation efforts using in lieu of the CCSDS standards? 
  I think this can feed into the recommendations in ICP.

Thanks, Mike

On 12/7/2014 1:47 AM, david.giaretta at stfc.ac.uk wrote:
> FYI
>
> ---------- Forwarded message ----------
> From: *Greg Newby* <gbnewby at petascale.org 
> <mailto:gbnewby at petascale.org>>
> Date: 14 November 2014 15:03
> Subject: [wg-all] New document published: Data Format Description 
> Language (DFDL) v1.0
> To: All WG <wg-all at ogf.org <mailto:wg-all at ogf.org>>
>
>
> Specification
> Reply-To: gbnewby at petascale.org <mailto:gbnewby at petascale.org>
> Fcc: =ed
>
> OGF Community:
>
> A new document has been published in the OGF series.  All OGF 
> documents (including any that are open for public comment) may be 
> found here:
>
> www.ogf.org/ogf/doku.php/documents/documents
> <http://www.ogf.org/ogf/doku.php/documents/documents>
>
> The new document:
>
> * GFD-R-P.207, "Data Format Description Language (DFDL) v1.0
>    Specification."  M. Beckerle & S. Hanson.  Via DFDL-WG.
>
> Abstract:
>    This document provides a definition of a standard Data Format 
> Description Language (DFDL).  This language allows description of 
> text, dense binary, and legacy data formats in a vendor-neutral 
> declarative manner. DFDL is an extension to the XML Schema Description 
> Language (XSDL).
>
>    -- Greg Newby, OGF Editor
>
> Gregory Newby, Manager of the Supercomputing Core Laboratory.
> King Abdullah University of Science and Technology (KAUST), Saudi Arabia.
> E: greg.newby at kaust.edu.sa <mailto:greg.newby at kaust.edu.sa>  M: +966 
> 54
> 470 1773 <tel:%2B966%2054%20470%201773>  O: +966 12 808 0678 
> <tel:%2B966%2012%20808%200678> Building 1 East-Room 3407, 4700-Thuwal, 
> Jeddah 23955-6900.
> ksl.kaust.edu.sa <http://ksl.kaust.edu.sa>
> --
>    The wg-all mailing list is an umbrella list for all
>    OGF mailing lists.  If you are subscribed to any OGF
>    mailing list, you will receive mails to wg-all.
>
>    wg-all mailing list
> wg-all at ogf.org <mailto:wg-all at ogf.org> 
> https://www.ogf.org/mailman/listinfo/wg-all
>
>
>
> _______________________________________________
> Moims-dai mailing list
> Moims-dai at mailman.ccsds.org
> http://mailman.ccsds.org/mailman/listinfo/moims-dai
>