[Moims-dai] FW: Further thoughts on adapters

david at giaretta.org david at giaretta.org
Sat Jul 10 19:43:43 UTC 2021


Hi Mike

 

See below.

 

..David

 

From: kearneysolutions at gmail.com <kearneysolutions at gmail.com> 
Sent: 10 July 2021 19:35
To: david at giaretta.org
Cc: 'Steve Hughes (398B)' <john.s.hughes at jpl.nasa.gov>; 'John Garrett'
<garrett at his.com>; 'MOIMS-DAI List' <moims-dai at mailman.ccsds.org>
Subject: RE: [Moims-dai] FW: Further thoughts on adapters

 

All good answers, David.  With that I'll start to update the EWA
presentation.  

 

A few more follow-up questions.  

 

On applicability:  

Also, you proposed changing the ArchiveAbstractionLayer name to the OAIS-IF
Abstraction Layer (OAL).  Because it serves more than archives.  I'm a
little confused by that because the word "Archive" is in "OAIS-IF", so I
don't think you've extended the applicability beyond archives by renaming it
from AAL to OAL.  Can you tell me more about why you want to make that
rename?  

To my ear when one uses AAL I tend to think this only applies to Archives.
When I hear OAL I tend to think of this as part of the OAIS suite of
standards.

MK - I thought all standards from DAI, meaning the OAIS suite of standards,
apply only to archives.  

Well certainly OAIS applies only to archives. But IPELTU. DEDSL, EAST,
Control Authorities can be used anywhere.

 

We're not trying to get it to apply to systems other than archives, like
other CCSDS working groups or other IT systems worldwide, right?  

We need systems other than archives to use OAIS-IF because otherwise we are
only dealing with archive-to-archive exchanges. If we want "users" to use
data from archives then the users need to use OAIS-IF 

 

Because earlier, I thought that was what you were saying.  I think we need
to stay "in our box" and indicate that it applies only to Archives (OAIS
archives or non-OAIS but OAIS-IF archives) by calling it the Archive
Abstraction Layer.  

 

On allocation of functions:  

*       InfoSourceSpecificAdapter (aka ArchiveSpecificAdapter when it is in
front of an archive)

This is the one which knows (for example through a configuration file - for
the same reasons for the InfoReqesterSpecificAdapter ) how to grab the
things needed to make up an InfoObject which will go into the InfoPackage
i.e. a DataObject and its RepInfo. The InfoObject may be ProvenanceInfo so
it will need to know where to ger the ProvenanceDataObject etc.

MK - all of those functions sound like functions *of the archive*.  It seems
like you're taking functions that belong in the archive, and putting them in
the user's software.  Like you're expecting the user to take the AIP and
build a DIP package by assembling "the things neeed to make up an
InfoObject" or InfoPackage.  Why would we do that?  

If an archive is the InfoSource then the InfoSourceSpecificAdapter is
sitting in the archive - NOT in the user software. Are the diagrams not
clear on that point?

 

 

   -=- Mike

 

Mike Kearney

Huntsville, Alabama, USA 

 

From: david at giaretta.org <mailto:david at giaretta.org>  <david at giaretta.org
<mailto:david at giaretta.org> > 
Sent: Saturday, July 10, 2021 6:49 AM
To: kearneysolutions at gmail.com <mailto:kearneysolutions at gmail.com> 
Cc: 'Steve Hughes (398B)' <john.s.hughes at jpl.nasa.gov
<mailto:john.s.hughes at jpl.nasa.gov> >; 'John Garrett' <garrett at his.com
<mailto:garrett at his.com> >; MOIMS-DAI List <moims-dai at mailman.ccsds.org
<mailto:moims-dai at mailman.ccsds.org> >
Subject: RE: [Moims-dai] FW: Further thoughts on adapters

 

Hi Mike

 

All good questions. Here is my take - of course you must realise that I do
not have a fully worked out design in  my head so (1) I think it makes
sense, but I may be wrong and (2) I hope you can forgive lack of details. 

 

My belief is that it should be relatively easy to complete enough of the
design to get a couple of prototype implementations together and then
iterate through once or twice more where we see what is missing.

 

Since this should be of general interest I am copying it to the full list so
we can get feedback from others - hope that is OK.

 

Regards

 

..David

 

From: kearneysolutions at gmail.com <mailto:kearneysolutions at gmail.com>
<kearneysolutions at gmail.com <mailto:kearneysolutions at gmail.com> > 
Sent: 10 July 2021 06:20
To: david at giaretta.org <mailto:david at giaretta.org> 
Cc: Steve Hughes (398B) <john.s.hughes at jpl.nasa.gov
<mailto:john.s.hughes at jpl.nasa.gov> >; John Garrett <garrett at his.com
<mailto:garrett at his.com> >
Subject: RE: [Moims-dai] FW: Further thoughts on adapters

 

David:  In support of the EWA pitch on OAIS-IF, I need to revamp my PPT
Architecture charts.  I need to incorporate your generic/specific adapter
concepts, modified by the discussions we had at the last telecon I was able
to attend.  

 

I'm in Nevada right now, and I'm not sure I'll be able to be at the telecon
next Tuesday 7/13. maybe, but it's tentative.  I definitely won't be able to
attend the telecon on 7/20, but I should be able to be at the one on 7/27.
But I want to work on this by email since the EWA presentation is coming up
pretty soon.  

 

I'm looking at your interaction patterns (attached) and I'd like to make
some statements, and have you tell me if you agree with them.  Basically to
insure I understood what you said at that last telecon.  

 

1.       The two generic adapters (Object 1 Generic Adapter and Object3
Generic Adapter2) can be inside the abstraction layer.  I think that means
the Abstraction Layer Blue Book will have interface descriptions for
InfoRequesterSpecificAdapter-to-Abstraction Layer and Abstraction Layer to
InformationSourceSpecificAdapter.  

Yes we need those interfaces - I hope we can just define one set of methods
in the generic adapter interface that includes source as well as requester
since both sides have inputs and outputs. On the other hand we spoke at the
meeting of dividing the generic adapter into a requester library and a
source library so make things easy. We also need to provide the interface
which allows the generic adapters to talk to each other - if we divide it
into 2 libraries this is where the requester library talks the source
library and vice-versa.

 

2.       At least for an archive instance, the
InformationSourceSpecificAdapter can be called the ArchiveSpecificAdapter.  

Sure, where it sits in front of an archive. That will probably make it
easier to understand. Of course there could/will be InfoSources other than
archives.

 

In your PPT, where it is clear that this is in front of an archive then it
will not cause confusion to call it an ArchiveSpecificAdapter (which the BB
can say is a sub-type of InformationSourceSpecificAdapter).

 

3.       The InfoRequesterSpecificAdapter sits between the User Interface
Software and the Archive Layer (at the Object1 GenericAdapter interface).
Maybe the InfoRequesterSpecificAdapter could also be called the
UserSpecificAdapter?  

 I tend to like the general name but if having a more user friendly name
would help then that would be fine unless it causes confusion where, for
example an Archive requests information from another InfoSource e.g. a
Registry.

 

In your PPT, where it is clear that this is in front of a User then it will
not cause confusion to call it an UserSpecificAdapter (which the BB can say
is a sub-type of InfoRequesterSpecificAdapter).

 

4.       The registry is a 3rd party generally external to the user and the
archive.  

Logically this should be the case since I rather imagine that there may be a
small number of Registries serving many InfoSources/InfoRequesters

 

5.       The Switchboard is also a 3rd party generally external to the user
and the archive.  

Same as for Registry

 

6.       Finally, it would help if you could just list a few functions
apiece for the following boxes.  I need more of a description of them than I
am getting from your discussion on the interaction patterns (below).  For
example, which ones interpret RepInfo, etc.  

Only the thing that the InfoRequesterAdapter is in front of e.g. the user
analysis software, actually needs to interpret RepInfo. All the other things
just deal with RepInfo blobs or identifiers. Of course they have to go back
to the "user analysis software" to get a hint about what is needed.

 

*       Registry

To minimize what we have to do I would look on this as an OAISArchive which
only contains Representation Information.

 

*       Switchboard

To minimize what we have to do I would see this as an archive (i.e. not
necessarily full OAIS) which only contains the connection related info. One
question is how long this needs to exist. It could be the same system as the
DNS i.e. there are expiry times of an entry after which a "keep-alive"
message is sent - but that is well developed by others..

 

*       InfoReqesterSpecificAdapter  (aka UserSpecificAdapter)

This speaks to the "user analysis software" - I assume that, because
initially no one will want to alter their software to deal with OAIS-IF, the
easiest way would be to have some configuration file which tells us what the
"user analysis software" can deal with e.g. I can deal with CSV files, or
this is what to do with Provenance (such as just pop up a text window)

 

*       InfoSourceSpecificAdapter (aka ArchiveSpecificAdapter when it is in
front of an archive)

This is the one which knows (for example through a configuration file - for
the same reasons for the InfoReqesterSpecificAdapter ) how to grab the
things needed to make up an InfoObject which will go into the InfoPackage
i.e. a DataObject and its RepInfo. The InfoObject may be ProvenanceInfo so
it will need to know where to ger the ProvenanceDataObject etc. I can see
that there will be some complications if the request is for a sub-set of a
DataObject - but that depends upon what the archive offers as a service -
again this may be specified in a configuration file.

Eventually archives may support OAIS-IF natively, which would make things
easier.

 

*       GenericAdapter subset of the Abstraction Layer.

This know how to communicate with another GenericAdapter, and how to talk to
the SpecificAdapter. It also knows how to pack and unpack PackagedInfoObject
which are sent over the network.

 

Also, you proposed changing the ArchiveAbstractionLayer name to the OAIS-IF
Abstraction Layer (OAL).  Because it serves more than archives.  I'm a
little confused by that because the word "Archive" is in "OAIS-IF", so I
don't think you've extended the applicability beyond archives by renaming it
from AAL to OAL.  Can you tell me more about why you want to make that
rename?  

To my ear when one uses AAL I tend to think this only applies to Archives.
When I hear OAL I tend to think of this as part of the OAIS suite of
standards.

 

Sorry to be having to work this across email rather than telecon, while I'm
traveling.  Once I get your replies, my plan is to modify the original
architecture PPTs to like up with your interaction pattern terminology.  

 

Thanks, 

 

   -=- Mike

 

Mike Kearney

Huntsville, Alabama, USA 

 

From: MOIMS-DAI <moims-dai-bounces at mailman.ccsds.org
<mailto:moims-dai-bounces at mailman.ccsds.org> > On Behalf Of
david at giaretta.org <mailto:david at giaretta.org> 
Sent: Friday, June 25, 2021 7:49 AM
To: MOIMS-DAI List <moims-dai at mailman.ccsds.org
<mailto:moims-dai at mailman.ccsds.org> >
Subject: [Moims-dai] FW: Further thoughts on adapters

 

As suggested during the meeting on Tuesday I have added numbers to the text
description to identify the corresponding arrows in the diagram.

 

1)      Each will need to buy/borrow/create their specific portion of the
adapter that depends upon its specific software design and capabilities e.g.


a.       for an InfoSource - is it a simple file server which we can
supplement with a database of RepInfo etc         [1]

b.       for an InfoRequester - can it deal with tables and/or images or is
it only for displaying documents etc               [2]

2)      They each then install the generic portion of the adapter - I think
this can be common to all.                                       [3], [4]

This generic portion does the following things:

a.       Find out about InfoSource using interface we define

b.       Register itself and the InfoSource with the Switchboard e.g. how
others can communicate with it - HTTP/port80 or HTTP/port 2678 or REST or
CORBA or RMI etc etc

This is done just once for each source (or requester)
[5], [6]

3)      During operations the requester discovers somehow that it needs some
information from an InfoSource - how this is done is not covered here. I
think lots of work has been done by others, and OAIS does not really provide
any guidance.

4)      InfoRequester generic adapter finds out from the Switchboard how to
talk to the InfoSource in order to fit into the "OAIS-IF world", and sets up
the communication [7].[8], [9]

5)      The specific adapter of InfoRequester passes on the request to the
generic adapter which passes on the request to the InformationSource generic
adapter         [10], [11]

6)      The generic adapter of the InfoSource receives the request and
passes it to the specific adapter which gets the DataObject and RepInfo (at
least the start of the RepInfoNetwork) and passes that back
[NOT SHOWN]

7)      The InfoSource generic adapter creates an PackagedInfo object and
sends that to the generic adapter of the InfoRequester  [12]

8)      The generic adapter of the InfoRequester asks the specific adapter
if this is OK. If not then the generic adapter exchanges messages with the
InfoSource generic adapter, which may need to talk to the InfoSource
specific adapter.
[13], [14], [15], [16], [17]

9)      PackagedInformation packages are send containing RepInfo until the
InfoRequester is satisfied, or else a strategy is decided to use
Transformation, if the InfoSource can do that
[18], [19], [20], [21]

10)   It may be that a Registry of RepInfo (essentially another
InformationSource) is available and known to the generic adapter, and
additional RepInfo may be obtained from that.
[22], [23]

 

 

Clearly the diagram needs some tweaks but I think this identifies the main
steps. 

 

Regards

 

..David

 

From: david at giaretta.org <mailto:david at giaretta.org>  <david at giaretta.org
<mailto:david at giaretta.org> > 
Sent: 22 June 2021 13:32
To: MOIMS-DAI List <moims-dai at mailman.ccsds.org
<mailto:moims-dai at mailman.ccsds.org> >
Subject: Further thoughts on adapters

 

Dearf all

 

Thinking some more about adapters and what they need to do I have the
following suggestion, making some additions to the diagram I sent out
yesterday.

 

I think we can separate the adapter into 2 parts, (1) a part which  is very
specific to the object that is sending or receiving information, (2) a
generic part which contains much of the logic about packages and requests.
The interface between these two parts can be specified - but I only have it
in outline at the moment.

 

Taking some inspiration from Mike's PPT diagrams I think we can describe the
setup and operations as follows for a given pair of InfoRequester and
InfoSource

 

1)      Each will need to buy/borrow/create their specific portion of the
adapter that depends upon its specific software design and capabilities e.g.


a.       for an InfoSource - is it a simple file server which we can
supplement with a database of RepInfo etc

b.       for an InfoRequester - can it deal with tables and/or images or is
it only for displaying documents etc

2)      They each then install the generic portion of the adapter - I think
this can be common to all. This generic portion does the following things:

a.       Find out about InfoSource using interface we define

b.       Register itself and the InfoSource with the Switchboard e.g. how
others can communicate with it - HTTP/port80 or HTTP/port 2678 or REST or
CORBA or RMI etc etc

This is done just once for each source (or requester)

3)      During operations the requester discovers somehow that it needs some
information from an InfoSource - how this is done is not covered here. I
think lots of work has been done by others, and OAIS does not really provide
any guidance.

4)      InfoRequester generic adapter finds out from the Switchboard how to
talk to the InfoSource in order to fit into the "OAIS-IF world", and sets up
the communication

5)      The specific adapter of InfoRequester passes on the request to the
generic adapter which passes on the request to the InformationSource generic
adapter

6)      The generic adapter of the InfoSource receives the request and
passes it to the specific adapter which gets the DataObject and RepInfo (at
least the start of the RepInfoNetwork) and passes that back

7)      The InfoSource generic adapter creates an PackagedInfo object and
sends that to the generic adapter of the InfoRequester

8)      The generic adapter of the InfoRequester asks the specific adapter
if this is OK. If not then the generic adapter exchanges messages with the
InfoSource generic adapter, which may need to talk to the InfoSource
specific adapter.

9)      PackagedInformation packages are send containing RepInfo until the
InfoRequester is satisfied, or else a strategy is decided to use
Transformation, if the InfoSource can do that

10)   It may be that a Registry of RepInfo (essentially another
InformationSource) is available and known to the generic adapter, and
additional RepInfo may be obtained from that.

 

A diagram which tries to capture this is attached - an update of the diagram
sent yesterday.

 

So we need to define

A)      The interface between the generic and specific adapters

B)      The outward facing generic adapter interfaces and logic - a lot of
this is in the GB draft

C)      The OAIS-IF objects sent over the wire - most of this Steve has
defined

D)      The Switchboard is just another InfoSource, as is the Registry

E)       The  interfaces to deposit information into an InfoSource - the
Producer/Archive interface - to be done later because it is rather more
difficult to do generically

 

Hope to discuss this at the Skype meeting later today.

 

Regards

 

..David

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ccsds.org/pipermail/moims-dai/attachments/20210710/c0610f47/attachment-0001.htm>


More information about the MOIMS-DAI mailing list