This page is listing technical discussions and notes on the OiDB project.

Specifications

See the definition of the OiDB project in the document from the 2013-06-26 meeeting.

Notes

Collection

Define a collection as a way for a submitter (dataPI) submitting data to group entries together.

Push/Pull for database insertion

Methods for adding data :

  • submit files/collection: the user make HTTP PUT request to an URL on the server for his OIFits files or collection of OIFits files. The portal displays a Web form for manually importing new data.

  • download files/collection: the application is in charge of automatically downlaading the files from the server specified by the user. TODO: define format: XML files with resources to analyze ? parsing HTML file listing ? The portal let the user specify the address of his server and the access conditions.

Data update

Files/collections may be modified by the owner from time to time. The application should regularly download and reprocess the distant repository to stay up to date.

Ask the user for reimporting the data (user resend data)? Automatic import (identify modifications on files: checksum)?

Is it possible to use the name of the file as observation ID?

Data display

To retrieve external data :

  • turn the portal into a proxy: at user request, the portal check the repository, download the file and return it to the user. The application knows for sure that the user has download the file.

  • the user perform a request on the application that return the URL for the requested resource. The user then make an HTTP GET request on this URL to download the document. The application can log the request (the user may have downloaded the file but not guaranteed).

  • make a redirection (similar to Google search results): slightly different from the previous proposition, the user performs a request on the portal that issue a redirect (URL forwarding, HTTP 303) to the external repository. Do all clients and libs support 303 redirections ?

Private data

Some data may not be downloadable until a given amount of time. Still should let people get in contact with the dataPI.

Changes to the current database schema

  • sort by collection.

  • link data and dataPI.

  • add fields for descriptions and comments on data (file, collection target).

  • make link between observation in the database and scientific publication (bibliographic reference)

  • keep versions from a same source.

Data model

field datatype L0 L1 L2 L3 SAMPLE Remarques
dataproduct_type           OpticalInterferometryData  
calib_level           3  
target_name           HD31964  
obs_id            
obs_collection           J/A+A/544/A91  
obs_creator_name           Mourard, D.  
obs_release_date            
obs_published_did            
bib_reference           2012A&A...544A..91M  
data_rights           public  
access_url           HA2009.11.05.12.51.fits  
access_format            
access_estsize            
s_rapos.eq.ra           0  
s_decpos.eq.dec           0  
s_fov            
s_region            
s_resolution            
t_min           2455141  
t_max           2455141  
t_exptime           0  
t_resolution            
em_min           6.54085e-07  
em_max           6.585352e-07  
em_res_power           0  
o_ucd            
pol_states            
facility_name           CHARA  
instrument_name           VEGA  
nb_channels           90  
nb_vis           90  
nb_vis2           0  
nb_t3           0  
subdate           2014-02-18 23:38:41.0  

OIFITS

Define/extract metadata from submitted OIFits files.

  • manage datum with flag=true (OI_VIS, OI_VIS2, OI_T3).

  • take into account repeated wavelength (more than one OI_WAVELENGTH in a single OIFits with the same wavelength value).

  • measurements from more than one instrument in the same file (OI_WAVELENGTH tables with different INSNAME while observing the same target).

Publishing the the VO

See :

Web portal examples

Web Site MultiCriterii ADQL TAP/ObsCore Samp Description/comments
http://portal-nvo.noao.edu/adql/query X X      
http://voparis-srv.obspm.fr/portal/vo.php X   ? X  
http://saada.u-strasbg.fr/taphandle/   X   X  
http://labs.adsabs.harvard.edu/adsabs/ X NC NC NC Not for observation data wink
http://polarbase.irap.omp.eu/ X   X    
http://herschel.esac.esa.int/Science_Archive.shtml         javaWebStart interface see Johan screenshots on the mailinglist : 05/02/2014 14:54

This topic: Jmmc/Software > OiDbImplementationsNotes
Topic revision: r7 - 2014-04-29 - GuillaumeMella
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback