Specifications

See the definition of the OiDB project in the document from the 2013-06-26 meeeting.

Notes

Collection

Define a collection as a way for a submitter (dataPI) submitting data to group entries together.

Push/Pull for database insertion

Methods for adding data :

submit files/collection: the user make HTTP PUT request to an URL on the server for his OIFits files or collection of OIFits files. The portal displays a Web form for manually importing new data.

download files/collection: the application is in charge of automatically downlaading the files from the server specified by the user. TODO: define format: XML files with resources to analyze ? parsing HTML file listing ? The portal let the user specify the address of his server and the access conditions.

Data update

Files/collections may be modified by the owner from time to time. The application should regularly download and reprocess the distant repository to stay up to date.

Ask the user for reimporting the data (user resend data)? Automatic import (identify modifications on files: checksum)?

Is it possible to use the name of the file as observation ID?

Data display

To retrieve external data :

turn the portal into a proxy: at user request, the portal check the repository, download the file and return it to the user. The application knows for sure that the user has download the file.

the user perform a request on the application that return the URL for the requested resource. The user then make an HTTP GET request on this URL to download the document. The application can log the request (the user may have downloaded the file but not guaranteed).

make a redirection (similar to Google search results): slightly different from the previous proposition, the user performs a request on the portal that issue a redirect (URL forwarding, HTTP 303) to the external repository. Do all clients and libs support 303 redirections ?

Private data

Some data may not be downloadable until a given amount of time. Still should let people get in contact with the dataPI.

Changes to the current database schema

sort by collection.

link data and dataPI.

add fields for descriptions and comments on data (file, collection target).

make link between observation in the database and scientific publication (bibliographic reference)

keep versions from a same source.

Data model

field	datatype	L0	L1	L2	L3	SAMPLE	description
dataproduct_type	text	opticalInterferometryData	opticalInterferometryData	opticalInterferometryData	opticalInterferometryData	opticalInterferometryData	High level scientific classification of the data product, taken from an enumeration
calib_level	smallint					3	Amount of data processing that has been applied to the data
target_name	text					HD31964	Object a targeted observation targeted
obs_id	text						Unique identifier for an observation
obs_collection	text					J/A+A/544/A91	Name of a data collection (e.g., project name) this data belongs to
obs_creator_name	TBD					Mourard, D.	TBD
obs_release_date	TBD						TBD
obs_published_did	TBD						TBD
bib_reference	TBD					2012A&A...544A..91M	TBD
data_rights	TBD					public	TBD
access_url	text					HA2009.11.05.12.51.fits	The URL at which to obtain the data set.
access_format	text						MIME type of the resource at access_url
access_estsize	bigint						Estimated size of data product
s_rapos.eq.ra	TBD					0	TBD
s_decpos.eq.dec	TBD					0	TBD
s_fov	double precision						Approximate spatial extent for the region covered by the observation
s_region	spoly						Region covered by the observation, as a polygon
s_resolution	double precision						Best spatial resolution within the data set
t_min	double precision					2455141	Lower bound of times represented in the data set, as MJD
t_max	double precision					2455141	Upper bound of times represented in the data set, as MJD
t_exptime	real					0	Total exporure time
t_resolution	real						Minimal significant time interval along the time axis
em_min	double precision					6.54085e-07	Minimal wavelength represented within the data set
em_max	double precision					6.585352e-07	Maximal wavelength represented within the data set
em_res_power	double precision					0	Spectral resolving power delta lambda/lamda
o_ucd	text						UCD for the product's observable
pol_states	text						List of polarization states in the data set
facility_name	text					CHARA	Name of the facility at which data was taken
instrument_name	text					VEGA	Name of the instrument that produced the data
nb_channels	TBD					90	TBD
nb_vis	TBD					90	TBD
nb_vis2	TBD					0	TBD
nb_t3	TBD					0	TBD
subdate	TBD					2014-02-18 23:38:41.0	TBD

OIFITS

Define/extract metadata from submitted OIFits files.

manage datum with flag=true (OI_VIS, OI_VIS2, OI_T3).

take into account repeated wavelength (more than one OI_WAVELENGTH in a single OIFits with the same wavelength value).

measurements from more than one instrument in the same file (OI_WAVELENGTH tables with different INSNAME while observing the same target).

Publishing the the VO

See :

http://wiki.ivoa.net/twiki/bin/view/IVOA/PublishingInTheVONew

Web portal examples

Web Site	MultiCriterii	ADQL	TAP/ObsCore	Samp	Description/comments
http://portal-nvo.noao.edu/adql/query	X	X
http://voparis-srv.obspm.fr/portal/vo.php	X		?	X
http://saada.u-strasbg.fr/taphandle/		X		X
http://labs.adsabs.harvard.edu/adsabs/	X	NC	NC	NC	Not for observation data
http://polarbase.irap.omp.eu/	X		X
http://herschel.esac.esa.int/Science_Archive.shtml					javaWebStart interface see Johan screenshots on the mailinglist : 05/02/2014 14:54

Topic revision: r8 - 2014-04-29 - GuillaumeMella