This page gathers the enhancements and current limitations on the JMMC OITools library focused on the OIFITS data handling.

Its main features are:

• read / write OIFITS files (V1 / V2 + IMAGE_OI extensions = OImaging)
• read / write FITS image(s) and cubes (primary HDU, extra Image HDU)
• data model description: meta-data for keywords, columns, tables
• in-memory data structures to deal with tables (columns, keywords) and derived columns (hard-coded & expression support)
• validation (command-line interface + web interface)
• serialization in TSV / XML formats (used by oidb)

Enhancements

The rules for OIFits Files

• The table for all rules (and apply to) has been created.
• There are also tables for DataModel version 1 and version 2

FeedBack OIFits2 Standard:

• We have different synthax for keyword %s, %f, %d for header keyword.

Tests

• TODO: describe the JUnit tests + file structure
• TODO: Headercards type is string for Int/float/double too
• TODO: Test update ? (History order) / (Hierarch)
  • see strict compare Nom.tam.fits
• TODO: imageHDU should be returned even if no image to hold keywords -> could rely on the firstOnly flag ?
• Improve the OIFitsValidator test to compare the validation results

Junit Tests

Copy files

Objective:

• Load & write reference FITS (images) & OIFits (V1, V2, with IMAGE_OI extensions) files
• Compare the OIFitsFile structures (all tables, all keywords, all columns, header cards): OK ?
• Compare the nom.tam.Fits (raw HDUs):

TODO: FIX the lenient comparison mode (strict is too exhaustive) to ensure the copied files have not lost or modified original keyword / data

See README in the src/test/resource folder describing the reference files

TODO

Improve the OIFITS validator (V1 / V2)

• improve the validation results:
  • extract the severity, origin (table/keyword or column) from the validation message
  • sort & filter validation results

Problem 1: We have multi min/max methodes

• In several file we have methode for calculate min/max value
• for exemple : oifitsFile.getMinWavelengthBound() and oiWavelength.getEffWaveMin()
• We need just 1 method to compute all kind of bound on any kind of mesurements (see also xmlOutputVisitor.appendRecord() )

Problem 2: We need to change Tam Fits Test for scrict comparison

• Currently we ignore some error for the strict test or we need to fix that
• We need to chose the best chose fix or ignore error, depending on severity.

See: oitools/src/test/resources/ref/WriteOIFitsTest.log

Ignore case:

• ARRAYXYZ: 123.0 vs 123.
• NAXIS1 = row size different (taille des String)
• TFORM = Data format
  • different value of header card[TFORM2] '6A' <> '16A'
  • different value of header card[TFORM3] 'I' <> '1I'

Extra Tables

Binary Tables that are not in the OIFits standards (V1 or V2) are totally skipped (OIFitsLoader? / OIFitsWriter? )

• Consequence: Copying a FITS file may loose 'unknown' tables (load then write) !
• Solution: When loading such a table, it seems possible to parse its header and create keyword / column descriptors on the fly then load it as usual (GenericTable extends Table)

Extra Columns in table

• Try to detect the displaced columns ... => avoid false positives

1. file: PRODUCT_V838_Mon_1-copy.90-2.53micron_2013-04-15T01_49_24.8034--------0.00000------------inf--------1--------2--------3

• keywords:
  • OI_VIS Missing header card TTYPE15 was = VISCOVRI
  • OI_VIS Missing header card TFORM15 was = 16D
• Columns:
  • different number of columns 17 <> 14 in VISCOVRI

1. file: GRAVITY-copy.2016-01-09T05-37-06_singlesci_calibrated

• different number of columns 16 <> 14 in ASTROMETRIC_DELAY
• different number of columns 3 <> 2 in POLAR

• OI_VIS Missing header card TTYPE15 was = ASTROMETRIC_DELAY
• OI_VIS Missing header card TTYPE15 was = VISCOVRI

• OI_T3 different number of header card 50 <> 52

Dimensions (TDIM)

1. file: AMBER_070409-copy

• OI_ARRAY Missing header card TDIM5 was = (3)
• OI_VIS Missing header card TDIM7 was = (20)
• different values for column[VISDATA]
• different values for column[VISERR]

See OIFits V2 standard that mentions how to use the TDIM keyword

Problem 6: Setters API

To easily fill tables (complex keywords, columns), a simple setter API will be implemented:

/**
* Define the column value for the given row (index)
*/
(OITable).setColumn(int row, (Type) value)

/**
* Define all column values for the given row (index)
*/
(OITable).setRow(int row, values...)

Limitations

Cube Image support: Fixed

Last version can load FITS image cubes from one ImageHDU but also write back to a FITS or OIFits file into one ImageHDU (with the proper wavelength keywords - AXIS3)

This use case is not yet required but next ones certainly will require code enhancement:

• support of polychromatic data in OImaging

External resources

• https://github.com/jsy1001/oifitslib provides some V2 test files