NEID Data Format ================ Overview -------- NEID data products are defined for three different data levels: * Level 0: Raw data produced by the NEID instrument control system at the WIYN observatory * Level 1: Extracted, wavelength calibrated spectra * Level 2: Derived products, including radial velocites, activity indicators, and telluric models Each of these data levels is a standardized, multi-extension FITS format, and can be read using standard fits tools. Filenames follow the convention: *neidL#_YYYYMMDDThhmmss.fits*, where # is 0, 1, or 2 for the data level, and YYYYMMDDThhmmss is the timestamp corresponding to the beginning of the exposure. Level 0 Data Format ------------------- A NEID Level 0 data file is generated for each exposure, either for science or calibration. Each file is in a standardized, multi-extension FITS format, and contains the header-data units (HDUs) listed in Table 1. The primary (zeroth) HDU contains header keywords for instrument, telescope, and observatory metadata, but no data. HDUs 1-16 contain 2-D spectroscopic image data for the 16 instrument readout channels (amplifiers). HDUs 17-20 contain the exposure-meter time-series data, guider image data, coherent-fiber-bundle data, and solar telescope data ====== ======================================================== HDU # Description ------ -------------------------------------------------------- 0 Instrument, telescope, and observatory metadata keywords 1-16 2-D spectroscopic image data for amplifer 1-16 17 2-D array of exposure-meter time-series data 18 Guider-camera image 19 Coherent Fiber Bundle data – data cube 20 Solar Telescope data ====== ======================================================== The primary header contains several hundred keywords describing the observation, spectrometer and port configuration, and telemetry from the WIYN and the instrument control software. Only about a dozen of these keywords, near the top of the header, are relevent to end users of the data. The remainder are soley for performing forensics on the data, e.g., to understand long term instrument trends that might be related to temperature changes, or failures in mechanisms. Keywords starting with a **Q** are injested from the NEID target database and should describe the target and exposure requested by the observation PI. Non-**Q** keywords describe the actual state of the instrument/observation. *Important Level 0 Primary Header Keywords:* ======== =============================== ============ ========= ============================================ Keyword Description Units Data type Value -------- ------------------------------- ------------ --------- -------------------------------------------- DATALVL Data Product Base Level N/A UInt 0, 1, 2 OBSERVER Observer name N/A String Queue Observer Name PROGRAM Observing program name N/A String NOAO Prog ID PINAME Program PI Name N/A String NOAO PI OBSTYPE Observation type N/A String “Sci”, “Eng”, “Cal”, or “Tst” OBS-MODE Observing mode N/A String “HR” or “HE” OBJECT Object Name N/A String Human Readable Object Name FILENAME Name of the FITS file N/A String “neidL#_YYYYMMDDThhmmss.fits” DATE File creation date/time, UTC String e.g., “2018-04-26T15:53:13.456” DATE-OBS Date/time at start of exposure UTC String e.g., “2018-04-26T15:53:13.456” OBSJD Julian date of DATE-OBS day Double e.g., 2450000.000000000 EXPTIME Exposure time s Float e.g., 29.43664 FIBERAPR Diameter of fiber on sky arcsec Float e.g., 0.92 SHUTOPEN Shutter-open date/time, UTC String e.g., “2018-04-26T15:53:13.456” SHUTCLSD Shutter-close date/time UTC String e.g., “2018-04-26T15:53:16.456” SCI-OBJ Object in science fiber N/A String CAL-OBJ Object in calibration fiber N/A String SKY-OBJ Object in sky fiber N/A String QTIME Queue data UNIX time UTC String 2018-04-26T15:53:13.456 QOBJECT Queue object name String DR2 NNNNNNNNNNNNNNNNNNN QRA Queue right ascension sexagesimal String HH:MM:SS.sss QDEC Queue declination sexagesimal String DD.mm.ss.sss QEQNX Queue equinox yr Float 2000.0 QEPOCH Queue epoch yr Float e.g., 2015.5 QPLX Queue parallax mas Float 20.45 QPMRA Queue proper motion RA “/yr Float e.g. 0.0345 QPMDEC Queue proper motion DEC “/yr Float e.g. 0.345 QPROG Queue program number N/A String QRV Queue RV for star km/s Float e.g. 45.654 QZ Queue Z for star Float e.g. 0.00000454 QID Queue database ID ULong QSNR Queue Signal-to-Noise Float e.g., 100.0 QSNRWL Queue SNR Ref. Wave-length Angstroms Float e.g., 5500.0 QEXPTIME Queue Exposure Time seconds Float e.g. 30.0 QTEFF Queue Teff K Float e.g. 4600 QSPT Queue Spectral Type String e.g., G5 OBSERVAT Observatory name N/A String KPNO TELESCOP Telescope name N/A String WIYN 3.5m OBSLON Observatory longitude deg Float -111.600562 OBSLAT Observatory latitude deg Float 31.958092 OBSALT Observatory altitude meters Float 2091.0 SEEING Seeing at beginning of exposure arcsec Float *Not populated* AIRMASS Airmass at beginning of exp secZ Float e.g. 1.3 FOCUSPOS Focus position mm Float e.g. 0.0 DOMESTAT Dome shutter status N/A String open, closed TCSTIME TCS-data date/time UTC String e.g., “2018-04-26T15:53:13” TCSRA TCS RA J2000 sexagesimal String e.g. 13:43:42.3456 TCSDEC TCS DEC J2000 sexagesimal String 45:45:45.4545 TCSRAOF TCS RA offset J2000 sexagesimal String TCSDECOF TCS DEC offset J2000 sexagesimal String TCSEQNX TCS equinox yr Float 2000.0 TCSEPOCH TCS epoch yr Float 2000.0 TCSEL TCS elevation angle deg Float e.g. 43.5 TCSZA TCS zenith angle deg Float e.g. 35.5435 TCSAZ TCS azimuth angle deg Float e.g 124.3 TCSHA TCS hour angle sexagesimal String e.g. -2:34:43.3455 TCSLST TCS local sidereal time sexagesimal String e.g. 14:23:43 TCSEQNX TCS equinox yr Float 2000.0 TCSEPOCH TCS epoch yr Float 2000.0 SUNAGL Sun Angle deg Float e.g. -18.5 MOONUP Moon above horizon N/A Bool T/F MOONAGL Target-Moon Angle deg Float 34.56 DEWPOINT Dewpoint Temp C Float e.g., 0.0 ENVDTMP Dome temperature C Float e.g., 5.00 ENVOTMP Outside temperature C Float e.g., 0.0 ENVDHUM Dome relative humidity % Float e.g., 56.6000 ENVOHUM Outside relative humidity % Float e.g., 56.6000 ENVWINDS Wind speed mph Float e.g., 23.4000 ENVWINDD Wind direction angle Float e.g., 133.5000 FILTER1 Filter wheel 1 (Cal ND) UInt e.g. 3 1-6 FILTER2 Filter wheel 2 (ShortPass) UInt 1-2 TURRETA Cal Turret Pos Angle deg Float e.g. 182.2433 TURRETS Cal Turret Source String LFC/Etalon/UNeM/UNeS/ThArM/ThArS/BBFlat/LDLS SHUTTER1 Cal HR shutter status N/A String open/closed SHUTTER2 Cal port shutter status N/A String open/closed SHUTTER3 Exp Meter shutter status N/A String open/closed SHUTTER4 Solar Feed shutter status N/A String open/closed SHUTTER5 2DFlat Feed shutter sta-tus N/A String open/closed THARM_ST ThAr master lamp status N/A String Ton/off THARS_ST ThAr slave lamp status N/A String on/offT UNEM_ST UNe master lamp status N/A String on/offT UNES_ST UNe slave lamp status N/A String on/offT THARM_SN ThAr master lamp serial number N/A String e.g. A045874 THARS_SN ThAr slave lamp serial number N/A String UNEM_SN UNe master lamp serial number N/A String UNES_SN UNe slave lamp serial number N/A String THARM_RT ThAr master run time s String e.g. 34600 THARS_RT ThAr slave run time s String e.g. 34600 UNEM_RT UNe master run time s String e.g. 34600 UNES_RT UNe slave run time s String e.g. 34600 ETALO_RT Etalon/SuperK run time s String e.g. 34600 LFCPCFRT LFC Red Arm run time s UInt e.g. 34600 LFCF0 LFC Filtered Offset Freq Hz String 5045000000.0 LFCF0REF LFC Filtered Offset Freq. Ref Hz String 5045000000.0 LFCFR LFC Filtered Rep Rate Hz String 20000000000.0 LFCFRREF LFC Filtered Rep Rate Ref Hz String 20000000000.0 LFCFLVIS LFC Fiberlock VIS Intensity counts Int e.g., 1500 LFCFLIR LFC Fiberlock IR Intensity counts Int e.g., 1500 LFCPCFSN LFC Red Arm PCF serial number N/A String LFCSTAT LFC Status bitfield 8*Char Hex bitfield (16 fields) LFC Status TIMSARCN TIMS Archon driver build ver. String YYYYDDMM PTCOVER Port Cover Open N/A Bool T/F PTACQFMR Port Acq Flip Mirror N/A String in/out PTACQFOC Port Acq Cam Focus Pos mm Float PTACQIMG Port Acq Cam Last Image N/A String Filename PTADC1 Port ADC 1 Position Degrees Float Angle PTADC2 Port ADC 2 Position Degrees Float Angle PTADCST Port ADC Status N/A String PTCALBM Port Cal Beam Inserter N/A String in/out PTCALSH Port Cal Fiber Shutter N/A String open/closed PTCALX Port Cal Fiber X Translator mm Float coords in mm PTCALY Port Cal Fiber Y Transla-tor mm Float coords in mm PTTTMR Port Tip Tilt Mirror Status N/A UInt Bitstring PTPGI Port Prism/Grating In-serter N/A String Prism, Grating, Open PTGFIMG Port Guide Camera Last Image N/A String Filename PTSCIFBR Port Science Fiber Selector N/A String HE/HR PTHESH Port HE Shutter N/A String open/closed PTHRSH Port HR Shutter N/A String open/closed PTCFBFBP Port CFB Filter Bandpass N/A String e.g. V PTCFBND Port CFB Filter Neutral Density N/A String e.g. Open PTCFBIMG Port CFB Last Image N/A String Filename ======== =============================== ============ ========= ============================================ NEID uses a e2v CCD 290-99 CCD. The serial readout direction of each amplifer is specified in e2v document `A1A-778871 `_. The position of each amplifier within the 9216x9232 pixel array is fully defined in the header of each amplifier extension using hte AMPSEC, DETSEC, and DATASEC keywords. Each amplifier includes three overscan sections, which are defined via the BIASSEC1, BIASSEC2, and BIASSEC3 keywords. The Guider-camera extension includes a single snapshot of the target from the NEID port guide camera, taken immediately before closing the tip-tilt loop. The Coherent Fiber Bundle (CFB) extension contains a time-series of postage stamp images for each of the six coherent fiber bundles. This data is stored as a cube. The postion of each CFB and the time step is stored in the the extension header. The solar telescope extension is populated for solar telescope data only (hdu[0].header['OBJECT']='Sun'. This extension contains a fits table time series of voltages and derived fluxes from the solar telescope pyroheliometer data spanning the NEID integration. Level 1 Data Format ------------------- Level 1 data are derived from Level 0 products, and inherit all header keywords that were present in the Level 0 primary header. Level 0 data extensions are not preserved in Level 1 and Level 2 data (i.e. there is no duplication of the Level 0 data products into reduced products, to minimize archive files sizes). Level 1 file primary headers include the Level 0 primary header, and additional Level 1 keywords appended. Level 1 files store all data in 2D FITS image arrays where the x-axis of the 2D arrays correspond exactly to the x-axis on the detector and the y-axis corresponds to the echelle order index number. **The NEID DRP v1.1.0 extracts 122 spectral orders, corresponding to echelle orders 173-52. Index 0 corresponds to echelle order 173.** ====== ======================================================== HDU # Description ------ -------------------------------------------------------- 0 Level 0 + 1 keywords describing observation 1 1D science fiber flux in electrons 2 1D sky fiber flux in electrons 3 1D calibration fiber flux in electrons 4 1D science fiber variance in electrons^2 5 1D sky fiber variance in electrons^2 6 1D calibration fiber variance in electrons^2 7 Science fiber vacuum wavelength in Angstroms 8 Sky fiber vacuum wavelength in Angstroms 9 Calibration fiber vacuum wavelength in Angstroms ====== ======================================================== *Important Level 1 primary header keywords* ======== =============================== ============ ========= ============================================ Keyword Description Units Data type Value -------- ------------------------------- ------------ --------- -------------------------------------------- E_VER Pipeline version String e.g., v1.1.0 EXTRACT Type of 1D extraction String e.g., sum, flatrelativeoptimal EXTSNR Extracted signal-to-noise FLOAT e.g., 245.35 EXTSNRWL Wavelength of EXTSNR Angstroms FLOAT e.g., 5530.0 WAVECAL Wavelength calib. algorithm String dailymodel0 DRIFTFUN Wavelength calib. drift func. String MDSPFILE Maser wavelength dispersion String Filename DRIFTREF Template file for drift String Filename SSBRVNNN Barycentric Corr for order NNN km/s FLOAT SSBJDNNN Barycentric JD order NNN BJD_TDB DOUBLE SSBZNNN Barycentric Corr for order NNN z FLOAT ======== =============================== ============ ========= ============================================ Level 2 Data Format ------------------- Level 2 data expand upon Level 1, but only exist for astrophysical observation, not calibrations. These files include all of the Level 1 header keywords and data extensions, and add additional Level 2 specific keywords and data. ====== ======================================================== HDU # Description ------ -------------------------------------------------------- 10 1D Telluric Model 11 Reserved for future sky model 12 Cross-correlation functions and RVs 13 Activity Indicator Parameters 14 Processed Exposure Meter Data ====== ======================================================== The HDU 10 data section contains the calculated telluric model, formatted in an array with equivalent shape to the Level 1 spectra. The HDU 12 data section contains the calculated cross-correlation functions. The HDU 12 header contains the measured radial velocities, flux centered time stamps, and order weights. The HDU 13 data section contains a fits table with activity indicators measured for a variety of spectral lines. The HDU 14 data section contains the processed exposure meter time series for each fiber. This is provided for users wishing to reconstruct the flux weighted midpoints. This extension differs signficantly from the exposure meter data in the Level 0 frames due to the complicated cross-talk correction that must be applied to disentangle light from the different fibers. *Important Level 2 HDU12 keywords:* ======== ==================================== ============ ========= ============================================ Keyword Description Units Data type Value -------- ------------------------------------ ------------ --------- -------------------------------------------- CCFSTART Start of RV range km/s FLOAT e.g. 19.8 CCFSTEP Step of RV range km/s FLOAT e.g. 0.25 MAXCPP Max count/pixel of CCF e- FLOAT e.g. 10672.4 CCFLINES Number of lines used ULONG 4907 CCFMASK Mask name and version String e.g. ‘G2_V1’ CCFNOISE Photon noise on CCF RV km/s FLOAT e.g. 0.30 CCFRVNNN Baryc corr. RV for order N km/s FLOAT e.g. 0.30 CCFZNNN Baryc corr. Z for order N FLOAT CCFJDSUM Baryc corr. time for all orders BJD_TDB FLOAT CCFRVSUM Baryc corr. RV for all orders km/s FLOAT e.g. 0.30 DVRMS RV uncertainty for all orders km/s FLOAT e.g. 0.30 CCFJDMOD Baryc corr. time reweighted orders BJD_TDB FLOAT CCFRVMOD Baryc corr. RV for reweighted orders km/s FLOAT e.g. 0.30 DVRMSMOD RV uncertainty for reweighted orders km/s FLOAT CCFWTNNN Flux reweight for order NNN normalized FLOAT 0-1.0 ======== ==================================== ============ ========= ============================================ *Last Updated, 2021-08-09, CFB*