NEID Data Format

Overview

NEID data products are defined for three different data levels:

• Level 0: Raw data produced by the NEID instrument control system at the WIYN observatory

• Level 1: Extracted, wavelength calibrated spectra

• Level 2: Derived products, including radial velocites, activity indicators, and telluric models

Each of these data levels is a standardized, multi-extension FITS format, and can be read using standard fits tools.

Filenames follow the convention: neidL#_YYYYMMDDThhmmss.fits, where # is 0, 1, or 2 for the data level, and YYYYMMDDThhmmss is the timestamp corresponding to the beginning of the exposure.

Level 0 Data Format

A NEID Level 0 data file is generated for each exposure, either for science or calibration. Each file is in a standardized, multi-extension FITS format, and contains the header-data units (HDUs) listed in Table 1. The primary (zeroth) HDU contains header keywords for instrument, telescope, and observatory metadata, but no data. HDUs 1-16 contain 2-D spectroscopic image data for the 16 instrument readout channels (amplifiers). HDUs 17-20 contain the exposure-meter time-series data, guider image data, coherent-fiber-bundle data, and solar telescope data

HDU #	Description
0	Instrument, telescope, and observatory metadata keywords
1-16	2-D spectroscopic image data for amplifer 1-16
17	2-D array of exposure-meter time-series data
18	Guider-camera image
19	Coherent Fiber Bundle data – data cube
20	Solar Telescope data

The primary header contains several hundred keywords describing the observation, spectrometer and port configuration, and telemetry from the WIYN and the instrument control software. Only about a dozen of these keywords, near the top of the header, are relevent to end users of the data. The remainder are soley for performing forensics on the data, e.g., to understand long term instrument trends that might be related to temperature changes, or failures in mechanisms. Keywords starting with a Q are injested from the NEID target database and should describe the target and exposure requested by the observation PI. Non-Q keywords describe the actual state of the instrument/observation.

Important Level 0 Primary Header Keywords:

Keyword	Description	Units	Data type	Value
DATALVL	Data Product Base Level	N/A	UInt	0, 1, 2
OBSERVER	Observer name	N/A	String	Queue Observer Name
PROGRAM	Observing program name	N/A	String	NOAO Prog ID
PINAME	Program PI Name	N/A	String	NOAO PI
OBSTYPE	Observation type	N/A	String	“Sci”, “Eng”, “Cal”, or “Tst”
OBS-MODE	Observing mode	N/A	String	“HR” or “HE”
OBJECT	Object Name	N/A	String	Human Readable Object Name
FILENAME	Name of the FITS file	N/A	String	“neidL#_YYYYMMDDThhmmss.fits”
DATE	File creation date/time,	UTC	String	e.g., “2018-04-26T15:53:13.456”
DATE-OBS	Date/time at start of exposure	UTC	String	e.g., “2018-04-26T15:53:13.456”
OBSJD	Julian date of DATE-OBS	day	Double	e.g., 2450000.000000000
EXPTIME	Exposure time	s	Float	e.g., 29.43664
FIBERAPR	Diameter of fiber on sky	arcsec	Float	e.g., 0.92
SHUTOPEN	Shutter-open date/time,	UTC	String	e.g., “2018-04-26T15:53:13.456”
SHUTCLSD	Shutter-close date/time	UTC	String	e.g., “2018-04-26T15:53:16.456”
SCI-OBJ	Object in science fiber	N/A	String
CAL-OBJ	Object in calibration fiber	N/A	String
SKY-OBJ	Object in sky fiber	N/A	String
QTIME	Queue data UNIX time	UTC	String	2018-04-26T15:53:13.456
QOBJECT	Queue object name		String	DR2 NNNNNNNNNNNNNNNNNNN
QRA	Queue right ascension	sexagesimal	String	HH:MM:SS.sss
QDEC	Queue declination	sexagesimal	String	DD.mm.ss.sss
QEQNX	Queue equinox	yr	Float	2000.0
QEPOCH	Queue epoch	yr	Float	e.g., 2015.5
QPLX	Queue parallax	mas	Float	20.45
QPMRA	Queue proper motion RA	“/yr	Float	e.g. 0.0345
QPMDEC	Queue proper motion DEC	“/yr	Float	e.g. 0.345
QPROG	Queue program number	N/A	String
QRV	Queue RV for star	km/s	Float	e.g. 45.654
QZ	Queue Z for star		Float	e.g. 0.00000454
QID	Queue database ID		ULong
QSNR	Queue Signal-to-Noise		Float	e.g., 100.0
QSNRWL	Queue SNR Ref. Wave-length	Angstroms	Float	e.g., 5500.0
QEXPTIME	Queue Exposure Time	seconds	Float	e.g. 30.0
QTEFF	Queue Teff	K	Float	e.g. 4600
QSPT	Queue Spectral Type		String	e.g., G5
OBSERVAT	Observatory name	N/A	String	KPNO
TELESCOP	Telescope name	N/A	String	WIYN 3.5m
OBSLON	Observatory longitude	deg	Float	-111.600562
OBSLAT	Observatory latitude	deg	Float	31.958092
OBSALT	Observatory altitude	meters	Float	2091.0
SEEING	Seeing at beginning of exposure	arcsec	Float	Not populated
AIRMASS	Airmass at beginning of exp	secZ	Float	e.g. 1.3
FOCUSPOS	Focus position	mm	Float	e.g. 0.0
DOMESTAT	Dome shutter status	N/A	String	open, closed
TCSTIME	TCS-data date/time	UTC	String	e.g., “2018-04-26T15:53:13”
TCSRA	TCS RA J2000	sexagesimal	String	e.g. 13:43:42.3456
TCSDEC	TCS DEC J2000	sexagesimal	String	45:45:45.4545
TCSRAOF	TCS RA offset J2000	sexagesimal	String
TCSDECOF	TCS DEC offset J2000	sexagesimal	String
TCSEQNX	TCS equinox	yr	Float	2000.0
TCSEPOCH	TCS epoch	yr	Float	2000.0
TCSEL	TCS elevation angle	deg	Float	e.g. 43.5
TCSZA	TCS zenith angle	deg	Float	e.g. 35.5435
TCSAZ	TCS azimuth angle	deg	Float	e.g 124.3
TCSHA	TCS hour angle	sexagesimal	String	e.g. -2:34:43.3455
TCSLST	TCS local sidereal time	sexagesimal	String	e.g. 14:23:43
TCSEQNX	TCS equinox	yr	Float	2000.0
TCSEPOCH	TCS epoch	yr	Float	2000.0
SUNAGL	Sun Angle	deg	Float	e.g. -18.5
MOONUP	Moon above horizon	N/A	Bool	T/F
MOONAGL	Target-Moon Angle	deg	Float	34.56
DEWPOINT	Dewpoint Temp	C	Float	e.g., 0.0
ENVDTMP	Dome temperature	C	Float	e.g., 5.00
ENVOTMP	Outside temperature	C	Float	e.g., 0.0
ENVDHUM	Dome relative humidity	%	Float	e.g., 56.6000
ENVOHUM	Outside relative humidity	%	Float	e.g., 56.6000
ENVWINDS	Wind speed	mph	Float	e.g., 23.4000
ENVWINDD	Wind direction	angle	Float	e.g., 133.5000
FILTER1	Filter wheel 1 (Cal ND)		UInt	e.g. 3 1-6
FILTER2	Filter wheel 2 (ShortPass)		UInt	1-2
TURRETA	Cal Turret Pos Angle	deg	Float	e.g. 182.2433
TURRETS	Cal Turret Source		String	LFC/Etalon/UNeM/UNeS/ThArM/ThArS/BBFlat/LDLS
SHUTTER1	Cal HR shutter status	N/A	String	open/closed
SHUTTER2	Cal port shutter status	N/A	String	open/closed
SHUTTER3	Exp Meter shutter status	N/A	String	open/closed
SHUTTER4	Solar Feed shutter status	N/A	String	open/closed
SHUTTER5	2DFlat Feed shutter sta-tus	N/A	String	open/closed
THARM_ST	ThAr master lamp status	N/A	String	Ton/off
THARS_ST	ThAr slave lamp status	N/A	String	on/offT
UNEM_ST	UNe master lamp status	N/A	String	on/offT
UNES_ST	UNe slave lamp status	N/A	String	on/offT
THARM_SN	ThAr master lamp serial number	N/A	String	e.g. A045874
THARS_SN	ThAr slave lamp serial number	N/A	String
UNEM_SN	UNe master lamp serial number	N/A	String
UNES_SN	UNe slave lamp serial number	N/A	String
THARM_RT	ThAr master run time	s	String	e.g. 34600
THARS_RT	ThAr slave run time	s	String	e.g. 34600
UNEM_RT	UNe master run time	s	String	e.g. 34600
UNES_RT	UNe slave run time	s	String	e.g. 34600
ETALO_RT	Etalon/SuperK run time	s	String	e.g. 34600
LFCPCFRT	LFC Red Arm run time	s	UInt	e.g. 34600
LFCF0	LFC Filtered Offset Freq	Hz	String	5045000000.0
LFCF0REF	LFC Filtered Offset Freq. Ref	Hz	String	5045000000.0
LFCFR	LFC Filtered Rep Rate	Hz	String	20000000000.0
LFCFRREF	LFC Filtered Rep Rate Ref	Hz	String	20000000000.0
LFCFLVIS	LFC Fiberlock VIS Intensity	counts	Int	e.g., 1500
LFCFLIR	LFC Fiberlock IR Intensity	counts	Int	e.g., 1500
LFCPCFSN	LFC Red Arm PCF serial number	N/A	String
LFCSTAT	LFC Status bitfield		8*Char	Hex bitfield (16 fields) LFC Status
TIMSARCN	TIMS Archon driver build ver.		String	YYYYDDMM
PTCOVER	Port Cover Open	N/A	Bool	T/F
PTACQFMR	Port Acq Flip Mirror	N/A	String	in/out
PTACQFOC	Port Acq Cam Focus Pos	mm	Float
PTACQIMG	Port Acq Cam Last Image	N/A	String	Filename
PTADC1	Port ADC 1 Position	Degrees	Float	Angle
PTADC2	Port ADC 2 Position	Degrees	Float	Angle
PTADCST	Port ADC Status	N/A	String
PTCALBM	Port Cal Beam Inserter	N/A	String	in/out
PTCALSH	Port Cal Fiber Shutter	N/A	String	open/closed
PTCALX	Port Cal Fiber X Translator	mm	Float	coords in mm
PTCALY	Port Cal Fiber Y Transla-tor	mm	Float	coords in mm
PTTTMR	Port Tip Tilt Mirror Status	N/A	UInt	Bitstring
PTPGI	Port Prism/Grating In-serter	N/A	String	Prism, Grating, Open
PTGFIMG	Port Guide Camera Last Image	N/A	String	Filename
PTSCIFBR	Port Science Fiber Selector	N/A	String	HE/HR
PTHESH	Port HE Shutter	N/A	String	open/closed
PTHRSH	Port HR Shutter	N/A	String	open/closed
PTCFBFBP	Port CFB Filter Bandpass	N/A	String	e.g. V
PTCFBND	Port CFB Filter Neutral Density	N/A	String	e.g. Open
PTCFBIMG	Port CFB Last Image	N/A	String	Filename

NEID uses a e2v CCD 290-99 CCD. The serial readout direction of each amplifer is specified in e2v document A1A-778871. The position of each amplifier within the 9216x9232 pixel array is fully defined in the header of each amplifier extension using hte AMPSEC, DETSEC, and DATASEC keywords. Each amplifier includes three overscan sections, which are defined via the BIASSEC1, BIASSEC2, and BIASSEC3 keywords.

The Guider-camera extension includes a single snapshot of the target from the NEID port guide camera, taken immediately before closing the tip-tilt loop.

The Coherent Fiber Bundle (CFB) extension contains a time-series of postage stamp images for each of the six coherent fiber bundles. This data is stored as a cube. The postion of each CFB and the time step is stored in the the extension header.

The solar telescope extension is populated for solar telescope data only (hdu[0].header[‘OBJECT’]=’Sun’. This extension contains a fits table time series of voltages and derived fluxes from the solar telescope pyroheliometer data spanning the NEID integration.

Level 1 Data Format

Level 1 data are derived from Level 0 products, and inherit all header keywords that were present in the Level 0 primary header. Level 0 data extensions are not preserved in Level 1 and Level 2 data (i.e. there is no duplication of the Level 0 data products into reduced products, to minimize archive files sizes). Level 1 file primary headers include the Level 0 primary header, and additional Level 1 keywords appended.

Level 1 files store all data in 2D FITS image arrays where the x-axis of the 2D arrays correspond exactly to the x-axis on the detector and the y-axis corresponds to the echelle order index number.

The NEID DRP v1.1.0 extracts 122 spectral orders, corresponding to echelle orders 173-52. Index 0 corresponds to echelle order 173.

HDU #	Description
0	Level 0 + 1 keywords describing observation
1	1D science fiber flux in electrons
2	1D sky fiber flux in electrons
3	1D calibration fiber flux in electrons
4	1D science fiber variance in electrons^2
5	1D sky fiber variance in electrons^2
6	1D calibration fiber variance in electrons^2
7	Science fiber vacuum wavelength in Angstroms
8	Sky fiber vacuum wavelength in Angstroms
9	Calibration fiber vacuum wavelength in Angstroms

Important Level 1 primary header keywords

Keyword	Description	Units	Data type	Value
E_VER	Pipeline version		String	e.g., v1.1.0
EXTRACT	Type of 1D extraction		String	e.g., sum, flatrelativeoptimal
EXTSNR	Extracted signal-to-noise		FLOAT	e.g., 245.35
EXTSNRWL	Wavelength of EXTSNR	Angstroms	FLOAT	e.g., 5530.0
WAVECAL	Wavelength calib. algorithm		String	dailymodel0
DRIFTFUN	Wavelength calib. drift func.		String
MDSPFILE	Maser wavelength dispersion		String	Filename
DRIFTREF	Template file for drift		String	Filename
SSBRVNNN	Barycentric Corr for order NNN	km/s	FLOAT
SSBJDNNN	Barycentric JD order NNN	BJD_TDB	DOUBLE
SSBZNNN	Barycentric Corr for order NNN	z	FLOAT

Level 2 Data Format

Level 2 data expand upon Level 1, but only exist for astrophysical observation, not calibrations. These files include all of the Level 1 header keywords and data extensions, and add additional Level 2 specific keywords and data.

HDU #	Description
10	1D Telluric Model
11	Reserved for future sky model
12	Cross-correlation functions and RVs
13	Activity Indicator Parameters
14	Processed Exposure Meter Data

The HDU 10 data section contains the calculated telluric model, formatted in an array with equivalent shape to the Level 1 spectra.

The HDU 12 data section contains the calculated cross-correlation functions. The HDU 12 header contains the measured radial velocities, flux centered time stamps, and order weights.

The HDU 13 data section contains a fits table with activity indicators measured for a variety of spectral lines.

The HDU 14 data section contains the processed exposure meter time series for each fiber. This is provided for users wishing to reconstruct the flux weighted midpoints. This extension differs signficantly from the exposure meter data in the Level 0 frames due to the complicated cross-talk correction that must be applied to disentangle light from the different fibers.

Important Level 2 HDU12 keywords:

Keyword	Description	Units	Data type	Value
CCFSTART	Start of RV range	km/s	FLOAT	e.g. 19.8
CCFSTEP	Step of RV range	km/s	FLOAT	e.g. 0.25
MAXCPP	Max count/pixel of CCF	e-	FLOAT	e.g. 10672.4
CCFLINES	Number of lines used		ULONG	4907
CCFMASK	Mask name and version		String	e.g. ‘G2_V1’
CCFNOISE	Photon noise on CCF RV	km/s	FLOAT	e.g. 0.30
CCFRVNNN	Baryc corr. RV for order N	km/s	FLOAT	e.g. 0.30
CCFZNNN	Baryc corr. Z for order N		FLOAT
CCFJDSUM	Baryc corr. time for all orders	BJD_TDB	FLOAT
CCFRVSUM	Baryc corr. RV for all orders	km/s	FLOAT	e.g. 0.30
DVRMS	RV uncertainty for all orders	km/s	FLOAT	e.g. 0.30
CCFJDMOD	Baryc corr. time reweighted orders	BJD_TDB	FLOAT
CCFRVMOD	Baryc corr. RV for reweighted orders	km/s	FLOAT	e.g. 0.30
DVRMSMOD	RV uncertainty for reweighted orders	km/s	FLOAT
CCFWTNNN	Flux reweight for order NNN	normalized	FLOAT	0-1.0

Last Updated, 2021-08-09, CFB