NEID Master Files¶
Overview¶
The NEID Data Reduction Pipeline relies on a set of Master files, which contain various static definitions and long term calibration files that are used during pipeline runs. These files are maintained in the git repository https://github.com/SpecSoftware/NEID-MASTERFILES .
Each entry below contains a brief description of the purpose of the master file, followed by the name and version number of the currently used file(s).
Note: The NEID spectrograph restart that was necessitated by the Contreras Fire required that many of the master files be rebuilt from newly obtained calibration sequences. The DRP has been modified to use these files in a TimeDependent mode, whereby the each master file is associated with a time window over which it is valid, and the DPR automatically chooses the correct set of files for the data frame being processed. File names in the documentation below represent the original set of files used. Subsequent TimeDependent files have updated date codes that correspond to when the data frames used to build the file were obtained, not necessarily the window over which that file is valid (i.e., Some master files are used for calibrating files obtained before the underlying calibrations were obtained
Master File Definitions¶
Target Information¶
Star Database File: The star database file maintains a live record of all stellar targets processed by the NEID-DRP. For each target, the positional coordinates (Epoch, Equinox, RA, DEC, Proper Motion, Parallax, Systemic velocity, redshift) and stellar coordinates (Spectral Type, Teff, CCF Mask) are recorded. These values are initially taken from the Q keywords provided by program PIs, and the star file is automatically updated with new targets each time the pipeline is run on new data. Existing targets are not updated, even if the Q data accompanying them differs from the database record. This database serves as the source for all coordinate and stellar parameters used in the pipeline, and ensure that observations of any given star are analyzed identically across epochs. Level2 files have their Q parameters updated to reflect the values in the database and which were used in calculating the barycentric correction and radial velocities. Errors in this database can be corrected by hand, and will be utilized in subsequent pipeline reprocessing of data.
neidMaster_StarDB_v000.config
Detector Information¶
Bad Pixel Mask: 2D image indicating location of known bad pixels.
neidMaster_BadPixelMask20210430.fits
Calibration Frame Validation¶
Illumination Check Thresholds: Text file containing thresholds used for testing whether calibration frames are within expected flux levels.
neidMaster_CalibrationChecker20210219.txt
Flat Fielding and Scattered Light¶
2D Flat Template: 2D flat lamp illumination pattern template used for normalization of 2D flats. The epoch of this flat fixes our relative chromatic QE variation corrections.
neidMaster_illumination20210406.fits
2DScatter Templates: There are 2D high S/N ratio templates of bright light sources in one fiber at a time. They are used for scaling and subtracting the scattered light from these sources while espoused simultaneously with other sources. These files also have corresponding pixel mask files which labels the pixels to use for scaling calculation.
neidMaster_HESci_2DScatter_FlatBB_20210515_mask.fits
neidMaster_HESci_2DScatter_LDLS_20210515_mask.fits
neidMaster_HESky_2DScatter_FlatBB_20210515_mask.fits
neidMaster_HESky_2DScatter_LDLS_20210515_mask.fits
neidMaster_HRCal_2DScatter_Etalon_20210126.fits
neidMaster_HRCal_2DScatter_Etalon_20210126_mask.fits
neidMaster_HRCal_2DScatter_EtalonSP_20210507.fits
neidMaster_HRCal_2DScatter_EtalonSP_20210507_mask.fits
neidMaster_HRCal_2DScatter_FlatBB_20210213.fits
neidMaster_HRCal_2DScatter_FlatBB_20210213_mask.fits
neidMaster_HRCal_2DScatter_LDLS_20210213.fits
neidMaster_HRCal_2DScatter_LDLS_20210213_mask.fits
neidMaster_HRSci_2DScatter_Etalon_20210313.fits
neidMaster_HRSci_2DScatter_Etalon_20210313_mask.fits
neidMaster_HRSci_2DScatter_FlatBB_20210213.fits
neidMaster_HRSci_2DScatter_FlatBB_20210213_mask.fits
neidMaster_HRSci_2DScatter_LDLS_20210213.fits
neidMaster_HRSci_2DScatter_LDLS_20210213_mask.fits
neidMaster_HRSci_2DScatter_Star_20201222.fits
neidMaster_HRSci_2DScatter_Star_20201222_mask.fits
neidMaster_HRSci_2DScatter_Star_20210322.fits
neidMaster_HRSci_2DScatter_Star_20210322_mask.fits
neidMaster_HRSci_2DScatter_Sun_20210218.fits
neidMaster_HRSci_2DScatter_Sun_20210218_mask.fits
neidMaster_HRSky_2DScatter_FlatBB_20210213.fits
neidMaster_HRSky_2DScatter_FlatBB_20210213_mask.fits
neidMaster_HRSky_2DScatter_LDLS_20210213.fits
neidMaster_HRSky_2DScatter_LDLS_20210213_mask.fits
SmoothLampSED: These are the smooth templates of the continuum lamp spectrum. They are generated by Gaussian Process modelling of the sum extraction of the 2D Flat corrected continuum lamp spectrum.
neidMaster_HR_SmoothLampSED_20210322_v02.fits
neidMaster_HE_SmoothLampSED_20210322_v02.fits
Echellogram Definition¶
BeamTrace: These are the echelle order aperture beam traces used for extraction.
neidMaster_BeamTrace20201114HESci_v1.txt
neidMaster_BeamTrace20201114HESky_v1.txt
neidMaster_BeamTrace20201114HRCal_v1.txt
neidMaster_BeamTrace20201114HRSci_v1.txt
neidMaster_BeamTrace20201114HRSky_v1.txt
FiberMask: These are binary masks which masks the pixels along the aperture of each fiber on neid data. These are used for identifying pixels for scattered light modelling.
neidMaster_FiberMask20201114HESci_v1.fits
neidMaster_FiberMask20201114HESky_v1.fits
neidMaster_FiberMask20201114HRCal_v1.fits
neidMaster_FiberMask20201114HRSci_v1.fits
neidMaster_FiberMask20201114HRSky_v1.fits
Free Spectral Range Mask: Binary mask file which labels the free spectral range of each echelle order. This mask represents the as-built spectrograph and was measured from flat field data.
neidMaster_FSR_Mask20210331_v001.fits
A csv file with the start and end of the FSR for each order can be found
here. The columns are:“order”: zero-indexed spectral order.
“fsr_start”: first pixel in the order within the FSR
“fsr_end”: last pixel in the order within the FSR
Orders without a defined FSR have nan values for “fsr_start” and “fsr_end”.
LDLS Fringe Correction¶
LDLS Fringe Model: Template LDLS Fringe Model, which is well-modeled with a sinusoid function.
neidMaster_LDLSfringepattern_20210518.fits
Exposure Meter¶
Wavelength Solutions: The wavelength solution of the exposure meter traces
neidMaster_ExpMeterWavlSolutions_v1.fits
Optical Cross Talk: The Exposure meter optical cross talk coefficient matrix
neidMaster_OpticalCrossTalk_20201125_v1.fits
Average Bias: Bias of a typical exposure meter frame
neidMaster_AvgExpMeterBiasFullFrame_20210117_v1.fits
Wavelength Solution¶
Master wavelength calibration: These are based on a high-SNR spectrum of ThAr that spans the full NEID bandpass. It does not contain refinements based on the more information-rich LFC or FP sources, and so is known to have some systematics, particularly towards the extremes of spectral orders.
neidMaster_HR_Wavelength20210218_v003.fits
neidMaster_HE_Wavelength20210318_v001.fits
Master line lists for the LFC and the Etalon: These linelists locate the approximate positions (in pixels) for each mode of the LFC and Etalon sources. HE and HR modes have different files; the small offset between HR and HE fibers in the fiber head produces a small shift on the focal plane that must be accounted for.
neidMaster_LFCMode20201216_v001.npy
neidMaster_candidateHE_LFCModes20210331T005340_v002.npy
neidMaster_EtalonPeaks20201113_v001.npy
neidMaster_candidateHE_EtalonPeaks20210131T005337_v001.npy
Master set of precise LFC mode positions: These are used for measuring the drift of the instrument in shifting the ThAr wavelength calibration in the initial construction of the nightly (or daily) wavelength calibration.
neidMaster_LFCMode20201216_v001.npy
LFC mode positions at time of master wavelength calibration: These files have the approximate pixel positions for each LFC mode at the epoch of the ThAr-based master wavelength calibration (described above). This is used to measure a day or night’s zero-point drift relative to the master wavelength solution. There are separate files for HR and HE mode. New versions of this file are generated when a new master wavlength solution is made (such as pre and post fire).
neidMaster_peakinfos_lfc_combined_20210119_forThArHR_v002.npy
neidMaster_peakinfos_lfc_combined_20221214_forThArHR_v001.npy
neidMaster_peakinfos_lfc_combined_20210204_forThArHE_v001.npy
neidMaster_peakinfos_lfc_combined_20221205_forThArHE_v001.npy
LFC F0 offsets: A time log recording known jumps in the LFC, typically due being locked to the wrong mode of the seed comb. A one comb mode jump corresponds to 250 MHz.
neidMaster_LFC_f0_offsets.txt
Master template drift file: This traces the typical drift pattern of the instrument, which is dominated by loading from the daily LN2 fill.
neidMaster_DailyDriftTemplate_v0.1.fits
System Throughput¶
NEID + WIYN/Solar Telescope Instrument Response: HR and HE files for Sci and Sky fiber instrument response (throughput) derived by observing a CALSPEC flux standard in units of electrons per specific flux unit (erg/s/cm2/A).
neidMaster_InstrumentResponse_HR_20210323_v6.fits
neidMaster_InstrumentResponse_HE_20210323_v6.fits
neidMaster_SolarInstrumentResponse_HR_20210323_v6.fits
CCF Weights File: (neidMaster_CCFWeights[HR,Solar].config) Contains a static set of weights used for reweighting the CCFs. These were calculated by multiplying the system throughput by blackbody functions corresponding to the temperatures of each CCF mask.
neidMaster_CCFWeightsHR.config
neidMaster_CCFWeightsSolar.config
Telluric Modeling¶
Telluric Grid: Each telluric grid master file contains a 1-d array of telluric absorption values corresponding to each wavelength specificed in the master telluric grid wavelength file. The master telluric grid wavelength file is a 1-d array of wavelength values (angstrom). The telluric grid models are calculated at set points in zenith angle (degrees) and precipitable water vapor (mm). At each grid point there are two absorption models: one with the continuum telluric absorption and one with the line telluric absorption. The two types of absorption have been separated because only the line absorption is necessary for fitting the precipitable water vapor value of each on-sky science frame. The full telluric absorption model spectrum can be obtained by multiplying these two models together.
neidMaster_TelluricModel_ContinuumAbsorption_ZXXPWVYY_20230726_v001.npy (XX: zenith angle in degrees, YY: precipitable water value in mm)
neidMaster_TelluricModel_LineAbsorption_ZXXPWVYY_20230726_v001.npy (XX: zenith angle in degrees, YY: precipitable water value in mm)
neidMaster_TelluricModel_Wavelengths_20230726_v001.npy
Variable line spread function parameterization: (neidMaster_LSFParameterCoefficients20230824HR[Cal,Sci,Sky]_v1.txt) Contains information about the parameterization of the variable line spread function from order to order across the spectrum. The LSF is defined as a profile defined by N parameters (currently a top hat convolved gaussian, which is defined by a gaussian FWHM and the box width). This master file is in the form of the echelle order aperture beam trace files. For each order with a defined LSF profile, the master file contains polynomial coefficients to fits of the profile parameters as a function of dispersion pixel location. There is also a GLOBAL entry in the master file which contains information about the LSF profile parameterization. Currently only the HR Sci master file is being used.
neidMaster_LSFParameterCoefficients20230824HRCal_v1.txt
neidMaster_LSFParameterCoefficients20230824HRSci_v1.txt
neidMaster_LSFParameterCoefficients20230824HRSky_v1.txt
Last Updated: 2023-08-29, DMK
