NEID Data FAQ

This page contains a collection of Frequently Asked Questions that have been submitted to NOIRLab or NExScI by NEID users. Feel free to submit new questions that you think are of general interest to the NEID community.

Accessing Data:

Q: How do I give collaborators access to my data?

Email: neid-help@ipac.caltech.edu

Q: Do I need an account to download public data from the website or PyNEID API?

No. Data with no proprietary period (e.g. Solar, Standard Stars, Calibrations), or data who’s proprietary period has expired can be accessed without an account.

Q: How can I retrieve a subset of solar data, selected by date range?

Use the pyneid interface

from pyneid.neid import Neid
Neid.query_datetime('solarl2', '2021-07-06 00:00:00/2021-07-07 00:00:00', outpath='meta.tbl', format='ipac')
Neid.download(./meta.tbl', 'solarl2', 'ipac', '/Users/bfulton/Downloads/', start_row=0, end_row=999)

Data Processing:

Q: How quickly is data processed by NExScI?

Night Level 0 data is transferred from Kitt Peak to NExScI following the morning cals (approx 9:30 MST). Solar Level 0 data is transferred following the evening cals (approx. 19:00 MST). Processed data is generally available in the NExScI archive within 24 hours of these transfer windows, although may occasionally be delayed due to problems with the data transfer or maintenance on the HPC compute cluster or the NExScI archive computer.

Evaluating Data:

Q: In a series of RV observations for a target, one epoch is discrepant by tens of m/s. Why?

The NEID DRP wavelength solution module is highly dependent on the Laser Frequency Comb (LFC) calibration data. If the comb is unavailable, this module will fall-back to a more primitive algorithm. See Fallback and Wavecal Quality for more details. This result of this fallback is that the RVs are on a different reference frame from nominal observations.

Beginning in DRPv1.4.0, each LFC frame is vetted by the pipeline to assess the stability of the hardware during the exposure. Frames determined to have been taken when the LFC was unstable are rejected. This has resulted in substantial improvements in overall wavelength solution stability, and has resulted in corrections to the RVs for many of the observations that were previously discrepent. Many nights that previously fell-back to different wavelength algorithms have been recovered. However, it is possible that some nights that previously appeared to process correctly will now instead fall-back because their LFC frames were deemed unstable. This is a desirable result: the RVs for those nights were previously contaminated by systematic errors that were very difficult for the end user to detect, whereas now the fallback state is obvious. Users are encouraged to check the DRIFTFUN keyword in Level 1 and Level 2 to assess the wavelength algorithm utilized by the pipeline for their data.

Q: Are NEID wavelengths in vacuum or air?

The NEID DRP produces Level 1 and Level 2 data with vacuum wavelengths.

Q: How did the Contreras Fire on Kitt Peak affect NEID and NEID data?

On June 14, 2022, the Contreras Fire, which had started Baboquivari mountain range the previous week, began crossing the valley south of Kitt Peak National Observatory. The NEID instrument team and WIYN staff began preparations to make the spectrometer, telescope, and other facilities safe. This involved stopping all LN2 fills and bringing the instrument to ambient temperature. All electrical systems, including the instrument computer and the laser frequency comb, were powered down, to place the system in a safe condition to wait out the fire and an extended subsequent power outage. On June 16, final evacutions of summit technical staff were completed. In the early morning hours of June 17, the fire crossed the southwest ridge, and approached the main summit. All electrical power to the main summit was terminated. Happily, the fire teams were able to successfully defend WIYN and the rest of the main summit, and no serious damage was suffered to WIYN or the NEID spectrometer.

Line power was restored to WIYN on October 17, 2022, and NEID restart operations began on October 18. Over the following month the spectrometer was restored to nominal operating conditions and on-sky observations resumed under shared risk status in mid November. Thermally cycling the instrument resulted in a few micron shift in the position of the echellogram on the NEID CCD, which necessitated rebuilding many of the master files that are core pieces of the NEID DRP. This event is now captured as a distinct NEID RV Eras reference frame break.

Q: Should I fit for an offset between pre and post Contreras Fire data?

Yes. There is an offset between pre and post fire data (roughly Oct 18, 2022) that should be fit as an additional constant parameter in your orbit fits. This offset if roughly 10 m/s, but will vary slightly from star to star.

Q: Can I combine RV data processed with different NEID DRP versions (e.g. v1.2.1 with v1.3.0)?

No. Version changes often include major changes that are likely to affect the RV data, more often in the form of RV offsets between versions. It is safe though to combine RV data from different sub-versions (e.g. v1.2.0 with v1.2.2).

Target Information:

Q: I am continuing to observe a target that was originally submitted in a previous semester. I have updated the target parameters (e.g. coords, spectral type, systemic RV, etc.) in the NOIRLab Q-Portal, but the pipeline is still using the old parameters. Why?

The DRP tracks targets observed (but not programs that observed them) in an internal database. This ensures that all reductions of any given target are handled identically across semesters and programs. A target is entered into this database when it is first observed. Future observations will default to the original input parameters. These parameters can be determined by examining the ‘Q’ header keywords in Level 1 and Level 2 data, which are updated to reflect the values actually used. The values specified by the PI are contained in Level 0. If a target in the DRP star database needs to be updated, please contact the NExScI or NOIRLab, who will forward the request to the DRP team. The archival spectra will then be updated during the next full data reprocessing.

Time Accounting:

Q: Why do I see exposures in the NExScI archive that do not meet my SNR requirement?

All exposures that are sent to NExScI are processed and archived. The DRP computes the SNR of each frame, and NExScI checks that against the requested value. Exposures that fail this check are retained, but NOIRLab is notified to return the time to the PIs program. Some exposures that will clearly not meet the PIs requirements (e.g., clouds interrupt observing only a few seconds after an exposure starts) are rejected immediately by the observer and never sent to NExScI.

Last Updated: 2023-1-31, CFB