NEID DRP Version History
========================

Versioning Style
++++++++++++++++

The NEID Data Reduction Pipeline uses Semantic Versioning of format
X.Y.Z (Major.Minor.Patch). *The pipeline team does not recommend mixing
processed data from different minor versions. Within a minor version,
data from different patch versions is compatible.*

Upon release of a new Major or Minor version, all NEID data will be
reprocessed. Upon release of a new Patch version, only data affected
by the patched code will be reprocessed.

Current Release and Recommended Usage
+++++++++++++++++++++++++++++++++++++

**The current DRP release is v1.4.2.**

*The DRP Team recommends that data from August 26, 2024 onward
(semester 2024B) only be used with v1.4.1 or later. Data prior to this
can be used in any v1.4.x version.  Users are encouraged to discard
any data from 2024B that was processed with v1.4.0.  All 2024B data
will be reprocessed upon release of v1.4.1. The latest version, v1.4.2,
is a patch release to fix a wavelength calibration bug that prevented 
the processing of a handful of nights. Otherwise, v1.4.1 and v1.4.2 
are functionally equivalent.*

Release History
+++++++++++++++

**Minor version v1.4.0** was released in December 2024 and contained
substantial updates to the algorithms used to derive the wavelength
calibrations, nightly drift measurements, CCFs, and other derived
products.  This verison incorporated vetting the LFC exposures based on 
the quality of the fits to the ensemble of LFC lines. This has increased 
the reliability of the derived wavelength calibrations. 

**Patch version v1.4.1** was released in February 2025 and contains the
following significant changes:

* Handle calibration sequences when only a single LFC passes quality control checks
* Handle a bug where missing flat fields result in zeros for part or all of the extracted spectrum flux
* Changes the blue wavelength edge of the LFC that is used for wavelength solutions and drift correction after installation of the new PCF in August 2024 to be order index 40 (echelle order 134) @ 458nm. In v1.4.0 this was index 30 (echelle order 144) @ 426nm. See below for more details.

**Patch version v1.4.2** was released in April 2025 and contains the
following change:

* Handle HE fiber calibration sequences when one of the HE fibers (e.g., Sky) has all LFC frames fail vetting but the other HE fiber (e.g., Sci) passes vetting. 

For details on versions v1.3.x and earlier, see the `CHANGELOG <https://neid.ipac.caltech.edu/docs/CHANGELOG.md>`_.

  
LFC Calibrations affecting v1.4.0
+++++++++++++++++++++++++++++++++

The week of August 24, 2024, the LFC was upgraded with a new Photonic
Crystal Fiber (PCF) that provided light blueward of 510nm. This
represented a significant upgrade to NEID and facilitated using spectral 
orders significantly bluer than previously available to derive per-night 
wavelength solutions and the nightly instrument drifts. In the future, 
the expanded LFC range will be used to create new master wavelength 
solutions (used during fallback nights) and map the LSF bluer than it 
currently is. DRP v1.4.0 was released using light down to 426nm for the 
LFC-based wavelength solution and drift correction (spectral order index 
30, echelle order 144). However, over the course of the 2024B semester 
the PCF spectral envelope degraded, and light at the bluest wavelengths 
became too faint to be suitable for fitting the individual LFC modes. 
DRP v1.4.1 changed the blue LFC cutoff to 458nm (spectral order index 40, 
echelle order 134).

*The choice of cutoff wavelength affects the overall zero point of the derived RVs*: making a change would introduce a new `RV Era <rveras.html>`_.
Consequently, to avoid this complication we intend to reprocess
all 2024B data using v1.4.1.  Data taken prior to this semester used
an older style PCF that produced light only down to ~510nm, so the
change at the beginning of 2024B necessarily forced the introduction
of a new era.  Data prior to 2024B continues to use the ~510nm cutoff
for LFC-based wavelength solutions and drift measurements. Consequently, such data is agnostic to whether
it was processed with v1.4.0 or v1.4.1 and can safely be intermixed.

*Last Updated: 2025-04-07, DMK*