Missions

_read_fits_event(fits_path::String)

Reads the standard columns for timing analysis ("TIME", "PI", "GTI") from a FITS file, returns InstrumentData type filled with the relevant data

_read_fits_hdu(fits_file::FITS, hdu_id::String; cols="auto")

Reads the HDU hdu_id from the loaded FITS file fits_file, returns the HDU data

Cannot read BitArray type columns due to FITSIO limitations

_save_cl_feather(feather_dir::String, instrument_name::Union{String,Symbol},
    fits_events_df::DataFrame, fits_gtis_df::DataFrame, fits_meta_df::DataFrame)

Due to Feather file restrictions, cannot save all the event and GTI data in one, so they are split up into three files: events, gtis, and meta. The meta file contains just the mission name, obsid, and observation start and stop times

read_cl(mission_name::Symbol, append_df::DataFrames.DataFrame, obsid::String; overwrite=false)

Calls master_query() to get the obs_row, then calls readcl(missionname::Symbol, obs_row::DataFrames.DataFrame; overwrite=false)

read_cl(mission_name::Symbol, obs_row::DataFrames.DataFrame; overwrite=false)

Attempts to read saved (feather) data, if none is found then the read_cl_fits function is ran and the data is saved with _save_cl_feather for future use

read_cl(mission_name::Symbol, obsid::String; overwrite=false)

Calls master_a(), then calls read_cl(mission_name::Symbol, append_df::DataFrames.DataFrame, obsid::String; overwrite=false)

read_cl_fits(mission_name::Symbol, obs_row::DataFrames.DataFrame)

Reads in FITS data for an observation, returns a Dict{Symbol,InstrumentData}, with the symbol as the instrument name. So instrument_data[:XTI] works for NICER, and either instrument_data[:FPMA] or instrument_data[:FPMB] work for NuSTAR

_group_return(data::BinnedData)

Uses the _group_select function to find the group each GTI belongs in

Using these groups, splits sections of light curve up into an group, then creates a new BinnedData for just the lightcurve of that one group, finally returns a Dict{Int64,JAXTAM.BinnedData} where each Int64 is for a different group

_group_select(data::BinnedData)

Checks the differennce in time between GTIs, if the difference is under a group_period (128 [sec] by default) then the GTIs are in the same group

Done as data frequently has small breaks between GTIs, even though there is no significant gap in the lightcurve. Groups are used during plotting, periodograms, and when grouping/averaging together power spectra

Returns GTIs with an extra :group column added in

_lc_bin(event_times::Array{Float64,1}, bin_time::Union{Float64,Int64}, time_start::Union{Float64,Int64}, time_stop::Union{Float64,Int64})

Bins the event times to bins of bin_time [sec] lengths

Performs binning out of memory for speed via OnlineStats.jl Hist function

Returns a range of times, with associated counts per time

_lc_filter_energy(event_times::Array{Float64,1}, event_energies::Array{Float64,1}, good_energy_max::Float64, good_energy_min::Float64)

Optionally filters events by a custom energy range, not just that given in the RMF files for a mission

Returns filtered event times and energies

_lc_read(lc_dir::String, instrument::Symbol, bin_time)

Reads the split files saved by _lcurve_save, combines them to return a single BinnedData type

_lcurve(instrument_data::InstrumentData, bin_time::Union{Float64,Int64})

Takes in the InstrumentData and desired bin_time

Runs functions to perform extra time (_lcurve_filter_time) and energy (_lc_filter_energy) filtering

Runs the binning (_lc_bin) function, then finally _group_select to append group numbers to each GTI

Returns a BinnedData lightcurve

_lcurve_filter_time(event_times::Arrow.Primitive{Float64}, event_energies::Arrow.Primitive{Float64},

Largely not used, as the GTI filtering is enough to deal with out-of-time-range events, and manually filtering those events out early is computationally intensive

Function takes in event times and energies, then filters any events outside of the start and stop times

Optionally performs early filtering to remove low count (under 1/sec) GTIs, disabled by default as this is performed later anyway

Returns array of filtered times, enegies, and GTIs

_lcurve_save(lightcurve_data::BinnedData, lc_dir::String)

Takes in BinnedData and splits the information up into three files, meta, gtis, and data

Saves the files in a lightcurve directory (/JAXTAM/lc/$bin_time/*) per-instrument

lcurve(mission_name::Symbol, obs_row::DataFrame, bin_time::Number; overwrite=false)

Main function, handles all the lightcurve binning

Runs binning functions if no files are found, then saves the generated BinnedData

Loads saved files if they exist

Returns Dict{Symbol,BinnedData}, with the instrument as a symbol, e.g. lc[:XTI] for NICER, lc[:FPMA]/lc[:FPMB] for NuSTAR

lcurve(mission_name::Symbol, obsid::String, bin_time::Number; overwrite=false)

Runs master_query to find the desired obs_row for the observation

Calls main lcurve(mission_name::Symbol, obs_row::DataFrame, bin_time::Number; overwrite=false) function

_read_calibration(pis::Union{Array,Arrow.Primitive{Int16}}, path_rmf::String)

Loads the RMF calibration data, creates PI channels for energy conversion

Channel bounds are the average of the min and max energy range

_read_calibration(pis::Union{Array,Arrow.Primitive{Int16}}, mission_name::Symbol)

Loads the RMF path from the mission configuration file, then calls _read_calibration(pis::Union{Array,Arrow.Primitive{Int16}}, path_rmf::String)

_read_rmf(path_rmf::String)

Reads an RMF calibration file (from HEASARC caldb), loads in energy bands and PI channels for use when filtering events out of a good energy range

Returns the PI channels, and the min/max good energy ranges

_read_rmf(mission_name::Symbol)

Calls _read_rmf(path_rmf) using the path_rmf loaded from a mission configuration file

calibrate(mission_name::Symbol, append_df::DataFrames.DataFrame, obsid::String)

Calls master_query to load in the relevant obs_row

Calls and returns calibrate(mission_name::Symbol, obs_row::DataFrames.DataFrame)

calibrate(mission_name::Symbol, obs_row::DataFrames.DataFrame)

Loads in the calibrated event data, as well as the mission calibration RMF file, then filters the events by the energy ranges/PI channels in the RMF file

Saves the calibrated files as a calib.feather if none exists

Loads calib.feater file if it does exist

Returns a calibrated (filtered to contain only good energies) InstrumentData type

calibrate(mission_name::Symbol, obsid::String)

Calls master_a to load in the master table

Calls and returns calibrate(mission_name::Symbol, append_df::DataFrames.DataFrame, obsid::String)

_gtis(lc::BinnedData)

Calls _lc_filter_gtis using BinnedData input

_gtis_load(gti_dir, instrument, bin_time)

Loads and parses the _meta and _gti files, puts into a GTIData constructor, returns Dict{Int,GTIData}

_gtis_save(gtis, gti_dir::String)

Splits up the GTI data into a _meta.feather file containing non-array variables for each GTI (index, start/stop times, etc...) and multiple _gti.feather files for each GTI containing the counts and times

_lc_filter_gtis(binned_times::StepRangeLen, binned_counts::Array{Int,1}, gtis::DataFrames.DataFrame, mission::Symbol, instrument::Symbol, obsid::String; min_gti_sec=16)

Splits the lightcurve (count) data into GTIs

First, removes GTIs under min_gti_sec, then puts the lightcurve data into a Dict{Int,GTIData}, with the key as the index of the GTI

gtis(mission_name::Symbol, obs_row::DataFrames.DataFrame, bin_time::Number; overwrite=false)

Handles file management, checks to see if GTI files exist already and loads them, if files do not exist then the _gits function is ran, then the data is saved

gtis(mission_name::Symbol, obsid::String, bin_time::Number; overwrite=false)

Runs master_query to load the obs_row for a given obsid, runs main gits function