diviner package

Submodules

diviner.NER_calculation module

diviner.ana_utils module

diviner.bintools module

diviner.bintools.cols_to_descriptor(iterable)

diviner.calib module

diviner.compare_calibs module

diviner.convert_to_h5 module

diviner.data_prep module

diviner.data_prep.cutoff_msec(ts)

diviner.data_prep.define_sdtype(df)

diviner.data_prep.format_time(intime)

diviner.data_prep.generate_date_index(dataframe)

Parse date fields/columns with pandas date converter parsers.

Parse the date columns and create a date index from it In: pandas dataframe read in from diviner div38 data Out: DatetimeIndex

diviner.data_prep.get_bb_selector(df)

Create dataframe selector for pointing limits of divconstants ‘c’ file

diviner.data_prep.get_st_selector(df)

Create dataframe selector for pointing limits of divconstants ‘c’ file

diviner.data_prep.get_stowed_selector(df)

diviner.data_prep.get_sv_selector(df)

Create dataframe selector for pointing limits of divconstants ‘c’ file

diviner.data_prep.index_by_time(df, drop_dates=True)

must return a new df because the use of drop

diviner.data_prep.parse_divdata_times(df, drop_dates=True)

diviner.data_prep.prepare_data(df_in)

Declare NaN value and pad nan data for some.

diviner.div_l1a_fix module

diviner.divcalib module

class diviner.divcalib.OptionParser(argv)

Bases: object

Manage command-line options for divcalib

parse_timestr(timestr)

diviner.divcalib.main()

diviner.divcalib.parse_options()

diviner.divcalib.usage()

diviner.divconstants module

diviner.divgui module

diviner.divtweet module

diviner.exceptions module

exception diviner.exceptions.DivCalibError

Bases: Exception

Base class for exceptions in this module.

exception diviner.exceptions.DivTimeLengthError(tstr, expected)

Bases: Exception

exception diviner.exceptions.L1ANotFoundError

Bases: OSError

exception diviner.exceptions.MeanTimeCalcError(t)

Bases: diviner.exceptions.DivCalibError

exception diviner.exceptions.NoOfViewsError(view, wanted, value, where)

Bases: diviner.exceptions.DivCalibError

exception diviner.exceptions.RDRR_NotFoundError

Bases: OSError

exception diviner.exceptions.RDRS_NotFoundError

Bases: OSError

exception diviner.exceptions.UnknownMethodError(method, location)

Bases: diviner.exceptions.DivCalibError

exception diviner.exceptions.ViewLengthError(view, expected, received)

Bases: diviner.exceptions.DivCalibError

Exception for view length (9 ch * 21 det * 80 samples = 15120).

SV/BBV/ST_LENGTH_TOTAL defined in divconstants.

exception diviner.exceptions.WrongTypeError(type_required, type_current)

Bases: diviner.exceptions.DivCalibError

diviner.file_utils module

diviner.fix_something module

diviner.fix_something.fixing_columns(fname)

diviner.flags module

class diviner.flags.CalibFlag(value=0)

Bases: diviner.flags.Flag

bits = {'constants_only': 8, 'interp_marker_oob': 1, 'nearest_marker': 2, 'nearest_marker_oob': 4}

bits_data = (('interp_marker_oob', 1), ('nearest_marker', 2), ('nearest_marker_oob', 4), ('constants_only', 8))

t = ('constants_only', 8)

class diviner.flags.Flag(value=0, dic=None)

Bases: object

Helper class to deal with control words or flags.

Bit setting and checking methods are implemented.

check_bit(bit)

clear_bit(bit)

set_bit(bit)

set_members()

class diviner.flags.GeomFlag(value=0)

Bases: diviner.flags.Flag

bits = {'ephem_def': 8, 'ephem_none': 128, 'ephem_pred': 32, 'ephem_res': 2, 'pointing_def': 4, 'pointing_none': 64, 'pointing_pred': 16, 'pointing_res': 1}

bits_data = (('pointing_res', 1), ('ephem_res', 2), ('pointing_def', 4), ('ephem_def', 8), ('pointing_pred', 16), ('ephem_pred', 32), ('pointing_none', 64), ('ephem_none', 128))

t = ('ephem_none', 128)

class diviner.flags.MiscFlag(value=0)

Bases: diviner.flags.Flag

bits = {'abnormal_instrument_state': 8, 'abnormal_instrument_temp_drift': 16, 'ch1_saturation': 64, 'eclipse': 2, 'moving': 128, 'noise': 32, 'rfu0': 1, 'turn_on_transient': 4}

bits_data = (('rfu0', 1), ('eclipse', 2), ('turn_on_transient', 4), ('abnormal_instrument_state', 8), ('abnormal_instrument_temp_drift', 16), ('noise', 32), ('ch1_saturation', 64), ('moving', 128))

t = ('moving', 128)

diviner.formats module

class diviner.formats.Formatter

Bases: object

flag = 'b'

flags = ['o', 'v', 'i', 'm', 'q', 'p', 'e', 'z', 't', 'h', 'd', 'n', 's', 'a', 'b']

format_list = [(0, 'date', '"{:>11s}"'), (1, 'utc', '"{:>12s}"'), (2, 'jdate', '{:17.9f}'), (3, 'orbit', '{:5d}'), (4, 'sundist', '{:7.5f}'), (5, 'sunlat', '{:8.5f}'), (6, 'sunlon', '{:9.5f}'), (7, 'sclk', '{:016.5f}'), (8, 'sclat', '{:9.5f}'), (9, 'sclon', '{:9.5f}'), (10, 'scrad', '{:11.5f}'), (11, 'scalt', '{:11.5f}'), (12, 'el_cmd', '{:7.3f}'), (13, 'az_cmd', '{:7.3f}'), (14, 'af', '{:4d}'), (15, 'orientlat', '{:9.5f}'), (16, 'orientlon', '{:9.5f}'), (17, 'c', '{:1d}'), (18, 'det', '{:3d}'), (19, 'vlookx', '{:9.6f}'), (20, 'vlooky', '{:9.6f}'), (21, 'vlookz', '{:9.6f}')]

format_nan = [(0, 'date', '"{:>11s}"'), (1, 'utc', '"{:>12s}"'), (2, 'jdate', '{:17.9f}'), (3, 'orbit', '{:5d}'), (4, 'sundist', '{:7.5f}'), (5, 'sunlat', '{:8.5f}'), (6, 'sunlon', '{:9.5f}'), (7, 'sclk', '{:016.5f}'), (8, 'sclat', '{:9.5f}'), (9, 'sclon', '{:9.5f}'), (10, 'scrad', '{:11.5f}'), (11, 'scalt', '{:11.5f}'), (12, 'el_cmd', '{:7.3f}'), (13, 'az_cmd', '{:7.3f}'), (14, 'af', '{:4d}'), (15, 'orientlat', '{:9.5f}'), (16, 'orientlon', '{:9.5f}'), (17, 'c', '{:1d}'), (18, 'det', '{:3d}'), (19, 'vlookx', '{:9.6f}'), (20, 'vlooky', '{:9.6f}'), (21, 'vlookz', '{:9.6f}'), (22, 'radiance', '{:10.1f}'), (23, 'tb', '{:8.1f}'), (24, 'clat', '{:9.1f}'), (25, 'clon', '{:9.1f}'), (26, 'cemis', '{:9.1f}'), (27, 'csunzen', '{:9.1f}'), (28, 'csunazi', '{:9.1f}'), (29, 'cloctime', '{:8.1f}'), (30, 'cphase', '{:9.5f}'), (31, 'roi', '{:4d}'), (32, 'o', '{:1d}'), (33, 'v', '{:1d}'), (34, 'i', '{:1d}'), (35, 'm', '{:1d}'), (36, 'q', '{:1d}'), (37, 'p', '{:1d}'), (38, 'e', '{:1d}'), (39, 'z', '{:1d}'), (40, 't', '{:1d}'), (41, 'h', '{:1d}'), (42, 'd', '{:1d}'), (43, 'n', '{:1d}'), (44, 's', '{:1d}'), (45, 'a', '{:1d}'), (46, 'b', '{:1d}')]

format_nominal = [(0, 'date', '"{:>11s}"'), (1, 'utc', '"{:>12s}"'), (2, 'jdate', '{:17.9f}'), (3, 'orbit', '{:5d}'), (4, 'sundist', '{:7.5f}'), (5, 'sunlat', '{:8.5f}'), (6, 'sunlon', '{:9.5f}'), (7, 'sclk', '{:016.5f}'), (8, 'sclat', '{:9.5f}'), (9, 'sclon', '{:9.5f}'), (10, 'scrad', '{:11.5f}'), (11, 'scalt', '{:11.5f}'), (12, 'el_cmd', '{:7.3f}'), (13, 'az_cmd', '{:7.3f}'), (14, 'af', '{:4d}'), (15, 'orientlat', '{:9.5f}'), (16, 'orientlon', '{:9.5f}'), (17, 'c', '{:1d}'), (18, 'det', '{:3d}'), (19, 'vlookx', '{:9.6f}'), (20, 'vlooky', '{:9.6f}'), (21, 'vlookz', '{:9.6f}'), (22, 'radiance', '{:10.4f}'), (23, 'tb', '{:8.3f}'), (24, 'clat', '{:9.5f}'), (25, 'clon', '{:9.5f}'), (26, 'cemis', '{:9.5f}'), (27, 'csunzen', '{:9.5f}'), (28, 'csunazi', '{:9.5f}'), (29, 'cloctime', '{:8.5f}'), (30, 'cphase', '{:9.5f}'), (31, 'roi', '{:4d}'), (32, 'o', '{:1d}'), (33, 'v', '{:1d}'), (34, 'i', '{:1d}'), (35, 'm', '{:1d}'), (36, 'q', '{:1d}'), (37, 'p', '{:1d}'), (38, 'e', '{:1d}'), (39, 'z', '{:1d}'), (40, 't', '{:1d}'), (41, 'h', '{:1d}'), (42, 'd', '{:1d}'), (43, 'n', '{:1d}'), (44, 's', '{:1d}'), (45, 'a', '{:1d}'), (46, 'b', '{:1d}')]

format_solartarget = [(0, 'date', '"{:>11s}"'), (1, 'utc', '"{:>12s}"'), (2, 'jdate', '{:17.9f}'), (3, 'orbit', '{:5d}'), (4, 'sundist', '{:7.5f}'), (5, 'sunlat', '{:8.5f}'), (6, 'sunlon', '{:9.5f}'), (7, 'sclk', '{:016.5f}'), (8, 'sclat', '{:9.5f}'), (9, 'sclon', '{:9.5f}'), (10, 'scrad', '{:11.5f}'), (11, 'scalt', '{:11.5f}'), (12, 'el_cmd', '{:7.3f}'), (13, 'az_cmd', '{:7.3f}'), (14, 'af', '{:4d}'), (15, 'orientlat', '{:9.5f}'), (16, 'orientlon', '{:9.5f}'), (17, 'c', '{:1d}'), (18, 'det', '{:3d}'), (19, 'vlookx', '{:9.6f}'), (20, 'vlooky', '{:9.6f}'), (21, 'vlookz', '{:9.6f}'), (22, 'radiance', '{:10.4f}'), (23, 'tb', '{:8.3f}'), (24, 'clat', '{:9.1f}'), (25, 'clon', '{:9.1f}'), (26, 'cemis', '{:9.1f}'), (27, 'csunzen', '{:9.5f}'), (28, 'csunazi', '{:9.5f}'), (29, 'cloctime', '{:8.1f}'), (30, 'cphase', '{:9.5f}'), (31, 'roi', '{:4d}'), (32, 'o', '{:1d}'), (33, 'v', '{:1d}'), (34, 'i', '{:1d}'), (35, 'm', '{:1d}'), (36, 'q', '{:1d}'), (37, 'p', '{:1d}'), (38, 'e', '{:1d}'), (39, 'z', '{:1d}'), (40, 't', '{:1d}'), (41, 'h', '{:1d}'), (42, 'd', '{:1d}'), (43, 'n', '{:1d}'), (44, 's', '{:1d}'), (45, 'a', '{:1d}'), (46, 'b', '{:1d}')]

format_space = [(0, 'date', '"{:>11s}"'), (1, 'utc', '"{:>12s}"'), (2, 'jdate', '{:17.9f}'), (3, 'orbit', '{:5d}'), (4, 'sundist', '{:7.5f}'), (5, 'sunlat', '{:8.5f}'), (6, 'sunlon', '{:9.5f}'), (7, 'sclk', '{:016.5f}'), (8, 'sclat', '{:9.5f}'), (9, 'sclon', '{:9.5f}'), (10, 'scrad', '{:11.5f}'), (11, 'scalt', '{:11.5f}'), (12, 'el_cmd', '{:7.3f}'), (13, 'az_cmd', '{:7.3f}'), (14, 'af', '{:4d}'), (15, 'orientlat', '{:9.5f}'), (16, 'orientlon', '{:9.5f}'), (17, 'c', '{:1d}'), (18, 'det', '{:3d}'), (19, 'vlookx', '{:9.6f}'), (20, 'vlooky', '{:9.6f}'), (21, 'vlookz', '{:9.6f}'), (22, 'radiance', '{:10.4f}'), (23, 'tb', '{:8.3f}'), (24, 'clat', '{:9.1f}'), (25, 'clon', '{:9.1f}'), (26, 'cemis', '{:9.1f}'), (27, 'csunzen', '{:9.1f}'), (28, 'csunazi', '{:9.1f}'), (29, 'cloctime', '{:8.1f}'), (30, 'cphase', '{:9.5f}'), (31, 'roi', '{:4d}'), (32, 'o', '{:1d}'), (33, 'v', '{:1d}'), (34, 'i', '{:1d}'), (35, 'm', '{:1d}'), (36, 'q', '{:1d}'), (37, 'p', '{:1d}'), (38, 'e', '{:1d}'), (39, 'z', '{:1d}'), (40, 't', '{:1d}'), (41, 'h', '{:1d}'), (42, 'd', '{:1d}'), (43, 'n', '{:1d}'), (44, 's', '{:1d}'), (45, 'a', '{:1d}'), (46, 'b', '{:1d}')]

i = 14

nan = ['"{:>11s}"', '"{:>12s}"', '{:17.9f}', '{:5d}', '{:7.5f}', '{:8.5f}', '{:9.5f}', '{:016.5f}', '{:9.5f}', '{:9.5f}', '{:11.5f}', '{:11.5f}', '{:7.3f}', '{:7.3f}', '{:4d}', '{:9.5f}', '{:9.5f}', '{:1d}', '{:3d}', '{:9.6f}', '{:9.6f}', '{:9.6f}', '{:10.1f}', '{:8.1f}', '{:9.1f}', '{:9.1f}', '{:9.1f}', '{:9.1f}', '{:9.1f}', '{:8.1f}', '{:9.5f}', '{:4d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}']

nominal = ['"{:>11s}"', '"{:>12s}"', '{:17.9f}', '{:5d}', '{:7.5f}', '{:8.5f}', '{:9.5f}', '{:016.5f}', '{:9.5f}', '{:9.5f}', '{:11.5f}', '{:11.5f}', '{:7.3f}', '{:7.3f}', '{:4d}', '{:9.5f}', '{:9.5f}', '{:1d}', '{:3d}', '{:9.6f}', '{:9.6f}', '{:9.6f}', '{:10.4f}', '{:8.3f}', '{:9.5f}', '{:9.5f}', '{:9.5f}', '{:9.5f}', '{:9.5f}', '{:8.5f}', '{:9.5f}', '{:4d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}']

solartarget = ['"{:>11s}"', '"{:>12s}"', '{:17.9f}', '{:5d}', '{:7.5f}', '{:8.5f}', '{:9.5f}', '{:016.5f}', '{:9.5f}', '{:9.5f}', '{:11.5f}', '{:11.5f}', '{:7.3f}', '{:7.3f}', '{:4d}', '{:9.5f}', '{:9.5f}', '{:1d}', '{:3d}', '{:9.6f}', '{:9.6f}', '{:9.6f}', '{:10.4f}', '{:8.3f}', '{:9.1f}', '{:9.1f}', '{:9.1f}', '{:9.5f}', '{:9.5f}', '{:8.1f}', '{:9.5f}', '{:4d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}']

space = ['"{:>11s}"', '"{:>12s}"', '{:17.9f}', '{:5d}', '{:7.5f}', '{:8.5f}', '{:9.5f}', '{:016.5f}', '{:9.5f}', '{:9.5f}', '{:11.5f}', '{:11.5f}', '{:7.3f}', '{:7.3f}', '{:4d}', '{:9.5f}', '{:9.5f}', '{:1d}', '{:3d}', '{:9.6f}', '{:9.6f}', '{:9.6f}', '{:10.4f}', '{:8.3f}', '{:9.1f}', '{:9.1f}', '{:9.1f}', '{:9.1f}', '{:9.1f}', '{:8.1f}', '{:9.5f}', '{:4d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}', '{:1d}']

subformat_nan = [(22, 'radiance', '{:10.1f}'), (23, 'tb', '{:8.1f}'), (24, 'clat', '{:9.1f}'), (25, 'clon', '{:9.1f}'), (26, 'cemis', '{:9.1f}'), (27, 'csunzen', '{:9.1f}'), (28, 'csunazi', '{:9.1f}'), (29, 'cloctime', '{:8.1f}')]

subformat_nominal = [(22, 'radiance', '{:10.4f}'), (23, 'tb', '{:8.3f}'), (24, 'clat', '{:9.5f}'), (25, 'clon', '{:9.5f}'), (26, 'cemis', '{:9.5f}'), (27, 'csunzen', '{:9.5f}'), (28, 'csunazi', '{:9.5f}'), (29, 'cloctime', '{:8.5f}')]

subformat_rest = [(30, 'cphase', '{:9.5f}'), (31, 'roi', '{:4d}'), (32, 'o', '{:1d}'), (33, 'v', '{:1d}'), (34, 'i', '{:1d}'), (35, 'm', '{:1d}'), (36, 'q', '{:1d}'), (37, 'p', '{:1d}'), (38, 'e', '{:1d}'), (39, 'z', '{:1d}'), (40, 't', '{:1d}'), (41, 'h', '{:1d}'), (42, 'd', '{:1d}'), (43, 'n', '{:1d}'), (44, 's', '{:1d}'), (45, 'a', '{:1d}'), (46, 'b', '{:1d}')]

subformat_solartarget = [(22, 'radiance', '{:10.4f}'), (23, 'tb', '{:8.3f}'), (24, 'clat', '{:9.1f}'), (25, 'clon', '{:9.1f}'), (26, 'cemis', '{:9.1f}'), (27, 'csunzen', '{:9.5f}'), (28, 'csunazi', '{:9.5f}'), (29, 'cloctime', '{:8.1f}')]

subformat_space = [(22, 'radiance', '{:10.4f}'), (23, 'tb', '{:8.3f}'), (24, 'clat', '{:9.1f}'), (25, 'clon', '{:9.1f}'), (26, 'cemis', '{:9.1f}'), (27, 'csunzen', '{:9.1f}'), (28, 'csunazi', '{:9.1f}'), (29, 'cloctime', '{:8.1f}')]

diviner.formats.create_formatdic_for_dataframe()

Generate format dictionary to be used for pandas.DataFrame.to_string().

Currently not in use, because of DataFrame.to_string bug: https://github.com/pydata/pandas/issues/4158

Saving this part, can be used for when to_string() is being repaired.

diviner.formats.read_csv_format_file()

diviner.ground_calib module

class diviner.ground_calib.DarkScaler(data_in, data_out)

Bases: object

apply_fit(in_)

do_fit()

fractional

residual

scaled

class diviner.ground_calib.PolyScaler(data_in, data_out, rank=2)

Bases: diviner.ground_calib.DarkScaler

Manage polynomial fits. Default rank is 2.

do_fit()

model(x=None, p=None)

perr

poly

rank

diviner.ground_calib.data_prep(data, hk)

diviner.ground_calib.define_sdtype(df)

diviner.ground_calib.get_bb_selector(df)

diviner.ground_calib.get_st_selector(df)

Create dataframe selector for pointing limits of divconstants ‘c’ file

diviner.ground_calib.get_sv_selector(df)

diviner.hot_orbits module

diviner.jdate module

diviner.joblib_test module

diviner.joblib_test.process_channel(tstr, c, month)

diviner.joblib_test.process_month(month)

diviner.joblib_test.process_tstr(tstr, month)

diviner.l1a_header module

diviner.map_plotting module

diviner.metadata module

diviner.metadata_scan module

diviner.metadata_scan_multiprocess module

diviner.mypool module

class diviner.mypool.MyPool(processes=None, initializer=None, initargs=(), maxtasksperchild=None, context=None)

Bases: multiprocessing.pool.Pool

Process

alias of NoDaemonProcess

class diviner.mypool.NoDaemonProcess(group=None, target=None, name=None, args=(), kwargs={}, *, daemon=None)

Bases: multiprocessing.context.Process

daemon

diviner.mypool.sleepawhile(t)

diviner.mypool.test()

diviner.mypool.work(num_procs)

diviner.noise module

diviner.noise.fix_columns(df)

diviner.noise.get_abs_fft(data)

diviner.noise.get_datasetname(fname)

diviner.noise.get_label(dataframe, label, ch, det=11)

diviner.noise.get_new_fname(datasetname, col_str)

diviner.noise.isodd(number)

diviner.noise.plot_all(ax, tbdata, azdata, elevdata, title)

diviner.noise.plot_channel_means(ax, df, col_str, ch_start=1, ch_end=9)

diviner.noise.plot_channel_stds(ax, df, col_str)

diviner.noise.plot_csunzen(ax, df)

diviner.noise.plot_fft(ax, datatuple, title)

diviner.noise.prep_data(fname)

diviner.noise.process_fname(fname_col_str)

diviner.physics module

diviner.physics.exp_term(T, wavelength=None, nu=None)

diviner.physics.planck_frequency(T, nu)

diviner.physics.planck_modis(T, wave)

diviner.physics.planck_wavelength(T, wavelength)

diviner.physics.rayleigh_jeans(T, wavelength)

diviner.physics.wien_approx(T, wavelength)

diviner.plot_utils module

class diviner.plot_utils.ProgressBar(iterations)

Bases: object

animate(iter)

update_iteration(elapsed_iter)

diviner.plot_utils.create_plot_pointings(azim_start=-60, azim_min=-80, azim_max=-40, elev_start=30, elev_min=-10)

diviner.plot_utils.plot3d_animation(df_in)

diviner.plot_utils.plot_all_calib_blocks(df, **kwargs)

Plot all calibration blocks found in the provided dataframe.

df:

pandas Dataframe with block labels defined (went through define_sdtype())

kwargs:

same keyword arguments as plot_calib_block

diviner.plot_utils.plot_all_channels(df_in, det_list, only_thermal=True, **kwargs)

plot the data for each det in det_list for all channels.

Parameters:

• pandas DataFrame (df) –
• list of detector numbers between 1..21 (det_list) –
• keyword arguments for subplots call (**kwargs) –

diviner.plot_utils.plot_calib_block(df, label, id, det='a6_11', limits=None, **kwargs)

Plot one designated calibration block.

df:

pandas Dataframe that has the block labels defined to use as filter.

label:

one of ‘calib’,’bb’,’sv’,’st’

id:

number of block label to be plotted

det:

identifier of the detector, a6_11 is default being used.

diviner.plot_utils.save_to_www(fname, **kwargs)

diviner.processing module

diviner.produce_h5 module

diviner.production module

diviner.rdrpy module

diviner.rdrx module

diviner.reduce_all module

diviner.status_checker module

diviner.test_io_perf module

diviner.test_io_perf.write_numpy_arrays()

diviner.test_production module

diviner.write_month module

diviner.write_rdr20_file module

Module contents