API reference

This section documents the public Python API: the CLI entry point, the main hst123 orchestration class, CLI registration in hst123.utils.options, path helpers, optional HDF5 export for DOLPHOT tables, and shared primitive utilities.

Entry point

hst123._pipeline.main()[source]

Command-line entry point for the full HST123 pipeline.

Reads sys.argv[1:3] as right ascension and declination, then applies all flags from hst123.add_options(). Orchestrates MAST download or raw ingest, builds the observation table, alignment, optional drizzle-all, DOLPHOT, and catalog scraping according to the selected options.

Pipeline class

The class hst123 holds pipeline state (options, coordinates, input lists, reference path) and delegates to primitives (_fits, _phot, _astrom, _dolphot, _scrape_dolphot) for instrument-specific work.

class hst123._pipeline.hst123[source]

End-to-end HST reduction and photometry pipeline.

Construct once, then call handle_args() (or set self.options['args'] programmatically) before invoking download, alignment, drizzle, or DOLPHOT steps.

Attributes:

input_imageslist of str: Paths to science FITS after ingest.
obstableastropy.table.Table or None: Per-exposure metadata table from input_list().
referencestr: Path to the drizzled reference image when set.
optionsdict: global_defaults, detector_defaults, instrument_defaults, acceptable_filters, drizzle_defaults, and args (CLI namespace).
coordastropy.coordinates.SkyCoord or None: Target position from the command line.

CLI options and redo helpers

CLI options: RA/Dec, work dir, download, tweakreg, drizzle, DOLPHOT, scrape-dolphot.

hst123.utils.options.add_options(parser=None, usage=None, version=None)[source]: Register hst123 arguments on an argparse parser. Returns the parser.

hst123.utils.options.want_redo_astrodrizzle(args) → bool[source]

Return True if AstroDrizzle should overwrite existing drizzle outputs.

True when --clobber, --redo, or --redo-astrodrizzle is set.

hst123.utils.options.want_redo_astrometry(args) → bool[source]

Return True if alignment should not be skipped based on prior success headers.

True when --clobber, --redo, or --redo-astrometry is set.

Path helpers

Path helpers for FITS files (stable across os.chdir()).

The pipeline changes the working directory to <work-dir>/workspace for alignment; relative paths must be normalized to absolute before that.

hst123.utils.paths.normalize_fits_path(path: str) → str[source]

Return an absolute, normalized path for FITS references.

Parameters:

pathstr: User-supplied path; empty string is returned unchanged.

Returns:

str: os.path.normpath of the expanded absolute path.

Notes

After alignment, the process cwd is <work-dir>/workspace when --work-dir is set; unresolved relative paths would double-resolve (e.g. test_data/foo.fits under work_dir).

hst123.utils.paths.normalize_work_and_raw_dirs(work_dir, raw_dir)[source]

Resolve CLI --work-dir and --raw-dir to absolute paths.

Default raw directory is <work-dir>/raw. Call once after parsing arguments so paths remain valid after os.chdir(work_dir).

Parameters:

work_dirstr or None: --work-dir value, or None for current directory.
raw_dirstr or None: --raw-dir; if None, ".", or "./", use <work>/raw.

Returns:

tuple of str: (work_abs, raw_abs) absolute paths.

hst123.utils.paths.pipeline_chip_output_dir(work_dir: str | PathLike[str] | None) → str | None[source]

Directory for DOLPHOT *.chipN.fits and *.chipN.sky.fits (base work_dir).

Chip sidecars are written here even when the parent exposure lives under pipeline_workspace_dir().

hst123.utils.paths.pipeline_workspace_dir(work_dir: str | PathLike[str] | None) → str | None[source]

Return <work_dir>/workspace — calibrated inputs, TweakReg scratch, etc.

Drizzled stacks (per-epoch *.ut*.drc.fits and --drizzle-all) are written under <work_dir>/drizzle/, not here. The main reference drizzle (*.ref_*.drc.fits) stays in work_dir root.

Returns:

str or None: Absolute path, or None if work_dir is missing/empty.

Parallel BLAS / OpenMP guard

Context managers to silence stdout/stderr during noisy imports or C I/O.

Use suppress_stdout() for Python-level streams and suppress_stdout_fd() when C libraries write to file descriptor 1.

limit_blas_threads_when_parallel() avoids BLAS/OpenMP oversubscription when DrizzlePac uses multiple workers (num_cores > 1).

hst123.utils.stdio.limit_blas_threads_when_parallel(num_cores: int) → Iterator[None][source]

When num_cores > 1, set common thread env vars to 1 for the duration.

Multi-worker DrizzlePac plus multi-threaded NumPy/BLAS often oversubscribes CPUs; capping worker-local BLAS threads usually improves wall time without changing outputs.

hst123.utils.stdio.suppress_stdout() → Iterator[None][source]

Redirect sys.stdout and sys.stderr to os.devnull until exit.

Yields:

None

Notes

Does not affect the C runtime stdout; see suppress_stdout_fd().

hst123.utils.stdio.suppress_stdout_fd() → Iterator[None][source]

Redirect OS file descriptor 1 (stdout) to os.devnull.

Yields:

None

Notes

For C extensions (e.g. DrizzlePac photeq) that write to the C stdio stdout. Python suppress_stdout() does not catch those writes. Stderr (fd 2) is left unchanged so logging handlers on stderr still work.

Work directory cleanup

After AstroDrizzle or TweakReg, the pipeline can remove scratch FITS and replay external logs into the session log. Use --keep-drizzle-artifacts to retain intermediates.

Relocate or remove drizzlepac / TweakReg scratch files in the work directory.

Default pipeline behavior keeps the work tree tidy: logs go under logs/ and common intermediate FITS/text files from AstroDrizzle are deleted after a successful step. Use --keep-drizzle-artifacts to skip this.

Intermediate names such as single_sci.fits, single_wht.fits, staticMask.fits, and drz_med.fits are produced by DrizzlePac during run_astrodrizzle. WFPC2 pipeline scratch files matching *hst123drz*_c0m.fits / *hst123drz*_c1m.fits come from wfpc2_astrodrizzle_scratch_paths in hst123.utils.astrodrizzle_helpers.

hst123.utils.workdir_cleanup.cleanup_after_astrodrizzle(work_dir: str, *, log: Logger, keep_artifacts: bool = False, base_work_dir: str | PathLike[str] | None = None) → None[source]

After a successful AstroDrizzle run: remove well-known scratch products.

astrodrizzle.log is normally written under .hst123_runfiles/ and ingested into the session log (nothing to archive in the work root).

Matching uses both fixed basenames (legacy drizzle cwd) and globs because DrizzlePac often emits root-prefixed mask names (e.g. *_crmask.fits) rather than bare crmask.fits in work_dir.

When --drizzle-all is used, drizzle products live under <base_work_dir>/drizzle/. work_dir is typically <base>/workspace (the AstroDrizzle scratch/output tree); DrizzlePac may also drop staticMask.fits and similar files in the base work directory, so pass base_work_dir to scrub both trees.

hst123.utils.workdir_cleanup.cleanup_after_tweakreg(work_dir: str, *, log: Logger, keep_artifacts: bool = False) → None[source]: After TweakReg: replay shift / headerlet text into the session log, then remove those files (no separate log copies under logs/ unless keep_artifacts). Also remove shifts WCS helper and extracted headerlet FITS.

hst123.utils.workdir_cleanup.remove_files_matching_globs(work_dir: str | PathLike[str], patterns: Sequence[str], *, log: Logger | None = None, log_each_path: bool = False) → int[source]

Remove regular files under work_dir matching each glob in patterns.

Patterns are joined with work_dir (not the process CWD), so cleanup finds products under --work-dir (e.g. test_data/*.drc.noise.fits) regardless of where the CLI was launched.

The same path is only removed once even if multiple patterns match.

Parameters:

work_dir: Working directory containing pipeline outputs.
patterns: Glob patterns relative to work_dir (e.g. *drc.noise.fits).
log: If set and log_each_path is True, log.info each removed file.
log_each_path: Log every removal (default False for silent bulk deletes).

Returns:

int: Number of files removed.

hst123.utils.workdir_cleanup.remove_superseded_instrument_mask_reference_drizzle(logical_drc_path: str | PathLike[str], *, log: Logger, keep_artifacts: bool = False) → None[source]

Remove the instrument-only reference drizzle produced when n_input < 3.

hst123.hst123.pick_reference() first drizzles to {inst}.ref.drc.fits (e.g. acs_wfc_full.ref.drc.fits) to build static masks, then drizzles again to the filter-named product (e.g. acs.f814w.ref_0001.drc.fits). The first file is interstitial and should be deleted after the final reference succeeds.

DOLPHOT catalog → HDF5

Parse DOLPHOT multi-extension output (catalog, .columns, .param, .data, .info, .warnings) and export a single HDF5 file with a labeled catalog plus metadata.

Designed against DOLPHOT 3.x text products: whitespace-separated numeric dpXXXX catalog, one descriptive line per column in dpXXXX.columns, and sidecar metadata files.

class hst123.utils.dolphot_catalog_hdf5.DolphotColumn(index_1based: int, index_0based: int, description: str, raw_line: str)[source]

One row from a DOLPHOT *.columns file.

description: str

index_0based: int

index_1based: int

raw_line: str

hst123.utils.dolphot_catalog_hdf5.dolphot_columns_to_astropy_table(catalog: ndarray, columns: list[DolphotColumn], names: list[str] | None = None)[source]

Build an astropy.table.Table with descriptive column names.

Parameters:

catalogndarray: Output of load_dolphot_catalog_array().
columnslist of DolphotColumn: From parse_dolphot_columns_file().
nameslist of str, optional: HDF5-safe names; defaults to unique_hdf5_column_names().

hst123.utils.dolphot_catalog_hdf5.find_column_index_0based(columns: list[DolphotColumn], key: str, image: str = '') → int | None[source]

Match DOLPHOT *.columns semantics used by hst123.primitives.scrape_dolphot.ScrapeDolphotPrimitive.get_dolphot_column().

key must appear in the column description. If image is non-empty, it must appear in the description (after stripping a .fits suffix). If image is empty, any column whose description contains key may match—the first such column is returned (same as substring "" in line being true for every line).

hst123.utils.dolphot_catalog_hdf5.group_param_by_image(param: Mapping[str, str]) → tuple[dict[str, str], dict[str, dict[str, str]]][source]: Split flat imgNNNN_* keys into per-image dicts (img0001, …).

hst123.utils.dolphot_catalog_hdf5.load_dolphot_catalog_array(path: str | Path) → ndarray[source]

Load the main DOLPHOT catalog (numeric rows, whitespace-separated).

Parameters:

pathpath-like: Path to dpXXXX (no extension), same as dolphot['base'] in the pipeline.

Returns:

numpy.ndarray: 2-D float array of shape (n_sources, n_columns).

Notes

Uses numpy.loadtxt(). Scraping uses a single load plus vectorized filtering (see hst123.primitives.scrape_dolphot) so the catalog is read once per scrape.

hst123.utils.dolphot_catalog_hdf5.merge_image_metadata(param: Mapping[str, str], info: Mapping[str, Any], data: Mapping[str, Any], warnings: Mapping[str, Any]) → dict[str, Any][source]: Combine parsed sidecars into one JSON-serializable tree keyed by img0000 .. imgNNNN.

hst123.utils.dolphot_catalog_hdf5.parse_column_index_and_description(line: str) → tuple[int, str] | None[source]

Parse a single *.columns line of the form N. description.

Returns:

(1-based index, description) or None if the line does not match.

hst123.utils.dolphot_catalog_hdf5.parse_dolphot_columns_file(path: str | Path) → list[DolphotColumn][source]

Read a DOLPHOT *.columns file into structured column definitions.

Parameters:

pathpath-like: Path to dpXXXX.columns.

Returns:

list of DolphotColumn: One entry per line, in file order (column 1 .. N).

hst123.utils.dolphot_catalog_hdf5.parse_dolphot_data_file(path: str | Path) → dict[str, Any][source]: Parse dpXXXX.data: WCS, alignment iteration counts, PSF quality, aperture-correction stats.

hst123.utils.dolphot_catalog_hdf5.parse_dolphot_info_file(path: str | Path) → dict[str, Any][source]

Parse dpXXXX.info: MJD per input image, limits, filter/exptime lines, alignment, apcor.

Structure varies slightly by instrument; unknown lines are kept in extra_lines.

hst123.utils.dolphot_catalog_hdf5.parse_dolphot_param_file(path: str | Path) → dict[str, str][source]: Parse dpXXXX.param (key = value lines, DOLPHOT style).

hst123.utils.dolphot_catalog_hdf5.parse_dolphot_warnings_file(path: str | Path) → dict[str, Any][source]: Parse dpXXXX.warnings into all lines and optional buckets by image path.

hst123.utils.dolphot_catalog_hdf5.read_dolphot_catalog_hdf5(path: str | Path, photometry_path: str = 'photometry')[source]

Read back a table written by write_dolphot_catalog_hdf5().

Returns:

astropy.table.Table

hst123.utils.dolphot_catalog_hdf5.unique_hdf5_column_names(columns: list[DolphotColumn]) → list[str][source]: Return safe column names, de-duplicated with numeric suffixes if needed.

hst123.utils.dolphot_catalog_hdf5.write_dolphot_catalog_hdf5(out_path: str | Path, base: str | Path, *, photometry_path: str = 'photometry', include_raw_sidecars: bool = True, compression: bool = True, dolphot_dir: str | Path | None = None, embed_dolphot_directory_text: bool = False) → Path[source]

Write one HDF5 file with the full DOLPHOT catalog and metadata.

Parameters:

out_pathpath-like: Output .h5 file.
basepath-like: DOLPHOT output base path without extension (same as pipeline dolphot['base']).
photometry_pathstr: HDF5 path for the Table (Astropy default metadata).
include_raw_sidecarsbool: If True, store UTF-8 text of .param, .info, .data, .warnings, and .columns as datasets under metadata/raw/.
compressionbool: If True, use gzip compression on the table (requires Astropy + h5py).
dolphot_dirpath-like, optional: Directory scanned for a full file manifest (default: base.parent, i.e. the DOLPHOT output folder such as <work>/dolphot/). Every file path, size, and mtime are stored. Optional text embedding under metadata/dolphot_directory/text_embed/ when embed_dolphot_directory_text is True.
embed_dolphot_directory_textbool, optional: If True, embed small UTF-8 text files from dolphot_dir (can be slow for large trees). Default False (manifest only).

Returns:

pathlib.Path: Path to the written file.

Raises:

ImportError: If h5py is not installed.

Data model

Lightweight HST file handle with path properties and metadata (dict-based).

class hst123.datamodels.hst_model.HSTModel(filepath, meta: dict | None = None)[source]

Base class for associating a FITS path with metadata.

Subclasses may extend on_init() for additional setup.

Parameters:

filepathstr or None: Path to an HST data file, or None for in-memory-only use.
metadict, optional: Initial metadata merged into meta.

Attributes:

filepathstr or None: Path passed at construction.
metadict: Metadata dictionary (e.g. model_type, filename).

property basename: str | None

Base filename of filepath, or None if no path is set.

Returns:

str or None

property dirname: str | None

Directory containing filepath, or None if no path is set.

Returns:

str or None

init_meta(meta: dict | None = None) → None[source]

Reset metadata and merge optional user values.

Parameters:

metadict, optional: Keys merged into meta.

on_init() → None[source]

Hook invoked after init_meta(); default sets model type and filename.

Override in subclasses for custom initialization.

Primitives base class

class hst123.primitives.base.BasePrimitive(pipeline)[source]

Bases: object

Attach to the pipeline and share options/state via self._p.

Parameters:

pipeline: The running hst123 instance (required).

Attributes:

_p: Same object as pipeline.
_log: Logger named after the primitive module (hst123.primitives…).

Raises:

TypeError: If pipeline is None.

property pipeline

Pipeline instance (alias of self._p).

Returns:

object