template_project API

Top-level verbs

Template project for oceanographic Python packages.

Public API

Verb-style convenience functions delegate to the subpackages:

import template_project as tp datasets = tp.read(β€œrapid”) # -> readers.load_dataset; returns a list of Datasets ds = datasets[0] tp.write(ds, β€œout.nc”) # -> writers.save_dataset tp.plot(ds) # -> plotters.plot_monthly_transport tp.process(ds) # -> processors.process

tp.read returns a list of Datasets (one per file). The subpackages remain importable directly, e.g. from template_project.readers import load_dataset.

template_project.plot(ds: Dataset, **kwargs: Any) → tuple[Any, Any][source]

Plot monthly transport (delegates to plotters.plot_monthly_transport).

template_project.process(ds: Dataset, **kwargs: Any) → Dataset[source]

Process a Dataset (delegates to processors.process).

template_project.read(array_name: str = 'rapid', **kwargs: Any) → list[Dataset][source]

Load dataset(s) for an observing array (delegates to readers.load_dataset).

template_project.write(ds: Dataset, output_file: str | Path | None = None, **kwargs: Any) → bool[source]

Save a Dataset to NetCDF (delegates to writers.save_dataset).

output_file defaults to <cwd>/data/test.nc when omitted.

Subpackages

Data loading functions for oceanographic datasets.

The load_dataset() dispatcher maps an array_name to a reader implementation. Add a new array by writing a reader submodule (see rapid) and registering it in _get_reader().

template_project.readers.load_dataset(array_name: str, source: str | None = None, file_list: str | list[str] | None = None, transport_only: bool = True, data_dir: str | Path | None = None, redownload: bool = False) → list[Dataset][source]

Load raw datasets from a selected AMOC observing array.

Parameters:

  • array_name (str) – The name of the observing array to load. Options are: - β€˜rapid’ : RAPID 26N array

  • source (str, optional) – URL or local path to the data source. If None, the reader-specific default source will be used.

  • file_list (str or list of str, optional) – Filename or list of filenames to process. If None, the reader-specific default files will be used.

  • transport_only (bool, optional) – If True, restrict to transport files only.

  • data_dir (str, optional) – Local directory for downloaded files.

  • redownload (bool, optional) – If True, force redownload of the data.

Returns:

List of datasets loaded from the specified array.

Return type:

list of xarray.Dataset

Raises:

ValueError – If an unknown array name is provided.

template_project.readers.load_sample_dataset(array_name: str = 'rapid') → Dataset[source]

Load a sample dataset for quick testing.

Currently supports: - β€˜rapid’ : loads the β€˜RAPID_26N_TRANSPORT.nc’ file

Parameters:

array_name (str, optional) – The name of the observing array to load. Default is β€˜rapid’.

Returns:

A single xarray Dataset from the sample file.

Return type:

xr.Dataset

Raises:

ValueError – If the array_name is not recognised.

template_project.readers.read_rapid(source: str | Path | None, file_list: str | list[str], transport_only: bool = True, data_dir: str | Path | None = None, redownload: bool = False) → list[Dataset][source]

Load the RAPID transport dataset from a URL or local file path into an xarray.Dataset.

Parameters:

  • source (str, optional) – URL or local path to the NetCDF file(s). Defaults to the RAPID data repository URL.

  • file_list (str or list of str, optional) – Filename or list of filenames to process. If None, will attempt to list files in the source directory.

  • transport_only (bool, optional) – If True, restrict to transport files only.

  • data_dir (str, Path or None, optional) – Optional local data directory.

  • redownload (bool, optional) – If True, force redownload of the data.

Returns:

The loaded xarray dataset with basic inline metadata.

Return type:

xr.Dataset

Raises:

  • ValueError – If the source is neither a valid URL nor a directory path.

  • FileNotFoundError – If no valid NetCDF files are found in the provided file list.

Data writing functionality for oceanographic datasets.

template_project.writers.save_dataset(ds: Dataset, output_file: str | Path | None = None, *, compress: bool = True, complevel: int = 4, optimise_dtype: bool = True, keep_dtype: list[str] | None = None, delete_existing: bool = False, prompt_user: bool = False) → bool[source]

Save a Dataset to NetCDF with optional compression and dtype optimisation.

Dask-backed datasets stream to disk chunk-by-chunk (the data is never loaded into memory here), so this works for datasets larger than RAM.

Parameters:

  • ds (xarray.Dataset) – The dataset to be saved.

  • output_file (str or Path, optional) – The path to the output NetCDF file. Defaults to <cwd>/data/test.nc (via get_default_data_dir()).

  • compress (bool) – Apply lossless zlib compression to every data variable (writes NETCDF4). Defaults to True. When False, writes uncompressed NETCDF4_CLASSIC.

  • complevel (int) – zlib compression level 1-9 (higher = smaller/slower). Defaults to 4.

  • optimise_dtype (bool) – Downcast data variables to a smaller storage dtype via cast_output_dtypes() (e.g. float64->float32) before writing. Defaults to True. Coordinates and datetime/*time* variables are always preserved; float32 is lossy (~7 significant digits), so set this False β€” or list precision-critical variables in keep_dtype β€” when full precision matters.

  • keep_dtype (list of str, optional) – Data-variable names to keep at full precision when optimise_dtype is True.

  • delete_existing (bool) – Whether to delete the file if it already exists. Defaults to False.

  • prompt_user (bool) – Whether to prompt interactively before deleting an existing file. Defaults to False (safe for notebooks, scripts, and CI); set True for interactive use.

Returns:

  • bool – True if the dataset was saved successfully, False otherwise.

  • Based on (https://github.com/pydata/xarray/issues/3743)

Visualization utilities for oceanographic data.

template_project.plotters.plot_monthly_transport(ds: Dataset, var: str = 'moc_mar_hc10') → tuple[Any, Any][source]

Plot original and monthly averaged transport time series.

Parameters:

  • ds (xr.Dataset) – Dataset with a time dimension and a transport variable.

  • var (str, optional) – Name of the variable to plot. Default is β€œmoc_mar_hc10”.

template_project.plotters.show_attributes(data: str | Dataset) → DataFrame[source]

Extract attribute information from a Dataset or netCDF file as a DataFrame.

Parameters:

xr.Dataset) (data (str or)

Returns:

pandas.DataFrame –

  • Attribute: The name of the attribute.

  • Value: The value of the attribute.

Return type:

A DataFrame containing the following columns:

template_project.plotters.show_variables(data: str | Dataset) → Styler[source]

Extract variable information from a Dataset or netCDF file as a styled DataFrame.

Parameters:

xr.Dataset) (data (str or)

Returns:

pandas.io.formats.style.Styler –

  • dims: The dimension of the variable (or β€œstring” if it is a string type).

  • name: The name of the variable.

  • units: The units of the variable (if available).

  • comment: Any additional comments about the variable (if available).

Return type:

A styled DataFrame containing the following columns:

Data processing and unit-conversion utilities.

Public API

  • reformat_units_var(), convert_units_var() β€” unit helpers (in units).

  • process() β€” the package-level process verb; a small example that normalises unit strings across a Dataset. This is the extension point for a real processing pipeline (compare oceanarray.processors).

template_project.processors.convert_units_var(var_values: Any, current_unit: str, new_unit: str, unit_conversion: dict[str, dict[str, str | float]] = {'Celsius': {'factor': 1, 'units_name': 'degrees_Celsius'}, 'Pa': {'factor': 0.0001, 'units_name': 'dbar'}, 'S m-1': {'factor': 0.1, 'units_name': 'mS cm-1'}, 'S/m': {'factor': 0.1, 'units_name': 'mS/cm'}, 'cm': {'factor': 0.01, 'units_name': 'm'}, 'cm s-1': {'factor': 0.01, 'units_name': 'm s-1'}, 'cm/s': {'factor': 0.01, 'units_name': 'm/s'}, 'dbar': {'factor': 10000, 'units_name': 'Pa'}, 'degrees_Celsius': {'factor': 1, 'units_name': 'Celsius'}, 'g m-3': {'factor': 0.001, 'units_name': 'kg m-3'}, 'kg m-3': {'factor': 1000, 'units_name': 'g m-3'}, 'km': {'factor': 1000, 'units_name': 'm'}, 'm': {'factor': 100, 'units_name': 'cm'}, 'm s-1': {'factor': 100, 'units_name': 'cm s-1'}, 'm/s': {'factor': 100, 'units_name': 'cm/s'}, 'mS cm-1': {'factor': 10, 'units_name': 'S m-1'}, 'mS/cm': {'factor': 10, 'units_name': 'S/m'}}) → Any[source]

Convert the units of variables in an xarray Dataset to preferred units. This is useful, for instance, to convert cm/s to m/s.

Parameters:

  • (xarray.Dataset) (ds)

  • (list) (preferred_units)

  • (dict) (unit_conversion)

  • string (Each key is a unit) –

    • β€˜factor’: The factor to multiply the variable by to convert it.

    • ’units_name’: The new unit name after conversion.

  • with (and each value is a dictionary) –

    • β€˜factor’: The factor to multiply the variable by to convert it.

    • ’units_name’: The new unit name after conversion.

Returns:

xarray.Dataset

Return type:

The dataset with converted units.

template_project.processors.process(ds: Dataset) → Dataset[source]

Normalise unit strings on every variable in a Dataset (example processor).

For each variable carrying a units attribute, rewrite it to the preferred string form via reformat_units_var(). Returns a copy; the input is unchanged.

Parameters:

ds (xr.Dataset) – Dataset whose variable units attributes should be normalised.

Returns:

A copy of ds with normalised unit strings.

Return type:

xr.Dataset

template_project.processors.reformat_units_var(ds: Dataset, var_name: str, unit_format: dict[str, str] = {'S/m': 'S m-1', 'cm/s': 'cm s-1', 'degrees_Celsius': 'Celsius', 'g/m^3': 'g m-3', 'm/s': 'm s-1', 'm^3/s': 'Sv', 'meters': 'm'}) → str[source]

Renames units in the dataset based on the provided dictionary for OG1.

Parameters:

  • (xarray.Dataset) (ds)

  • (dict) (unit_format)

Returns:

xarray.Dataset

Return type:

The dataset with renamed units.

General-purpose utilities for data handling and downloading.

template_project.utilities.apply_defaults(default_source: str, default_files: list[str]) → Callable[source]

Decorator to apply default values for β€˜source’ and β€˜file_list’ parameters if they are None.

Parameters:

  • default_source (str) – Default source URL or path.

  • default_files (list of str) – Default list of filenames.

Returns:

A wrapped function with defaults applied.

Return type:

Callable

template_project.utilities.cast_output_dtypes(ds: Dataset, keep_dtype: list[str] | None = None) → Dataset[source]

Cast each data variable to its optimal storage dtype for NetCDF output.

Calls find_best_dtype() per data variable and rebuilds only those whose dtype changes; attributes are preserved and the input dataset is not modified. Coordinates are never touched (so a TIME coordinate keeps full precision), and find_best_dtype already preserves datetime and *time*-named variables.

float64 -> float32 is lossy (~7 significant digits). That is fine for most geophysical measurements but wrong for high-dynamic-range quantities where error accumulates (e.g. a float time axis such as β€œseconds since 1970”). Pass such variable names in keep_dtype to preserve their dtype.

Parameters:

  • ds (xr.Dataset) – Dataset to cast.

  • keep_dtype (list of str, optional) – Data-variable names to leave at their original dtype.

Returns:

New dataset with optimised dtypes (or the same object if nothing changed).

Return type:

xr.Dataset

template_project.utilities.download_file(url: str, dest_folder: str, redownload: bool = False) → str[source]

Download a file from HTTP(S) or FTP to the specified destination folder.

Parameters:

  • url (str) – The URL of the file to download.

  • dest_folder (str) – Local folder to save the downloaded file.

  • redownload (bool, optional) – If True, force re-download of the file even if it exists.

Returns:

The full path to the downloaded file.

Return type:

str

Raises:

ValueError – If the URL scheme is unsupported.

template_project.utilities.find_best_dtype(var_name: str, da: DataArray) → type[source]

Determine the optimal storage dtype for a variable.

Parameters:

  • var_name (str) – Variable name.

  • da (xr.DataArray) – Data array to inspect.

Returns:

Recommended numpy dtype.

Return type:

type

Notes

Rules applied in order:

  • String / datetime / object variables: unchanged.

  • time in name: unchanged (preserve datetime64 / float encoding).

  • *_qc suffix or flag in name: int8 (name match is case-insensitive).

  • serial_number or serial: int32.

  • latitude / longitude in name: float64.

  • Signed 64-bit integer input: downsize to int32; unsigned integers are left unchanged (uint64 values can exceed the int32 range).

  • float64 input: float32.

  • Anything else: unchanged.

template_project.utilities.get_default_data_dir() → Path[source]

Return the default data directory (./data under the current working directory).

Resolved relative to the working directory rather than the installed package location, so downloads never land inside site-packages and the result is independent of the source layout (flat vs src/).

template_project.utilities.is_valid_url(url: str) → bool[source]

Validate if a given string is a valid URL with supported schemes.

Parameters:

url (str) – The URL string to validate.

Returns:

True if the URL is valid and uses a supported scheme (β€˜http’, β€˜https’, β€˜ftp’), otherwise False.

Return type:

bool

template_project.utilities.resolve_file_path(file_name: str, source: str | Path | None, download_url: str | None, local_data_dir: Path, redownload: bool = False) → Path[source]

Resolve the path to a data file, using local source, cache, or downloading if necessary.

Parameters:

  • file_name (str) – The name of the file to resolve.

  • source (str or Path or None) – Optional local source directory.

  • download_url (str or None) – URL to download the file if needed.

  • local_data_dir (Path) – Directory where downloaded files are stored.

  • redownload (bool, optional) – If True, force redownload even if cached file exists.

Returns:

Path to the resolved file.

Return type:

Path

template_project.utilities.safe_update_attrs(ds: Dataset, new_attrs: dict[str, str], overwrite: bool = False, verbose: bool = True) → Dataset[source]

Safely update Dataset attributes without overwriting existing keys.

Parameters:

  • ds (xr.Dataset) – The xarray Dataset whose attributes will be updated.

  • new_attrs (dict of str) – Dictionary of new attributes to add.

  • overwrite (bool, optional) – If True, allow overwriting existing attributes. Defaults to False.

  • verbose (bool, optional) – If True, emit a warning when skipping existing attributes. Defaults to True.

Returns:

The dataset with updated attributes.

Return type:

xr.Dataset