Report generation

Report generation#

class echopop.reports.reporter.Reporter(save_directory: str | Path, verbose: bool = True)#

Reporter - utility class for writing a suite of Excel reports used in the FEAT pipeline.

This class centralizes common report generation routines (age-length tables, kriging inputs, haul counts, transect reports, etc.) and provides consistent validation and file handling.

Attributes:

save_directorypathlib.Path: Directory where generated reports will be saved. Created if it does not exist.
verbosebool: If True, print brief status messages when files are written.

Methods

`aged_length_haul_counts_report`(filename, ...)	Create and write an aged-length haul counts Excel report.
`kriged_aged_biomass_mesh_report`(filename, ...)	Produce an Excel workbook containing kriged age-specific biomass sheets.
`kriged_length_age_abundance_report`(filename, ...)	Create kriged age-length abundance reports and write a 3-sheet workbook.
`kriged_length_age_biomass_report`(filename, ...)	Create kriged age-length biomass reports (values converted to metric megatonnes).
`kriged_mesh_results_report`(filename, ...)	Write a single-sheet kriged mesh results report with uncertainty metrics.
`kriging_input_report`(filename, sheetname, ...)	Save key kriging input variables to an Excel sheet.
`total_length_haul_counts_report`(filename, ...)	Create an Excel report of combined specimen and length haul totals.
`transect_aged_biomass_report`(filename, ...)	Produce a transect-level aged biomass workbook (three sex-specific sheets).
`transect_length_age_abundance_report`(...)	Write an un-kriged transect length-age abundance workbook (3 sheets).
`transect_length_age_biomass_report`(filename, ...)	Write an un-kriged transect age-length biomass workbook.
`transect_population_results_report`(filename, ...)	Write transect-level population results with FEAT-friendly column names.

Notes

Methods that accept datatables expect a dict-like container with at minimum an ‘aged’ DataFrame.
The helper function build_age_length_full_tables is used internally to consolidate logic used across multiple report methods (unifying indexing, stacking/unstacking and scaling).

Examples

Basic usage >>> from pathlib import Path >>> reports = Reporter(Path(“reports_out”), verbose=True) >>> # Write a kriging input sheet >>> reports.kriging_input_report(transect_df, “krig_input.xlsx”, “KrigeInput”)

Creating an aged-length haul counts report >>> # bio_data must include [‘sex’,’length’,’length_bin’,’haul_num’] >>> sheetnames = {“male”: “Male”, “female”: “Female”, “all”: “All”} >>> reports.aged_length_haul_counts_report(bio_data, “haul_counts.xlsx”, sheetnames)

Creating kriged age-length abundance/biomass reports >>> # datatables expected to contain keys ‘aged’ and ‘unaged’ (pandas DataFrames) >>> exclude_filter = {“stratum”: [“exclude_this_stratum”]} >>> reports.kriged_length_age_abundance_report(datatables, exclude_filter, “krig_abund.xlsx”, sheetnames) >>> reports.kriged_length_age_biomass_report(datatables, exclude_filter, “krig_biomass.xlsx”, sheetnames)

Preparing and writing transect reports >>> reports.transect_length_age_abundance_report(datatables, “transect_abund.xlsx”, sheetnames) >>> reports.transect_length_age_biomass_report(datatable, “transect_biomass.xlsx”, sheetnames)

aged_length_haul_counts_report(filename: str, sheetnames: dict[str, str], bio_data: DataFrame) → None#

Create and write an aged-length haul counts Excel report.

The method builds pivot tables of haul-level length counts stratified by sex and writes three sheets (male, female, all) to an Excel workbook in self.save_directory.

Parameters:

filenamestr: Name of the Excel file to create (relative to self.save_directory).
sheetnamesdict: Mapping with keys ‘male’,’female’,’all’ and values for the worksheet names to use.
bio_datapandas.DataFrame: Raw biological specimen data. Required columns: - ‘sex’ (values expected: ‘male’, ‘female’, potentially ‘unsexed’) - ‘length’ (numeric) - ‘length_bin’ (interval-like objects with .mid) - ‘haul_num’ (identifier for haul)

Returns:

None

Raises:

TypeError: If bio_data is not a pandas DataFrame or filename is not a str or sheetnames not a dict.
KeyError: If bio_data is missing any of the required columns or sheetnames lacks required keys.
ValueError: If bio_data has no rows after filtering to male/female (no content to pivot).

Examples

>>> from pathlib import Path
>>> reports = Reporter(Path("out"))
>>> sheetnames = {"male":"Male", "female":"Female", "all":"All"}
>>> reports.aged_length_haul_counts_report(bio_df, "haul_counts.xlsx", sheetnames)

kriged_aged_biomass_mesh_report(filename: str, sheetnames: dict[str, str], kriged_data: DataFrame, weight_data: DataArray, kriged_stratum_link: dict[str, str], exclude_filter: dict[str, Any] = None) → None#

Produce an Excel workbook containing kriged age-specific biomass sheets.

High-level steps: - Normalize the weight/age distribution (age-weight proportions) per stratum - Rename kriged_data stratum columns using kriged_stratum_link - Multiply kriged biomass by age/sex proportions to produce sex-specific aged biomass - Format/rename columns and write three sheets (all/female/male)

Parameters:

filenamestr: Output filename within self.save_directory.
sheetnamesdict: Mapping sex -> worksheet name with keys ‘all’, ‘female’, ‘male’.
kriged_datapandas.DataFrame: Kriged mesh with one row per mesh cell and at least the stratum identifier and biomass-related columns. The stratum must be mapped to weight_data using kriged_stratum_link.
weight_datapandas.DataFrame: Multi-indexed DataFrame representing age-weight distributions. Expected to have a MultiIndex for columns that includes ‘sex’ and a single stratum level (e.g. ‘stratum’).
kriged_stratum_linkdict: Mapping from column names in kriged_data to the stratum names used by weight_data.
exclude_filterdict, optional: Optional filters passed to utils.apply_filters to zero-out or exclude strata before computing age proportions.

Returns:

None

Raises:

TypeError: If kriged_data or weight_data are not pandas DataFrames or other args have wrong types.
ValueError: If weight_data does not contain exactly one stratum-level name in its column MultiIndex.
KeyError: If expected strata (after renaming) are missing.

Examples

>>> reports = Reporter("out")
>>> sheetnames = {"all":"All", "female":"Fem", "male":"Male"}
>>> reports.kriged_aged_biomass_mesh_report("krig_biomass.xlsx", kriged_df, weight_df,
{'old':'stratum'}, sheetnames)

kriged_length_age_abundance_report(filename: str, sheetnames: dict[str, str], datatables: dict[str, DataArray], exclude_filter: dict[str, Any] = None) → None#

Create kriged age-length abundance reports and write a 3-sheet workbook.

This method:

Builds aged/unaged full tables (male/female/all), optionally redistributing using the provided exclude_filter via apportion.redistribute_population_table
Converts interval indices to numeric midpoints for table rows/columns
Writes per-sex pivot tables using internal write helpers

Parameters:

filenamestr

Output workbook filename.

sheetnamesdict

Mapping sex -> sheet name (keys: ‘male’,’female’,’all’).

datatablesdict

Dictionary with keys:

‘aged’ : pandas.DataFrame
‘unaged’ : pandas.DataFrame

exclude_filterdict

Filter dictionary passed to apportion.reallocate_population_table() to zero/exclude strata.

Returns:

None

Raises:

TypeError: If datatables is not a dict or other args are wrong types.
KeyError: If required keys are missing from datatables (must include ‘aged’).
ValueError: If there’s a structural mismatch during table building.

Examples

>>> reports = Reporter("out")
>>> sheetnames = {"male":"Male","female":"Female","all":"All"}
>>> reports.kriged_length_age_abundance_report({'aged':aged_df, 'unaged':unaged_df},
{'stratum':['S1']}, "krig_abund.xlsx", sheetnames)

kriged_length_age_biomass_report(filename: str, sheetnames: dict[str, str], datatable: DataArray, exclude_filter: dict[str, Any] = None) → None#

Create kriged age-length biomass reports (values converted to metric megatonnes).

This is similar to kriged_length_age_abundance_report but scales the tables by 1e-9 (to convert from grams/tonnes as required) before writing.

Parameters:

filenamestr: Output filename.
sheetnamesdict: Mapping sex -> worksheet name.
datatablexarray.DataArray: DataArray of consolidated aged and unaged biomass estimates.
exclude_filterdict: Filter dict forwarded to apportion.reallocate_population_table().

Returns:

None

Raises:

TypeError: If inputs have incorrect types.
KeyError: If required keys are missing from datatables.

Examples

>>> reports = Reporter("out")
>>> reports.kriged_length_age_biomass_report(da_full, {},
"krig_biomass.xlsx", sheetnames)

kriged_mesh_results_report(filename: str, sheetname: str, kriged_data: DataFrame, kriged_stratum: str, kriged_variable: str, sigma_bs_data: DataFrame, sigma_bs_stratum: str) → None#

Write a single-sheet kriged mesh results report with uncertainty metrics.

Parameters:

filenamestr: File name to save (in self.save_directory).
sheetnamestr: Excel worksheet name to use.
kriged_datapandas.DataFrame: Kriged mesh containing at minimum columns for latitude, longitude, kriged_variable, and cell_cv. The stratum column is indicated by kriged_stratum.
kriged_stratumstr: Column name or index name in kriged_data identifying stratum; used to align sigma_bs_data.
kriged_variablestr: Name of the kriged estimate column to include (e.g., ‘biomass’).
sigma_bs_datapandas.DataFrame: DataFrame containing sigma_bs by stratum. Must include sigma_bs column and a column named by sigma_bs_stratum (or index).
sigma_bs_stratumstr: Column or index name in sigma_bs_data that identifies strata.

Returns:

None

Raises:

TypeError: If inputs are not pandas DataFrames or strings.
KeyError: If required columns are not found in kriged_data or sigma_bs_data.
ValueError: If shapes or indexes cannot be aligned.

Examples

>>> reports = Reporter("out")
>>> reports.kriged_mesh_results_report("krig_mesh.xlsx", "Mesh", krig_df, "stratum",
"biomass", sigma_df, "stratum")

kriging_input_report(filename: str, sheetname: str, transect_data: DataFrame) → None#

Save key kriging input variables to an Excel sheet.

Parameters:

filenamestr: Output file name saved under self.save_directory.
sheetnamestr: Worksheet name to use inside the workbook.
transect_datapandas.DataFrame: Transect-level DataFrame that must contain the following columns: [‘latitude’,’longitude’,’biomass_density’,’nasc’,’number_density’].

Returns:

None

Raises:

TypeError: If transect_data is not a pandas DataFrame or filename/sheetname not strings.
KeyError: If transect_data does not contain the required columns.

Examples

>>> reports = Reporter("out")
>>> reports.kriging_input_report(transect_df, "krig_input.xlsx", "Input")

total_length_haul_counts_report(filename: str, sheetnames: dict[str, str], bio_data: dict[str, DataFrame]) → None#

Create an Excel report of combined specimen and length haul totals.

The method expects bio_data to contain DataFrames for ‘specimen’ and ‘length’ and will build pivot tables, combine sex-specific and ‘all’ columns and write three sheets.

Parameters:

filenamestr: File name to create in self.save_directory.
sheetnamesDict[str, str]: Mapping sex -> sheet name.
bio_datadict: Dictionary with keys: - ‘specimen’ : pandas.DataFrame containing specimen records (must include ‘sex’, ‘length_bin’, ‘haul_num’) - ‘length’ : pandas.DataFrame containing length counts (must include ‘sex’, ‘length_bin’, ‘haul_num’, ‘length_count’)

Returns:

None

Raises:

TypeError: If arguments have incorrect types.
KeyError: If required keys/columns are missing.

Examples

>>> reports = Reporter("out")
>>> sheetnames = {"male":"Male","female":"Female","all":"All"}
>>> reports.total_length_haul_counts_report({'specimen':spec_df, 'length':len_df},
"total_counts.xlsx", sheetnames)

transect_aged_biomass_report(filename: str, sheetnames: dict[str, str], transect_data: DataFrame, weight_data: DataArray, exclude_filter: dict[str, Any] = None) → None#

Produce a transect-level aged biomass workbook (three sex-specific sheets).

Steps:

Create age-weight proportions per stratum from weight_data
Apply optional exclude_filter to zero/out strata
Multiply transect biomass by age/sex proportions and write per-sex sheets

Parameters:

filenamestr: Output filename.
sheetnamesdict: Mapping sex -> worksheet name.
transect_datapandas.DataFrame: Transect-level summary data with a stratum identifier consistent with weight_data.
weight_datapandas.DataFrame: Multi-indexed age-weight table whose column names include ‘sex’ and exactly one stratum level.
exclude_filterdict, optional: Passed to utils.apply_filters to zero excluded strata (default: {}).

Returns:

None

Raises:

TypeError: If inputs have wrong types.
ValueError: If weight_data does not contain exactly one stratum-level in columns.

Examples

>>> reports = Reporter("out")
>>> reports.transect_aged_biomass_report(transect_df, weight_df, "transect_biomass.xlsx",
sheetnames)

transect_length_age_abundance_report(filename: str, sheetnames: dict[str, str], datatables: dict[str, DataArray]) → None#

Write an un-kriged transect length-age abundance workbook (3 sheets).

Parameters:

filenamestr: Output filename saved into self.save_directory.
sheetnamesdict: Mapping sex -> worksheet name.
datatablesdict: Dictionary with keys ‘aged’ (required) and ‘unaged’ (optional). Expected structure matches the other age-length methods (interval indices, age_bin and sex columns).

Returns:

None

Raises:

TypeError: If inputs are of incorrect types.
KeyError: If required keys are missing.

Examples

>>> reports = Reporter("out")
>>> reports.transect_length_age_abundance_report({'aged':aged_df,'unaged':unaged_df},
"transect_abund.xlsx", sheetnames)

transect_length_age_biomass_report(filename: str, sheetnames: dict[str, str], datatable: DataArray) → None#

Write an un-kriged transect age-length biomass workbook.

This function expects datatable to be a DataFrame with interval-like index for length bins and column levels for age_bin and sex. It converts interval midpoints, composes male/female/all tables and writes an Excel workbook with three sheets (male/female/all).

Parameters:

filenamestr: Output workbook filename (relative to save_directory).
sheetnamesdict: Mapping sex -> worksheet name (keys: ‘male’,’female’,’all’).
datatablepandas.DataFrame: A DataFrame representing aged counts/weights indexed by intervals (length_bin) with column levels [‘age_bin’,’sex’] (or compatible). Values are expected to be numeric.

Returns:

None

Raises:

TypeError: If datatable is not a pandas DataFrame or filename/sheetnames have wrong types.
ValueError: If datatable lacks expected structure (index/columns).

Examples

>>> reports = Reporter("out")
>>> reports.transect_length_age_biomass_report(datatable, "transect_biomass.xlsx",
sheetnames)

transect_population_results_report(filename: str, sheetname: str, transect_data: DataFrame, weight_strata_data: DataArray, sigma_bs_stratum: DataFrame, stratum_name: str) → None#

Write transect-level population results with FEAT-friendly column names.

The method renames, aligns and exports the transect-level results along with computed sig_b and wgt_per_fish derived from sigma_bs_stratum and weight_strata_data respectively.

Parameters:

filenamestr: Output workbook filename.
sheetnamestr: Worksheet name to use.
transect_datapandas.DataFrame: Transect-level DataFrame. Must contain a stratum identifier that matches stratum_name.
weight_strata_datapandas.DataFrame: DataFrame (possibly multi-indexed) providing weight-per-fish or related metrics.
sigma_bs_stratumpandas.DataFrame: DataFrame indexed or columned by stratum that includes a numeric sigma_bs column.
stratum_namestr: Column or index name to use as the stratum key to align data between inputs.

Returns:

None

Raises:

TypeError: If arguments are not the expected types.
KeyError: If sigma_bs_stratum lacks sigma_bs or stratum_name cannot be found.

Examples

>>> reports = Reporter("out")
>>> reports.transect_population_results_report(transect_df, weight_df, sigma_df, "stratum",
"transect_pop.xlsx", "PopResults")

Report comparisons#

echopop.reports.compare.compute_dataset_differences(echopro_datasets: dict[str, dict[int, GeoDataFrame]], echopop_datasets: dict[str, dict[int, GeoDataFrame]], columns: list = None) → tuple[DataFrame, DataFrame]#

Compute magnitude and percent differences between EchoPro and Echopop reports.

Compute magnitude and percent differences between EchoPro and Echopop report outputs across survey years and report types (transect, kriging).

Parameters:

echopro_datasetsDict[str, Dict[int, geopandas.GeoDataFrame]]: Nested dictionary of EchoPro data keyed by report type ("transect", "kriging") and year.
echopop_datasetsDict[str, Dict[int, geopandas.GeoDataFrame]]: Nested dictionary of Echopop data keyed by report type and year.
columnslist: Columns to sum and compare. Defaults to ["abundance", "biomass", "nasc"].

Returns:

Tuple[pandas.DataFrame, pandas.DataFrame]

differences: Magnitude differences (EchoPro - Echopop), indexed by (report_type, year).
pct_diff: Percent differences relative to the mean of both datasets, indexed by (report_type, year).

echopop.reports.compare.load_all_geodata_reports(years: list, echopro_root: Callable[[int], Path], echopop_root: Callable[[int], Path], echopop_patterns: dict[str, str], echopro_patterns: dict[str, str], cache_dir: Path | None = None, max_workers: int | None = None, verbose: bool = False) → tuple[dict[str, dict[int, GeoDataFrame]], dict[str, dict[int, GeoDataFrame]]]#

Load all geodata reports across years and dataset types.

By default, files are loaded sequentially. Parallel loading can be enabled by setting max_workers to an integer greater than 1, which uses a thread pool sized accordingly.

Parameters:

yearslist: Survey years to process.
echopro_rootCallable[[int], Path]: Function mapping year -> EchoPro report directory.
echopop_rootCallable[[int], Path]: Function mapping year -> Echopop report directory.
echopop_patternsDict[str, str]: Regex filename patterns for Echopop reports, keyed by type.
echopro_patternsDict[str, str]: Regex filename patterns for EchoPro reports, keyed by type.
cache_dirOptional[Path]: Directory for caching parquet files. If None, caching is disabled.
max_workersOptional[int]: Number of threads for parallel loading. If None (default), files are loaded sequentially. Set to a positive integer (e.g. 8) to enable parallel loading.
verbosebool, default = False: Boolean argument that will iteratively print out the loaded filepaths when set to True.

Returns:

Dict[Tuple[str, str, int], geopandas.GeoDataFrame]: Dictionary keyed by (dataset, type, year).

echopop.reports.compare.plot_dataset_differences(percent_differences: DataFrame, save_filepath: Path | None = None, columns: list = None, figsize: tuple[int, int] = (14, 6)) → None#

Plot the percent differences between EchoPro and Echopop population estimates.

Plot percent differences between EchoPro and Echopop report outputs as a heatmap grid, with one panel per report type (transect, kriging).

Parameters:

percent_differencespandas.DataFrame: Percent differences indexed by (report_type, year), as returned by compute_dataset_differences.
save_filepathOptional[Path]: If provided, the figure is saved to this path at 300 dpi. If None, the figure is only displayed.
columnslist: Column display labels for the x-axis. Defaults to ["abundance", "biomass", "nasc"].
figsizeTuple[int, int]: Figure size in inches. Defaults to (14, 6).

echopop.reports.compare.plot_geodata(echopro: GeoDataFrame, echopop: GeoDataFrame, save_filepath: Path | dict[Any, Path], show_plot: bool = True, log_transform: bool = False) → None#: Plot georeferenced transect and kriging mesh data.

echopop.reports.compare.plot_haul_count_comparisons(echopro: dict[str, DataFrame], echopop: dict[str, DataFrame], save_filepath: Path | None = None, show_plot: bool = True)#

Plot the differences in fish counts per haul.

For each sex in the intersection of both dicts, plot:

EchoPro heatmap
EchoPop heatmap
Difference heatmap.

Parameters:

echoproDict[str, pd.DataFrame]
echopopDict[str, pd.DataFrame]
save_filepathOptional[Path]: If provided, the plot will be saved to this location.
show_plotbool: If True, the plot will be rendered in the user’s session.

echopop.reports.compare.plot_population_table_comparisons(echopro: dict[str, DataFrame], echopop: dict[str, DataFrame], save_filepath: Path | None = None, show_plot: bool = True, log_transform: bool = False)#

Plot the differences in population estimates.

For each sex in the intersection of both dicts, plot:

EchoPro heatmap
EchoPop heatmap
Difference heatmap.

Parameters:

echoproDict[str, pd.DataFrame]
echopopDict[str, pd.DataFrame]
save_filepathOptional[Path]: If provided, the plot will be saved to this location.
show_plotbool: If True, the plot will be rendered in the user’s session.
log_transform: bool: If True, al values within the DataFrames are log-transformed (base-10).

echopop.reports.compare.read_aged_geodata(filepath: Path) → dict[str, GeoDataFrame]#

Read in georeferenced aged population estimates from transects and kriged mesh nodes.

This outputs a geopandas.GeoDataFrame object with the geometry informed by columns associated with ‘longitude’ and ‘latitude’. The EPSG:4326 projection is automatically applied to the coordinates.

echopop.reports.compare.read_geodata(filepath: Path) → GeoDataFrame#

Read in georeferenced population estimates from along-transect intervals of kriging mesh nodes.

This outputs a geopandas.GeoDataFrame object with the geometry informed by columns associated with ‘longitude’ and ‘latitude’. The EPSG:4326 projection is automatically applied to the coordinates.

echopop.reports.compare.read_pivot_table_report(filepath: Path) → dict[str, DataFrame]#

Read all sheets from the aged length haul counts report Excel file.

This dynamically assigns sex to the sheet name and returns a dictionary {sex: DataFrame}. Drops last column and row (subtotal) for each column and row, respectively.

Report generation

Contents

Report generation#

Report comparisons#