General processing functions#
FEAT general-purpose workflow utility functions.
Provides spatial, interpolation, and data-manipulation helpers used across FEAT survey workflows, including transect extent extraction, kriging mesh preparation, and adaptive nearest-neighbor search utilities.
- echopop.utils.feat_functions.convert_afsc_nasc_to_feat(nasc_data: DataFrame, default_interval_distance: float = 0.5, default_transect_spacing: float = 10.0, inclusion_filter: dict[str, Any] = None, exclusion_filter: dict[str, Any] = None) DataFrame#
Convert AFSC-MACE to NWFSC-FEAT transect NASC format.
- Parameters:
- nasc_datapd.DataFrame
DataFrame containing AFSC NASC data.
- default_interval_distancefloat, optional
Default distance interval for transects, by default 0.5.
- default_transect_spacingfloat, optional
Default transect spacing, by default 10.0.
- inclusion_filterDict[str, Any], optional
Filter to include specific rows based on column values, by default {}.
- exclusion_filterDict[str, Any], optional
Filter to exclude specific rows based on column values, by default {}.
- Returns:
- pd.DataFrame
Transformed DataFrame corresponding to the expected NWFSC-FEAT format.
- echopop.utils.feat_functions.filter_transect_intervals(nasc_data: DataFrame, transect_filter: DataFrame | Path, survey_filter: str | None = None, transect_filter_sheet: str | None = None) DataFrame#
Filter transect intervals based on log start and end values.
- Parameters:
- nasc_datapandas.DataFrame
DataFrame containing NASC data with columns ‘transect_num’, ‘distance_s’, and ‘distance_e’
- transect_filterUnion[pandas.DataFrame, Path]
DataFrame containing transect filter data with columns ‘transect_num’, ‘log_start’, and ‘log_end’, or a filepath that reads in a file.
- survey_filterstr, optional
Query survey string to filter
transect_filter(e.g., “region_id == ‘A’”)- transect_filter_sheetstr, optional
Optional sheetname if a filename is input
- Returns:
- pandas.DataFrame
Filtered DataFrame containing only rows that overlap with the specified transect intervals
Examples
>>> nasc_data = pd.DataFrame({ ... 'transect_num': [1, 1, 2, 2], ... 'distance_s': [0, 1, 0, 1], ... 'distance_e': [1, 2, 1, 2], ... 'nasc': [10, 20, 30, 40] ... }) >>> filter_data = pd.DataFrame({ ... 'transect_num': [1, 2], ... 'log_start': [0.5, 0.5], ... 'log_end': [1.5, 1.5], ... 'region_id': ['A', 'B'] ... }) >>> result = filter_transect_intervals(nasc_data, filter_data)
- echopop.utils.feat_functions.get_survey_western_extents(transects: DataFrame, coordinate_names: tuple[str, str], latitude_threshold: float) DataFrame#
Get the western extents of each survey transect line.
Get the western extents of each survey transect that can be used to constrain the adaptive nearest neighbors search algorithm incorporated into the kriging interpolation algorithm.
- Parameters:
- transectspd.DataFrame
A dataframe containing georeferenced coordinates associated with a particular variable (e.g. biomass). This DataFrame must have at least two valid columns comprising the overall 2D coordinates (e.g. ‘x’ and ‘y’). Furthermore, this function requires that transects also contain a column called ‘latitude’.
- coordinates_namesTuple[str, str], default = (‘x’, ‘y’)
A tuple containing the ‘transects’ column names defining the coordinates. The order of this input matters where they should be defined as the (horizontal axis, vertical axis).
- latitude_thresholdfloat
A threshold that is applied to the georeferenced coordinates that further constrains any extrapolation that occurs during the kriging analysis.
- Returns:
- pd.DataFrame
A DataFrame comprising three columns: ‘transect_num’, and the column names supplied by the argument coordinate_names.
- echopop.utils.feat_functions.region_13_conditions(x, boundary_column: str, position: str)#
Map mesh regions 1 and 3 boundaries based on encoded transect boundary values.
This helper function interprets the encoded boundary values for regions 1 and 3, which use latitude-parallel transects. The encoding uses decimal fractions to indicate spatial positions (e.g., .1 for west, .4 for east, .6 for south, .9 for north).
- Parameters:
- xpd.DataFrame
DataFrame subset containing transect data for a specific transect number
- boundary_columnstr
Name of the column containing the encoded boundary values
- positionstr
Position identifier (‘east’, ‘west’, ‘north’, ‘south’) used for output column naming
- Returns:
- pd.Series
Series containing longitude and latitude coordinates for the specified position, with column names formatted as ‘longitude_{position}’ and ‘latitude_{position}’
Notes
The encoding scheme: - x.1: Western boundary (minimum longitude) - x.4: Eastern boundary (maximum longitude) - x.6: Southern boundary (minimum latitude) - x.9: Northern boundary (maximum latitude)
- echopop.utils.feat_functions.region_2_conditions(x, boundary_column: str, position: str)#
Map mesh region 2 boundaries based on encoded transect boundary values.
This helper function interprets the encoded boundary values for region 2, which uses longitude-parallel transects. The encoding uses decimal fractions to indicate spatial positions.
- Parameters:
- xpd.DataFrame
DataFrame subset containing transect data for a specific transect number
- boundary_columnstr
Name of the column containing the encoded boundary values
- positionstr
Position identifier (‘north’, ‘south’) used for output column naming
- Returns:
- pd.Series
Series containing longitude and latitude coordinates for the specified position, with column names formatted as ‘longitude_{position}’ and ‘latitude_{position}’
Notes
The encoding scheme for region 2: - x.1: Western boundary (minimum longitude) - x.4: Eastern boundary (maximum longitude) - x.6: Southern boundary (minimum latitude) - x.9: Northern boundary (maximum latitude)
Region 2 transects run parallel to longitudes (north-south orientation).
- echopop.utils.feat_functions.transect_ends_crop(transects: DataFrame, mesh: DataFrame, latitude_resolution: float, transect_mesh_region_function: Callable) tuple[DataFrame, DataFrame]#
Crop the kriging mesh by interpolating the eastern and western transect extents.
Crop the kriging mesh by interpolating the eastern and western extents of survey transects partitioned into discrete regions via user-defined sorting functions. This function processes survey transect data to define spatial boundaries for mesh cropping. It divides transects into three regions, interpolates their boundaries, and identifies mesh cells that fall within the survey extent.
- Parameters:
- transectspd.DataFrame
Georeferenced survey transect data used for defining the spatial extent for the kriging mesh grid. Must contain columns: ‘transect_num’, ‘longitude’, ‘latitude’.
- meshpd.DataFrame
Complete kriging mesh DataFrame that is subsequently cropped. Must contain columns: ‘longitude’, ‘latitude’.
- latitude_resolutionfloat
The latitudinal resolution (in degrees) used for the interpolation. This determines the spacing between interpolation points and affects the precision of boundary detection.
- transect_mesh_region_functionCallable
A sorting function that maps specific transect numbers to their respective discretized mesh regions. The outputs of this function are expected to be:
‘transect_start’ (np.number): the first transect number of a particular region,
‘transect_end’ (np.number): the last transect number of a particular region,
‘transect_lower_bound’ (List[np.number]): a list of encoded transect numbers indicating whether the transect is north, south, west, or east,
‘transect_upper_bound’ (List[np.number]): a list of encoded transect numbers indicating whether the transect is north, south, west, or east.
- Returns:
- Tuple[pd.DataFrame, pd.DataFrame]
A tuple comprising:
Cropped kriging mesh DataFrame containing only cells within the survey extent
Annotated transect data with each transect number’s respective mesh region assignment
Examples
>>> from echopop.nwfsc_feat.FEAT import transect_mesh_region_2019 >>> cropped_mesh, annotated_transects = transect_ends_crop( ... transects, mesh, 0.05, transect_mesh_region_2019 ... ) >>> print(f"Original mesh size: {len(mesh)}") >>> print(f"Cropped mesh size: {len(cropped_mesh)}")
- echopop.utils.feat_functions.western_boundary_search_strategy(kriging_mesh: DataFrame, western_extent: DataFrame, coordinate_names: tuple[str, str], sparse_radii: ndarray[int], valid_distances: ndarray[int], local_points: ndarray[float], distance_matrix_masked: ndarray[float], nearby_indices: ndarray[int], k_min: int, k_max: int, search_radius: float, wr_indices: ndarray[int], oos_indices: ndarray[number], oos_weights: ndarray[float], **kwargs) tuple[ndarray[number], ndarray[number], ndarray[number]]#
Search strategy that applies western boundary constraints for transect-based surveys.
- Parameters:
- kriging_meshpd.DataFrame
Kriging mesh used for interpolated data values via geostatistics.
- western_extentpd.DataFrame
DataFrame with the western-most extent of each transect line used for re-weighting the out-of-sample/extrapolated kriged values.
- coordinate_namesTuple[str, str]
Names of the coordinate columns when using DataFrames. Expected format: (x_col, y_col).
- sparse_radiinp.ndarray[int]
Indices where there are fewer than k_min nearest neighbors.
- valid_distancesnp.ndarray[int]
The number of masked distance matrix values where extrapolation is required.
- local_pointsnp.ndarray[float]
An array with the sorted distances (from nearest to furthest) relative to each point.
- distance_matrix_maskednp.ndarray[float]
An array with the search-radius-masked nearest neighbor distances.
- nearby_indicesnp.ndarray[int]
Indices of points that require extrapolation.
- k_minint
The minimum number of nearest neighbors required for including values for kriging within the search radius.
- k_maxint
The maximum number of nearest neighbors required for including values for kriging detected within the search radius.
- search_radiusfloat
The adaptive search radius that identifies the k-nearest neighbors around each georeferenced value that are subsequently kriged.
- wr_indicesnp.ndarray[int]
Indices of within-radius (WR) (i.e. < k_max) points.
- oos_indicesnp.ndarray[np.number]
Template array based on the size of the data input and k_min that will contain indices where extrapolation is required where there are fewer than k_min nearest neighbors.
- oos_weightsnp.ndarray[float]
Weights applied to extraplolated values.
- Returns:
- Tuple[np.ndarray[np.number], np.ndarray[np.number], np.ndarray[np.number]]
A tuple with updated values for wr_indices, oos_indices, and oos_weights via a search strategy that uses an extrapolation re-weighting based on transect extents.