Filter off-effort transect intervals

Filter off-effort transect intervals#

Removing off-effort intervals from transects#

Raw transect intervals from the Echoview exports sometimes require additional filtering to remove off-effort portions (e.g., net hauls, instrument retrieval) that were not completely omitted during processing. Indexing the transect intervals using vessel log distances delineating on-effort intervals. The complete set of these intervals can be used as an inclusion filter to effectively remove off-effort intervals. The result from this filter yields the final filtered transect intervals that are used in Echopop. There are three general cases that this filter is applied.

Transect ends (single intervals)#

The simplest case comprises single intervals at the start and end of the transects that are outside the extent of the on-effort intervals.

Transect filtering

Transect ends (multiple intervals)#

The second case is similar to the first where instead of just one interval at the start and/or end of the transects, there are multiple.

Transect filtering

Along-transect#

The third and final cases consists of off-effort intervals that occur at one or more points along a transect.

Transect filtering

Implementation within Echopop#

Off-effort transect intervals have to be removed via Echopop for some survey years (e.g. 2011). This can be accomplished using the filter_transect_intervals function from the utils sub-package. This function has four arguments:

  • nasc_data: A pandas.DataFrame containing the merged and consolidated NASC exports from Echoview.

  • transect_filter: This argument can either a pandas.DataFrame or pathlib.Path.

    • DataFrame: There are four required columns:

      • transect_num: Transect number.

      • log_start: The starting vessel log distance of teac along-transect interval.

      • log_end: The ending vessel log distance of each along-transect interval.

    • Path: This is the filepath to a *.xlsx and must be accompanied by a sheetname and filter (as explained below).

  • transect_filter_sheet: An optional argument that is only required when the transect_filter argument is a pathlib.Path object.

  • survey_filter: An optional argument that uses query strings to subset transect_filter. When used with a pathlib.Path object for transect_filter, then the read-in must have the column name defined in the string. For instance, a valid input for survey_filter could be "survey == 200316", which would remove any rows from the transect_filter pandas.DataFrame whose value for the survey column did not equal 200316. Another valid survey_filter could be "survey == 201103 and transect_num > 100", which would not only limit rows to those satisfying survey == 201103, but also only apply the filter to rows where transect_num > 100.

So calling this function would look like:

from pathlib import Path
from echopop.utils import filter_transect_intervals

# DATA ROOT DIRECTORY
DATA_ROOT = Path("C:/Data/EchopopData/echopop_2011")

# TRANSECT BOUNDARY FILE
TRANSECT_BOUNDARY_FILE = DATA_ROOT / "Kriging_files/Kriging_grid_files/Transect Bounds to 2011.xlsx"
# TRANSECT BOUNDARY SHEET
TRANSECT_BOUNDARY_SHEET = "1995-2011"
# SURVEY FILTER
SURVEY_FILTER = "survey == 201103"

# CALL FUNCTION
df_nasc_filtered = filter_transect_intervals(
    nasc_data=df_nasc,
    transect_filter=TRANSECT_BOUNDARY_FILE,
    transect_filter_sheet=TRANSECT_BOUNDARY_SHEET,
    survey_filter=SURVEY_FILTER
)

For this particularly example (i.e., the 2011 survey), the effect of the filtering can be visualized below, where 4140 off-effort intervals were removed from the NASC pandas.DataFrame: