Ingest AFSC acoustic survey data

Ingest AFSC acoustic survey data#

The 1995, 1998, and 2001 Pacific hake surveys were conducted by the Alaska Fisheries Science Center (AFSC). The resulting integrated NASC files are formatted slightly differently than those generated by the Northwest Fisheries Science Center (NWFSC).

Key differences from standard NASC ingestion#

The standard NWFSC ingestion pathway (read_nasc_file) expects a haul number column and constructs a unique haul identifier (UID) directly from the input file. AFSC-formatted files lack this column entirely, so a two-step ingestion pattern is required:

  1. Read the raw AFSC file with read_afsc_nasc_file.

  2. Convert the result into FEAT-compatible format with convert_afsc_nasc_to_feat .

Reading AFSC NASC files#

AFSC-formatted NASC exports are read using read_afsc_nasc_file from the ingest.nasc module. Unlike the standard NWFSC pathway, this function does not expect or construct a haul UID — that step is deferred to downstream processing. It accepts an Excel workbook path and worksheet name, and optionally imputes missing or invalid coordinate values (such as 999.0 or NaN) using linear interpolation from adjacent valid points. A column name mapping dictionary can also be provided to align the raw export’s column names with the Echopop-standard schema before any further processing occurs. It has four arguments:

  • filename: Path to the Excel workbook containing the NASC export.

  • sheetname: Name of the worksheet to read.

  • impute_coordinates: If True (default), imputes bad or missing latitude/longitude values using linear interpolation from adjacent valid points.

  • column_name_map: An optional dictionary that maps original column names to Echopop-standard names. All column names are lowercased before mapping is applied.

from echopop.ingest import nasc as ingest_nasc

FEAT_TO_ECHOPOP_COLUMNS = {
    "transect": "transect_num",
    "log": "distance",
    "lat": "latitude",
    "long": "longitude",
    "strata": "stratum_ks",
    "width": "transect_spacing",
    "mean layer depth": "layer_depth_mean",
    "depth": "bottom_depth",
}

df_nasc = ingest_nasc.read_afsc_nasc_file(
    filename="path/to/nasc_exports.xlsx",
    sheetname="Sheet1",
    column_name_map=FEAT_TO_ECHOPOP_COLUMNS,
)

Converting to FEAT format#

After reading, the DataFrame must be restructured to match the FEAT transect interval schema expected by the rest of the ingestion pipeline. This is handled by convert_afsc_nasc_to_feat, which computes interval end distances from interval start distances, fills missing transect spacing values with a configurable default, and applies inclusion or exclusion filters to restrict the data to a specific set of transects or other column values. It has five arguments:

  • nasc_data: The DataFrame returned by read_afsc_nasc_file.

  • default_interval_distance: Along-transect interval length in nautical miles used to compute distance_e from distance_s. Defaults to 0.5.

  • default_transect_spacing: Cross-transect spacing in nautical miles applied wherever transect_spacing is NaN. Defaults to 10.0.

  • inclusion_filter: An optional dictionary that keeps only rows where column values match the provided arrays or scalar values.

  • exclusion_filter: An optional dictionary that removes rows where column values match the provided arrays or scalar values.

import numpy as np
from echopop.workflows.nwfsc_feat import functions as feat

df_nasc = feat.convert_afsc_nasc_to_feat(
    nasc_data=df_nasc,
    default_interval_distance=0.5,
    default_transect_spacing=10.0,
    inclusion_filter={"transect_num": np.arange(1, 400)},
)

Typical usage#

A complete AFSC ingestion sequence combines both steps in order. After convert_afsc_nasc_to_feat returns, the resulting pandas.DataFrame matches the FEAT interval schema and can be passed directly into the transect-filtering, stratification, and inversion stages of the standard pipeline.

import numpy as np
from echopop.ingest import nasc as ingest_nasc
from echopop.utils import convert_afsc_nasc_to_feat

FEAT_TO_ECHOPOP_COLUMNS = {
    "transect": "transect_num",
    "log": "distance",
    "lat": "latitude",
    "long": "longitude",
    "strata": "stratum_ks",
    "width": "transect_spacing",
    "mean layer depth": "layer_depth_mean",
    "depth": "bottom_depth",
}

# Step 1: Read
df_nasc = ingest_nasc.read_afsc_nasc_file(
    filename="path/to/nasc_exports.xlsx",
    sheetname="Sheet1",
    column_name_map=FEAT_TO_ECHOPOP_COLUMNS,
)

# Step 2: Convert
df_nasc = convert_afsc_nasc_to_feat(
    nasc_data=df_nasc,
    default_interval_distance=0.5,
    default_transect_spacing=10.0,
    inclusion_filter={"transect_num": np.arange(1, 400)},
)

Post-ingestion requirements#

Because read_afsc_nasc_file does not add region, layer height, or layer mean depth columns, these must be added manually before continuing into downstream stages:

df_nasc["region_id"] = 999
df_nasc["layer_height"] = 0.0
df_nasc["layer_mean_depth"] = 0.0

These stand-in values satisfy the schema requirements of downstream processing stages when the original AFSC export does not provide layer metadata.