LexicalFrequencyLookupExtractor#

Defined in: voxatlas.features.lexical.frequency.lookup

class voxatlas.features.lexical.frequency.lookup.LexicalFrequencyLookupExtractor[source]#

Bases: BaseExtractor

Extract the lexical.frequency.lookup feature within the VoxAtlas pipeline.

This public extractor defines the reusable API for computing lexical.frequency.lookup from VoxAtlas structured inputs. It consumes token units and produces values aligned to token units, making the extractor a stable pipeline node that can be cited independently of the surrounding execution machinery.

Algorithm#

The extractor performs lemma-based frequency lookup for token units.

  1. Lexicon retrieval A language-specific frequency lexicon is loaded into memory and converted to a lookup dictionary keyed by lemma-like forms.

  2. Token normalization Each token row contributes the first available lemma/form/text field used for lexicon matching.

  3. Lookup The output value is

    \[x_i = L(w_i),\]

    where \(L\) maps a token lemma \(w_i\) to its raw corpus frequency.

  4. Packaging Values are returned as a token-aligned scalar series so later lexical statistics can operate on a common index.

Examples

>>> import pandas as pd
>>> from voxatlas.features.feature_input import FeatureInput
>>> from voxatlas.features.lexical.frequency.lookup import LexicalFrequencyLookupExtractor
>>> from voxatlas.units import Units
>>> tokens = pd.DataFrame([{"id": 1, "lemma": "hello"}])
>>> units = Units(tokens=tokens)
>>> params = LexicalFrequencyLookupExtractor.default_config.copy()
>>> params["lexicon"] = pd.DataFrame([{"lemma": "hello", "frequency": 10.0}])
>>> out = LexicalFrequencyLookupExtractor().compute(FeatureInput(audio=None, units=units, context={}), params)
>>> float(out.values.loc[0, "frequency"])
10.0
name: str = 'lexical.frequency.lookup'#
input_units: str | None = 'token'#
output_units: str | None = 'token'#
dependencies: list[str] = []#
default_config: dict = {'language': None, 'lexicon': None, 'resource_root': None}#
compute(feature_input, params)[source]#

Compute the extractor output for a single pipeline invocation.

This method is the reusable execution entry point for the extractor. It receives the standard FeatureInput bundle, applies the configured algorithm, and returns feature values aligned to the extractor output units for storage in the pipeline feature store.

Parameters:

  • feature_input (object) – Structured extractor input bundling audio, hierarchical units, and execution context for this feature computation.

  • params (object) – Resolved feature configuration for this invocation. Keys are feature-specific and merged from defaults and pipeline settings.

Returns:

Structured output aligned to the token unit level when applicable.

Return type:

FeatureOutput

Examples

>>> import pandas as pd
>>> from voxatlas.features.feature_input import FeatureInput
>>> from voxatlas.features.lexical.frequency.lookup import LexicalFrequencyLookupExtractor
>>> from voxatlas.units import Units
>>> tokens = pd.DataFrame([{"id": 1, "lemma": "hello"}])
>>> units = Units(tokens=tokens)
>>> params = LexicalFrequencyLookupExtractor.default_config.copy()
>>> params["lexicon"] = pd.DataFrame([{"lemma": "hello", "frequency": 10.0}])
>>> result = LexicalFrequencyLookupExtractor().compute(FeatureInput(audio=None, units=units, context={}), params)
>>> result.unit
'token'