treble_tsdk.scene.audioloaders

Functions

`cache_remote_file`(remote_path, cache_file_path)	Download a remote file to a local path, showing a progress bar.
`is_path_local`(path)	Return whether `path` exists on the local filesystem.

Classes

`AnechoicSoundsAudioLoader`	Load anechoic sound samples by fetching audio from a CDN URL stored in the parquet.
`AudioPathAudioLoader`	Load audio samples from a parquet that stores file paths.
`AudioSetAudioLoader`	Load AudioSet samples from a parquet with embedded audio bytes.
`BaseAudioLoader`	Base class for audio loaders backed by a parquet of sample metadata.
`CommonVoiceAudioLoader`	Load Common Voice samples from a parquet with embedded audio bytes.
`FMAAudioLoader`	Load FMA samples from a parquet with embedded audio bytes.
`GigaSpeechAudioLoader`	Load GigaSpeech samples from a parquet with embedded audio bytes.
`LibriSpeechAudioLoader`	Load LibriSpeech samples from a parquet with embedded audio bytes.
`MultilingualLibriSpeechAudioLoader`	Load Multilingual LibriSpeech samples from a parquet with embedded audio bytes.
`RowIndexAudioLoader`	Load samples from a parquet with embedded audio bytes and no natural id column.
`TedliumAudioLoader`	Load TED-LIUM samples from a parquet with embedded audio bytes.
`VoxPopuliAudioLoader`	Load VoxPopuli samples from a parquet with embedded audio bytes.

class treble_tsdk.scene.audioloaders.AnechoicSoundsAudioLoader

Load anechoic sound samples by fetching audio from a CDN URL stored in the parquet.

The parquet does not embed audio bytes; instead each row carries a url column pointing to a WAV file on the Treble CDN. Audio files are downloaded on first access and cached locally.

Required columns for a compatible parquet: - id (used to select a row). - url, the full CDN URL to the WAV file.

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.AudioPathAudioLoader

Load audio samples from a parquet that stores file paths.

Intended for datasets like https://huggingface.co/datasets/MahiA/GT-Music-Genre, where the parquet contains a path to a local folder of WAV files.

Requirements for a compatible parquet:

Must include an id column used to select a row.
Must include a path column (default: path) that points to a WAV file.
Path may be absolute or relative; relative paths are resolved against the parquet file’s parent directory.

To use your own dataset, create a parquet with the same structure (e.g. columns id, path, and optionally labels like classname) and ensure the referenced WAV files exist locally.

__init__(parquet_url: str, cache_directory: Path | None = None, path_column: str = 'path')

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.AudioSetAudioLoader

Load AudioSet samples from a parquet with embedded audio bytes.

Requirements for a compatible parquet:

Must include an identifier column (default: video_id) used to select a row.
Must include an audio column with either a bytes field containing audio data, or a path field with a URL to the audio file (e.g. hf://...). Stereo files are downmixed by taking the first channel.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="agkphysics/AudioSet",
    config="full",
    split="bal_train",
    audio_loader_class=treble.scene.AudioSetAudioLoader,
    schema_mapping={"id": "video_id"},
    drop_columns=["audio"]
)

__init__(parquet_url: str, cache_directory: Path | None = None, id_column: str = 'video_id')

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.BaseAudioLoader

Base class for audio loaders backed by a parquet of sample metadata.

__init__(loader_key: str, parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.CommonVoiceAudioLoader

Load Common Voice samples from a parquet with embedded audio bytes.

Based on https://huggingface.co/datasets/fixie-ai/common_voice_17_0

Required columns for a compatible parquet: - path (used to select a row, serves as the unique clip identifier). - client_id, identifies the speaker. - audio with a bytes field containing audio data. - sentence, the transcribed text.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="fixie-ai/common_voice_17_0",
    audio_loader_class=CommonVoiceAudioLoader,
    config="en",
    split="train",
    schema_mapping={"id": "path", "transcript": "sentence"},
    drop_columns=["audio"],
)

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.FMAAudioLoader

Load FMA samples from a parquet with embedded audio bytes.

Based on https://huggingface.co/datasets/benjamin-paine/free-music-archive-full

Requirements for a compatible parquet:

Must include a url column used to select a row.
Must include an audio column with a bytes field containing audio data. Stereo files are downmixed by taking the first channel.

Usage with AudioDataset:

dataset = treble.scene.AudioDataset.from_huggingface(
    repo_id="benjamin-paine/free-music-archive-full",
    audio_loader_class=FMAAudioLoader,
    split="train",
    schema_mapping={"id": "url"},
    drop_columns=["audio"]
)

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.GigaSpeechAudioLoader

Load GigaSpeech samples from a parquet with embedded audio bytes.

Based on https://huggingface.co/datasets/fixie-ai/gigaspeech

Required columns for a compatible parquet: - segment_id (used to select a row). - audio with a bytes field containing audio data. - text, the transcribed text.

The parquet also contains begin_time and end_time columns; when both are mapped via schema_mapping, AudioDataset automatically computes length_s = end_time - begin_time.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="fixie-ai/gigaspeech",
    audio_loader_class=GigaSpeechAudioLoader,
    config="xl",
    split="train",
    schema_mapping={
        "id": "segment_id",
        "transcript": "text",
        "begin_time": "begin_time",
        "end_time": "end_time",
    },
    drop_columns=["audio"],
)

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.LibriSpeechAudioLoader

Load LibriSpeech samples from a parquet with embedded audio bytes.

Based on https://huggingface.co/datasets/treble-technologies/librispeech_asr_sliced and https://huggingface.co/datasets/openslr/librispeech_asr.

Required columns for a compatible parquet: - id (used to select a row). - speaker_id, must be unique. - audio with a bytes field containing audio data.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="treble-technologies/librispeech_asr_sliced",
    split="test",
    audio_loader_class=treble.scene.LibriSpeechAudioLoader,
    schema_mapping={"transcript": "text"},
    drop_columns=["audio"]
)

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.MultilingualLibriSpeechAudioLoader

Load Multilingual LibriSpeech samples from a parquet with embedded audio bytes.

Based on https://huggingface.co/datasets/facebook/multilingual_librispeech

Required columns for a compatible parquet: - id (used to select a row). - audio with a bytes field containing audio data. - transcript, the transcribed text. - audio_duration, clip duration in seconds.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="facebook/multilingual_librispeech",
    audio_loader_class=MultilingualLibriSpeechAudioLoader,
    config="english",
    split="train",
    schema_mapping={"length_s": "audio_duration"},
    drop_columns=["audio"],
)

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.RowIndexAudioLoader

Load samples from a parquet with embedded audio bytes and no natural id column.

Intended for HF “audiofolder” datasets that only expose an audio column (plus labels), such as https://huggingface.co/datasets/437aewuh/dog-dataset. Since there is no column to key on, rows are addressed by position: pass no schema_mapping entry for id so AudioDataset synthesizes one as f"{shard_hash}_{row_index}", and this loader recovers row_index from that suffix.

Required columns for a compatible parquet:

audio with a bytes field containing audio data, or a path field with a URL to the audio file (e.g. hf://...). Stereo files are downmixed by taking the first channel.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="437aewuh/dog-dataset",
    config="default",
    split="train",
    audio_loader_class=treble.scene.RowIndexAudioLoader,
)

__init__(parquet_url: str, cache_directory: Path | None = None, audio_column: str = 'audio')

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.TedliumAudioLoader

Load TED-LIUM samples from a parquet with embedded audio bytes.

Based on https://huggingface.co/datasets/sanchit-gandhi/tedlium-data (Release 3)

Required columns for a compatible parquet: - id (used to select a row). - audio with a bytes field containing audio data. - text, the transcribed text.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="sanchit-gandhi/tedlium-data",
    audio_loader_class=TedliumAudioLoader,
    split="train",
    schema_mapping={"transcript": "text"},
    drop_columns=["audio"],
)

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

class treble_tsdk.scene.audioloaders.VoxPopuliAudioLoader

Load VoxPopuli samples from a parquet with embedded audio bytes.

Based on https://huggingface.co/datasets/facebook/voxpopuli

Required columns for a compatible parquet: - audio_id (used to select a row). - audio with a bytes field containing audio data. - normalized_text, the normalized transcript.

Usage with AudioDataset:

AudioDataset.from_huggingface(
    repo_id="facebook/voxpopuli",
    audio_loader_class=VoxPopuliAudioLoader,
    config="en",
    split="train",
    schema_mapping={"id": "audio_id", "transcript": "normalized_text"},
    drop_columns=["audio"],
)

__init__(parquet_url: str, cache_directory: Path | None = None)

Parameters:

loader_key – Unique identifier for this loader’s type, used for registry lookups.
parquet_url – Path or URL to the parquet file containing sample metadata.
cache_directory – Local directory for caching downloaded data. Defaults to an audio_datasets subfolder under the default TSDK cache directory.

classmethod get_default_loader_kwargs(schema_mapping: dict[str, str] | None) → dict[str, object]: Return default constructor kwargs derived from a schema_mapping. Subclasses override this to infer kwargs (e.g. id_column) from the mapping, so callers don’t need to pass audio_loader_kwargs explicitly.

get_registry_loader_kwargs() → dict[str, object]

Return constructor kwargs required to recreate this loader instance.

Returns:: Keyword arguments (beyond parquet_url and cache_directory) needed to reconstruct an equivalent loader instance.

treble_tsdk.scene.audioloaders.cache_remote_file(remote_path: str, cache_file_path: Path)

Download a remote file to a local path, showing a progress bar.

Parameters:

remote_path – Source URL, opened via fsspec.
cache_file_path – Local destination path; parent directories created as needed.

treble_tsdk.scene.audioloaders.is_path_local(path: str | Path) → bool: Return whether path exists on the local filesystem.