caput.pipeline.tasklib.io#

A collection of tasks for loading and saving files.

Most of these are just variations on loading and saving Container containers from different input strings or containers attributes (or combinations thereof).

File Groups#

Several tasks accept groups of files as arguments. These are specified in the YAML file as a dictionary like below.

list_of_file_groups:
    -   tag: first_group  # An optional tag naming the group
        files:
            -   'file1.h5'
            -   'file[3-4].h5'  # Globs are processed
            -   'file7.h5'

    -   files:  # No tag specified, implicitly gets the tag 'group_2'
            -   'another_file1.h5'
            -   'another_file2.h5'


single_group:
    files: ['file1.h5', 'file2.h5']

Classes#

`BaseLoadFiles`	Base class for loading containers from a file on disk.
`FindFiles`	Take a glob or list of files and pass on to other tasks.
`LoadFiles`	Load data from files passed into the setup routine.
`LoadFilesAndSelect`	Load a collection of files on setup and select specific entries on process.
`LoadFilesFromAttrs`	Load files from paths constructed using the attributes of another container.
`LoadFilesFromParams`	Load data from files given in the tasks parameters.
`LoadFilesFromPathAndTag`	Load files using all combinations of given paths and tags.
`Save`	Save out the input, and pass it on.
`SaveZarrZip`	Save a container as a .zarr.zip file.
`SelectionsMixin`	Mixin for parsing axis selections, typically from a yaml config.
`Truncate`	Precision truncate data prior to saving with compression.
`WaitZarrZip`	Collect Zarr-zipping jobs and wait for them to complete.
`ZarrZipHandle`	A handle for keeping track of background Zarr-zipping job.
`ZipZarrContainers`	Zip up a Zarr container into a single file.

Functions#

`list_of_filegroups`(groups)	Process a file group/groups.
`list_of_filelists`(files)	Take in a list of lists/glob patterns of filenames.
`list_or_glob`(files)	Take in a list of lists/glob patterns of filenames.