ZipZarrContainers

class caput.pipeline.tasklib.io.ZipZarrContainers

Bases: caput.pipeline.tasklib.base.ContainerTask

Zip up a Zarr container into a single file.

This is useful to save on file quota and speed up IO by combining the chunk data into a single file. Note that the file cannot really be updated after this process has been performed.

As this process is IO limited in most cases, it will attempt to parallelise the compression across different distinct nodes. That means at most only one rank per node will participate.

Attributes:

containerslist[str]

The names of the Zarr containers to compress. The zipped files will have the same names with .zip appended.

removebool

Remove the original data when finished. Defaults to True.

Methods

process()	Compress the listed zarr containers.
setup([_])	Setup the task.