MemDiskGroup

class caput.memdata.MemDiskGroup(data_group=None, distributed=False, comm=None, file_format=None)

Bases: _BaseGroup

Group whose data may either be stored on disk or in memory.

This container is intended to have the same basic API h5py.Group and MemGroup but whose underlying data could live either on disk or in memory.

Aside from providing a few convenience methods, this class isn’t that useful by itself. It is almost as easy to use h5py.Group or MemGroup directly. Where it becomes more useful is for creating more specialized data containers which can subclass this class.

This class also supports the same distributed features as MemGroup, but only when wrapping that class. Attempting to create a distributed object wrapping a h5py.File object will raise an exception. For similar reasons, MemDiskGroup.to_disk() will not work, however, MemDiskGroup.save() will work fine.

Parameters:

data_groupfile_or_group_like

Underlying h5py like data container where data will be stored. If a string, open a h5py file with that name. If not provided a new MemGroup instance will be created.

distributedbool

Allow the container to hold distributed datasets.

commMPI.Comm | None

MPI Communicator to distributed over. If not set, use MPI.COMM_WORLD.

file_formatFileFormat | None

File format to use. File format will be guessed if not supplied. Default None.

property ondisk

Whether the data is stored on disk as opposed to in memory.

Methods

__deepcopy__(memo, /)	Called when copy.deepcopy is called on this class.
__getitem__(name)	Retrieve an object.
close()	Closes file if on disk if file was opened on initialization.
copy([shared, shallow])	Return a deep copy of this class or subclass.
create_dataset(name, *args, **kwargs)	Create and return a new dataset.
create_group(name)	Create and return a new group.
dataset_common_to_distributed(name[, distributed_axis])	Convert a common dataset to a distributed one.
dataset_distributed_to_common(name)	Convert a distributed dataset to a common one.
dataset_name_allowed(name)	Used by subclasses to restrict creation of and access to datasets.
flush()	Flush the buffers of the underlying hdf5 file if on disk.
from_file(file_[, ondisk, distributed, comm, ...])	Create data object from analysis hdf5 file, store in memory or on disk.
from_group([data_group, detect_subclass])	Create data object from a given group.
group_name_allowed(name)	Used by subclasses to restrict creation of and access to groups.
save(filename[, convert_attribute_strings, ...])	Save data to hdf5/zarr file.
to_disk(filename[, file_format])	Return a version of this data that lives on disk.
to_memory()	Return a version of this data that lives in memory.