caput.util.mpitools#

Utilities for making MPI usage transparent.

This module exposes much of the functionality of mpi4py but will still run in serial if MPI is not present on the system. It is, thus, useful for writing code that can be run in either parallel or serial. Also it exposes all attributes of the mpi4py.MPI module by the _SelfWrapper class for convenience. You can just use:

mpitools.attr

instead of:

from mpi4py import MPI

MPI.attr

Attributes#

`rank`	MPI rank of the current process.
`rank0`	True if this rank is rank 0.
`size`	Total number of MPI ranks.
`world`	Global MPI communicator, if MPI is enabled.
`world_scomm`	Global shared-memory MPI communicator, if MPI is enabled.

Classes#

MPILogFilter

Filter log entries by MPI rank.

Functions#

`active`(→ mpi4py.MPI.Comm \| None)	Make processes listed in aprocs active, while others wait.
`active_comm`(→ mpi4py.MPI.Comm \| None)	Return a communicator consisting of processes listed in aprocs.
`allocate_hdf5_dataset`(→ tuple[int, int])	Create a hdf5 dataset and return its offset and size.
`allreduce`(→ Any)	Call comm.allreduce if MPI is enabled.
`available_memory_shared`(→ int)	Shared memory in bytes available to a process.
`barrier`(→ None)	Call comm.Barrier() if MPI is enabled.
`bcast`(→ Any)	Call comm.bcast if MPI is enabled.
`can_allocate`(→ bool)	Check if nbytes of memory is available to allocate.
`close`(→ None)	Send a message to the waiting processes to close their waiting.
`cpu_count`(→ int \| None)	Get the number of CPUs available to each process.
`enable_mpi_exception_handler`()	Install an MPI-aware exception handler.
`gather_local`(→ None)	Gather data array in each process to the global array in root process.
`lock_and_write_buffer`(→ None)	Write buffer contents to disk at a given locked offset.
`mpirange`(→ list)	MPI aware version of range, each process gets its own sub section.
`parallel_map`(→ list \| None)	Apply a parallel map using MPI.
`parallel_rows_write_hdf5`(→ None)	Write out array (distributed across processes row wise) into a HDF5 in parallel.
`partition_list_mpi`(→ list)	Return the partition of a list specific to the current MPI process.
`split_all`(→ numpy.ndarray)	Split a range of integers `[0, n)` into sub-ranges for each MPI Process.
`split_local`(→ numpy.ndarray)	Split a range of integers `[0, n)` into sub-ranges for each MPI Process.
`transpose_blocks`(→ numpy.ndarray)	Swap 2D matrix split from row-wise to columnwise.
`typemap`(→ mpi4py.MPI.Datatype)	Map a numpy dtype into an MPI_Datatype.

Module Contents#

rank: int#: MPI rank of the current process.

rank0: bool#: True if this rank is rank 0.

size: int#: Total number of MPI ranks.

world: mpi4py.MPI.Comm | None#: Global MPI communicator, if MPI is enabled.

world_scomm: mpi4py.MPI.Comm | None#: Global shared-memory MPI communicator, if MPI is enabled.