map_parallel

map_parallel.map_parallel(f: Callable, *args: Iterable, processes: Optional[int] = None, mode: str = 'multiprocessing', return_results: bool = True) → list[source]

equiv to map(f, *args) but in parallel

Parameters

  • mode (str) –

    backend for parallelization

    • multiprocessing: using multiprocessing from standard library

    • multithreading: using multithreading from standard library

    • dask: using dask.distributed

    • mpi: using mpi4py.futures. May not work depending on your MPI vendor

    • mpi_simple: using mpi4py with simple scheduling that divides works into equal chunks

    • serial: using map

  • processes (int) –

    no. of parallel processes

    (in the case of mpi, it is determined by mpiexec/mpirun args)

  • return_results (bool) – (Only affects mode == ‘mpi_simple’) if True, return results in rank 0.

map_parallel.starmap_parallel(f: Callable, args: Iterable[Iterable], processes: Optional[int] = None, mode: str = 'multiprocessing', return_results: bool = True) → list[source]

equiv to starmap(f, args) but in parallel

See docstring from map_parallel()