GatherWorkspaces v1

Summary

Stitches together the input workspaces provided by each of the processes into a single workspace.

Properties

Name	Direction	Type	Default	Description
InputWorkspace	Input	MatrixWorkspace	Mandatory	The InputWorkspace to be gathered. Input workspace is optional, except for the root process.
OutputWorkspace	Output	MatrixWorkspace		Output is optional - only the root process will output a workspace
PreserveEvents	Input	boolean	False	Keep the output workspace as an EventWorkspace, if the input has events. If false, then the workspace gets converted to a Workspace2D histogram (default to save memory for reduced data)
AccumulationMethod	Input	string	Append	Method to use for accumulating each chunk from MPI processors. - Add: the processed chunk will be summed to the previous output. - Append: the spectra of the chunk will be appended to the output workspace, increasing its size. Allowed values: [‘Add’, ‘Append’]
ChunkSize	Input	number	0	Number of spectra to process at a time. Use 0 for automatic chunk sizing (recommended, targets ~100MB chunks), or -1 to process all spectra at once (fastest but uses most memory). A positive value specifies exact number of spectra per chunk.

Description

GatherWorkspaces collects workspaces from all MPI processes and combines them on the root process (rank 0). The algorithm supports two accumulation methods: “Add” mode sums corresponding spectra across all processes (adding Y values and combining errors in quadrature), while “Append” mode concatenates all spectra to create a larger workspace. Only processes with input workspaces participate in the gather operation, and only the root process produces an output workspace. The algorithm uses chunked processing to handle large workspaces efficiently, balancing memory usage with communication overhead. All input workspaces must have the same number of bins and must all be either histogram or point data. For EventWorkspaces, the algorithm can preserve event-level data if requested, otherwise it converts to histogram format to reduce memory usage. This is a collective MPI operation requiring all participating processes to call the algorithm simultaneously.

Usage

Example - Append

The following code should be saved as gather.py

from mantid.simpleapi import CreateWorkspace, GatherWorkspaces
from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

# Each rank creates a rank-local workspace
x = [0.0, 1.0, 2.0, 3.0]
y = [rank + 1.0, rank + 2.0, rank + 3.0]
e = [1.0, 1.0, 1.0]

input_ws = CreateWorkspace(
    DataX=x,
    DataY=y,
    DataE=e,
    NSpec=1,
    OutputWorkspace=f"input_rank_{rank}"
)

output_ws = GatherWorkspaces(
    InputWorkspace=input_ws,
    AccumulationMethod="Append"
)

if rank == 0:
    print("Number of spectra in output:", output_ws.getNumberHistograms())
    for i in range(output_ws.getNumberHistograms()):
        print(f"Spectrum {i} Y:", output_ws.readY(i))

To run:

mpiexec -n 4 gather.py

Output:

Number of spectra in output: 4
Spectrum 0 Y: [1. 2. 3.]
Spectrum 1 Y: [2. 3. 4.]
Spectrum 2 Y: [3. 4. 5.]
Spectrum 3 Y: [4. 5. 6.]

Categories: AlgorithmIndex | MPI

Source

C++ header: GatherWorkspaces.h

C++ source: GatherWorkspaces.cpp