3.12

MergeMDFiles v1

../_images/MergeMDFiles-v1_dlg.png

MergeMDFiles dialog.

Table of Contents

Summary

Merge multiple MDEventWorkspaces from files that obey a common box format.

Properties

Name Direction Type Default Description
Filenames Input list of str lists Mandatory Select several MDEventWorkspace NXS files to merge together. Files must have common box structure. Allowed extensions: [‘.nxs’]
OutputFilename Input string   Choose a file to which to save the output workspace. Optional: if specified, the workspace created will be file-backed. If not, it will be created in memory. Allowed extensions: [‘.nxs’]
Parallel Input boolean False Run the loading tasks in parallel. This can be faster but might use more memory.
OutputWorkspace Output MDEventWorkspace Mandatory An output MDEventWorkspace.

Description

This algorithm is meant to merge a large number of large MDEventWorkspaces together into one file-backed MDEventWorkspace, without exceeding available memory.

First, you will need to generate a MDEventWorkspaces NXS file for each run with a fixed box structure:

  • You can call CreateMDWorkspace v1 with MinRecursionDepth = MaxRecursionDepth.
    • This will make the box structure identical. The number of boxes will be equal to SplitInto ^ (NumDims * MaxRecursionDepth).
    • Aim for the boxes to be small enough for all events contained to fit in memory; without there being so many boxes as to slow down too much.
  • This can be done immediately after acquiring each run so that less processing has to be done at once.

Then, enter the path to all of the files created previously. The algorithm avoids excessive memory use by only keeping the events from ONE box from ALL the files in memory at once to further process and refine it. This is why it requires a common box structure.

See also

MergeMD v1, for merging any MDWorkspaces in system memory (faster, but needs more memory).

Categories: Algorithms | MDAlgorithms\Creation

Source

C++ source: MergeMDFiles.cpp (last modified: 2017-06-19)

C++ header: MergeMDFiles.h (last modified: 2017-06-19)