Enhancement: Checkpointing and Resume Support for Dictionary Indexing in HPC Environments
Add checkpointing and resume capabilities to the dictionary indexing so that intermediate results are periodically written to disk and the program can resume from the last checkpoint after interruption. This is critical for high-performance computing environments where jobs are subject to unexpected failures or enforced wall time limits.
In HPC settings, jobs often have strict maximum runtime limits. For example, at LLNL jobs are capped at 24 hours and are automatically terminated when the wall time limit is reached. Dictionary indexing for large scans, especially for lower symmetry crystals, can exceed this limit. Without checkpointing, these jobs become impossible to run in HPC environments.
Enhancement: Checkpointing and Resume Support for Dictionary Indexing in HPC Environments
Add checkpointing and resume capabilities to the dictionary indexing so that intermediate results are periodically written to disk and the program can resume from the last checkpoint after interruption. This is critical for high-performance computing environments where jobs are subject to unexpected failures or enforced wall time limits.
In HPC settings, jobs often have strict maximum runtime limits. For example, at LLNL jobs are capped at 24 hours and are automatically terminated when the wall time limit is reached. Dictionary indexing for large scans, especially for lower symmetry crystals, can exceed this limit. Without checkpointing, these jobs become impossible to run in HPC environments.