Writing restart files

ESIO provides several methods that greatly simplify writing a sequence of metadata-rich restart files. Each restart file might contain a simulation snapshot at a particular time along with enough information to track how the data was generated and to allow data post-processing.

A complete, C-based MPI program using ESIO to write a (trivial) sequence of restart files might look like:

#include <stdlib.h>
#include <mpi.h>
#include <esio/esio.h>
 
int main(int argc, char *argv[])
{
    // Initialize MPI and arrange for its finalization
    MPI_Init(&argc, &argv);
    atexit((void (*) ()) MPI_Finalize);
    int world_size, world_rank;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
 
    // Create restart file template filled with metadata on a subset of ranks
    if (world_rank == 0) {
        esio_handle s = esio_handle_initialize(MPI_COMM_SELF);
        esio_file_create(s, "template.h5", 1 /* overwrite */);
        esio_string_set(s, "/", "program", argv[0]);
        esio_string_set(s, "/", "built",   __DATE__ " " __TIME__);
        esio_handle_finalize(s);
    }
    MPI_Barrier(MPI_COMM_WORLD);  // Prevent template create/clone race
 
    // Prepare a handle informed about the parallel domain decomposition
    esio_handle w = esio_handle_initialize(MPI_COMM_WORLD);
    esio_line_establish(w, 2*world_size, 2*world_rank, 2);
 
    // Main simulation loop would customarily be driven by time advancement
    double state[2];
    for (int i = 0; i < 10; ++i) {
 
        // Pretend to update simulation state for demonstration purposes
        state[0] = i * world_rank;
        state[1] = i * world_rank + 1;
 
        // Create an uncommitted restart file based on the template
        esio_file_clone(w, "template.h5", "uncommitted.h5", 1 /*overwrite*/);
 
        // Save the current simulation state in the uncommitted restart
        esio_line_write_double(w, "state", state, 1, "A comment");
 
        // Commit the restart file.  Here, up to 5 older restarts are retained
        esio_file_close_restart(w, "committed#.h5", 5 /*retain_count*/);
    }
 
    esio_handle_finalize(w);
}

This sample prepares a copy of loop-invariant data in the file "template.h5". Often this would include the program version, scenario parameters, and grid information. Then, when the code needs to write a new restart file during the simulation loop, a new uncommitted restart file called "uncommitted.h5" is cloned from "template.h5" using esio_file_clone(). After the program has successfully written all the necessary data to the restart file, the restart file is committed using esio_file_close_restart().

The documentation for the esio_file_close_restart() method contains several important details. The same functionality is also available in Fortran via the methods esio::esio_file_clone() and esio::esio_file_close_restart().

Using such an approach provides several benefits:

1. Common metadata is written only once. In a long-running simulation, this may provide some performance benefits compared to repeatedly writing the same metadata within the simulation loop.
2. Incomplete restart data is kept separate from complete restart data. This prevents a software or hardware failure from destroying your simulation's last known good state.
3. A history of restart data is maintained. This is useful when simulation statistics must be gathered across multiple snapshots.

ESIO also provides a standalone, non-HDF5 utility named esio_restart which can be used to test ESIO's restart file management capabilities. See esio_rename –help for more details.