MPI global execution time

In most cases, it is often enough to simply keep track of the start and end time on the master node and derive the global run time on the the master only.

One thing worth noting is that you must place a barrier before collecting the start time (to make sure all nodes are ready to proceed), and before the end time (to make sure all nodes are done).

double start, end;

MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);

MPI_Barrier(MPI_COMM_WORLD); /* IMPORTANT */
start = MPI_Wtime();

/* ... do work ... */

MPI_Barrier(MPI_COMM_WORLD); /* IMPORTANT */
end = MPI_Wtime();

MPI_Finalize();

if (rank == 0) { /* use time on master node */
    printf("Runtime = %f\n", end-start);
}

Doing the same on all nodes will give the almost the same results with small deviations depending on how quickly each node returns from the MPI_Barrier call. This is usually a very small value relative to most practical runs and can be discounted.

Trying to derive a time using start/end times from different nodes is not worth the effort, and can give your wrong answers if MPI_Wtime does not use a global synchronised clock. Note that synced Wtime is not supported in some MPI implementations (check MPI_WTIME_IS_GLOBAL).

MPI_Init() and MPI_Finalize() don't mark the beginning and end of parallel execution, only the beginning and end of where MPI calls are allowed. In MPI, all your processes run parallel from beginning to end and share no global data whatsoever.

You can use MPI_Reduce() to find the minimum starting time and maximum ending time of processes.