Folks: I’m interested in running some fault tolerance experiments using the ULFM extensions in MPICH, but I’m stumbling right out of the gate. When I run the following Python code, rank 0 broadcasts a monotonically increasing value to the other ranks, and it gets printed to stdout: import itertools import time from mpi4py import MPI comm = MPI.COMM_WORLD for count in itertools.count(): value = comm.bcast(count, root=0) print(f"{comm.rank}: {value}", flush=True) time.sleep(1) So far, so good. If I run the code using mpiexec -n 3 --disable-auto-cleanup python test.py I get the expected output. If I kill one of the three processes, the others keep running for a few more iterations thanks to --disable-auto-cleanup, until bcast() blocks. My assumption was that MPICH would eventually return an error code, rather than blocking. Otherwise, it doesn’t seem like my code will ever have the chance to use revoke() and shrink(). What am I missing here? Is there a way to specify timeouts for blocking operations? Am I limited to using async operations with ULFM? Could you refer me to any ULFM tests or examples that you used in development? Thanks in advance, Tim Timothy M. Shead Sandia National Laboratories [email protected]<mailto:[email protected]>