Hi Zaak, I'll try my best to explain here. There are a few things to consider. 1. Hydra: -disable-auto-cleanup means if an MPI process dies, let other processes in the job continue running. Since Hydra (mpiexec) is already monitoring MPI processes to detect when one dies, there is no impact inside Hydra from passing this option. 2. Application behavior: Since the default error handler in MPI is MPI_ERRORS_ARE_FATAL, some applications may rely on that fact and expect a running job to be aborted/cleaned up if a process quits. With -disable-auto-cleanup this will no longer be the case. An application can call MPI_Abort() to force the old behavior, however. Ken On 08/30/2017 12:29 PM, Zaak Beekman wrote:
OK, since there were no responses here to my previous email, perhaps a better question would be:
What is a good resource to learn about the impact of passing `--disable-auto-cleanup` at runtime?
Some google searches bring up discussions of what appear to be bugs in the standard and/or implementation, but I'm not sure where to look to find out about even the intended runtime semantics.
Any and all help pointing me in the right direction would be much appreciated.
Thanks, Zaak
On Wed, Aug 30, 2017 at 1:00 PM <[email protected] <mailto:[email protected]>> wrote:
Send discuss mailing list submissions to [email protected] <mailto:[email protected]>
To subscribe or unsubscribe via the World Wide Web, visit https://lists.mpich.org/mailman/listinfo/discuss or, via email, send a message with subject or body 'help' to [email protected] <mailto:[email protected]>
You can reach the person managing the list at [email protected] <mailto:[email protected]>
When replying, please edit your Subject line so it is more specific than "Re: Contents of discuss digest..."
Today's Topics:
1. question about -disable-auto-cleanup (Zaak Beekman) 2. Torque MPICH jobs stuck (Souparno Adhikary) 3. Re: Torque MPICH jobs stuck (Halim Amer)
----------------------------------------------------------------------
Message: 1 Date: Tue, 29 Aug 2017 21:22:49 +0000 From: Zaak Beekman <[email protected] <mailto:[email protected]>> To: [email protected] <mailto:[email protected]> Subject: [mpich-discuss] question about -disable-auto-cleanup Message-ID:
<CAAbnBwZrQ03YmmmayhcHEywh8bEFMZ_AycBydOqZFB023KeJZQ@mail.gmail.com <mailto:CAAbnBwZrQ03YmmmayhcHEywh8bEFMZ_AycBydOqZFB023KeJZQ@mail.gmail.com>> Content-Type: text/plain; charset="utf-8"
I know that --disable-auto-cleanup is required to enable the fault-tolerant MPI features, but are there downsides to passing this? Performance implications?
I ask, because over at https://github.com/sourceryinstitute/OpenCoarrays we've implemented much of the Fortran 2015 failed images feature on top of MPICH and other MPI implementations. But to use this, --disable-auto-cleanup must be passed to mpiexec. We provide wrapper scripts to try to abstract the back end (GASNet, MPI, OpenSHMEM etc.) in the form of a Fortran compiler wrapper, and an executable launcher. So I'm wondering, since failed images are part of the standard (2015) would it be dumb if we always pass --disable-auto-cleanup to mpiexec and only turn off support when explicitly asked for by the user, or is it safer/more performant to default to requiring the user to pass an additional flag to our wrapper script that results in --disable-auto-cleanup getting passed to mpiexec?
Feedback would be much appreciated. Feel free to post responses at https://github.com/sourceryinstitute/OpenCoarrays/issues/401 as well..
Thanks, Zaak