On Mon, Mar 2, 2015 at 9:01 PM, Roy, Hirak <[email protected]> wrote:
Hi Wesley,
As I mentioned in my email, MPI_disconnect hangs.
Here is a short program which you can run.
Please note that there is an “assert” in client.c
Compile :
mpicc server.c –o server
mpicc client.c –o client
To run use two shell/terminal :
Term1>> mpiexec –n 1 ./server
Term2>> mpiexec –n 1 ./client
Please press any key on the server terminal.
I earlier filed a bug related to this : http://trac.mpich.org/projects/mpich/ticket/2205
My question :
1> What happens if we don’t call MPI_Finalize and call exit(0)?
Mostly nothing. In theory, some things might not get cleaned up, but probably the worst you'll see is a nasty error message.
2> Is there anyway I can forcefully complete MPI_disconnect from server side ?
I don't know of anything that will allow you to do that.
Thanks,
Hirak
Ps : The reason of using sock is : bug in nemesis : http://trac.mpich.org/projects/mpich/ticket/1103 and http://trac.mpich.org/projects/mpich/ticket/79
*Wesley Bland* wbland at anl.gov <discuss%40mpich.org?Subject=Re%3A%20%5Bmpich-discuss%5D%20MPI_Finalize%20hangs%20in%20dynamic%20connection%20in%0A%09case%20of%20failed%20process&In-Reply-To=%3CF4B1C63E-062B-4036-9DE5-A4C93096F32C%40anl.gov%3E> *Thu Feb 26 10:13:42 CST 2015*
- Previous message: [mpich-discuss] MPI_Finalize hangs in dynamic connection in case of failed process <http://lists.mpich.org/pipermail/discuss/2015-February/003725.html> - Next message: [mpich-discuss] query ABI version <http://lists.mpich.org/pipermail/discuss/2015-February/003726.html> - *Messages sorted by:* [ date ] <http://lists.mpich.org/pipermail/discuss/2015-February/date.html#3740> [ thread ] <http://lists.mpich.org/pipermail/discuss/2015-February/thread.html#3740> [ subject ] <http://lists.mpich.org/pipermail/discuss/2015-February/subject.html#3740> [ author ] <http://lists.mpich.org/pipermail/discuss/2015-February/author.html#3740>
------------------------------
First, I believe the sock device is untested with most of the MPICH fault tolerance features, so YMMV here.
Is there a reason that you aren’t calling MPI_Disconnect for the failed process? Did you try it an something bad happened? That seems like the most straightforward way of doing things.
Otherwise, this sounds like a known issue that we’re seeing from time to time with MPI_Finalize and the FT work. It’s something I’m trying to figure out now. If you can reduce your code down to the minimum and send it to me, I can use it as a test case to try to fix the problem.
Thanks,
Wesley
* On Feb 19, 2015, at 5:15 AM, Roy, Hirak <Hirak_Roy at mentor.com <https://lists.mpich.org/mailman/listinfo/discuss>> wrote:*
* Hi All,*
* I am using MPICH with sock connection.*
* I also setup processes using dynamic connection method (MPI_Comm_connect/MPI_Comm_accept). It’s a master-slave architecture where master accepts the connections from slaves.*
* Now if one of the process dies (or get killed), I can still recover from this (without using checkpoint/restore method).*
* For the particular process in master, I do not call MPI_disconnect (it hangs and does not complete).*
* As a result, my MPI_Finalize in master hangs and does not complete.*
* Do you have a workaround to forcefully complete MPI_Finalize or MPI_disconnect?*
* I tried MPI_Comm_free on the failed connection. However, it does not solve the hang in finalize.*
* Thanks,*
* Hirak*
* _______________________________________________*
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss