Hi all, I am running a MPI program on two machines. I got errors as follows: ==================================================================== Fatal error in MPI_Test: Other MPI error, error stack: MPI_Test(153)......................: MPI_Test(request=0xa0a088, flag=0x7fff470e86fc, status=0x7fff470e86e0) failed MPIDI_CH3I_Progress(150)...........: MPID_nem_mpich2_test_recv(800).....: MPID_nem_tcp_connpoll(1720)........: state_commrdy_handler(1556)........: MPID_nem_tcp_recv_handler(1459)....: MPID_nem_handle_pkt(493)...........: MPIDI_CH3_PktHandler_EagerSend(589): Failed to allocate memory for an unexpected message. 261892 unexpected messages queued. Fatal error in MPI_Test: Other MPI error, error stack: MPI_Test(153)......................: MPI_Test(request=0xadb128, flag=0x7fff33cba448, status=0x7fff33cba430) failed MPIDI_CH3I_Progress(150)...........: MPID_nem_mpich2_test_recv(800).....: MPID_nem_tcp_connpoll(1720)........: state_commrdy_handler(1556)........: MPID_nem_tcp_recv_handler(1459)....: MPID_nem_handle_pkt(493)...........: MPIDI_CH3_PktHandler_EagerSend(589): Failed to allocate memory for an unexpected message. 261890 unexpected messages queued. rank 1 in job 11 slave_36134 caused collective abort of all ranks exit status of rank 1: killed by signal 9 ==================================================================== I have never seen such errors before. What is the cause of this error? Is it an out of memory error? ( There is 20% remaining memory on machines ) Any help would be greatly appreciated. Thanks in advance! -- Best Regards. --- Xing FENG PhD Candidate Database Research Group School of Computer Science and Engineering University of New South Wales NSW 2052, Sydney Phone: (+61) 413 857 288 _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss