I am running a MPI program on two machines. I got errors as follows:


====================================================================
Fatal error in MPI_Test: Other MPI error, error stack:
MPI_Test(153)......................: MPI_Test(request=0xa0a088, flag=0x7fff470e86fc,  status=0x7fff470e86e0) failed
MPIDI_CH3I_Progress(150)...........:
MPID_nem_mpich2_test_recv(800).....:
MPID_nem_tcp_connpoll(1720)........:
state_commrdy_handler(1556)........:
MPID_nem_tcp_recv_handler(1459)....:
MPID_nem_handle_pkt(493)...........:
MPIDI_CH3_PktHandler_EagerSend(589): Failed to allocate memory for an unexpected message. 261892 unexpected messages queued.
Fatal error in MPI_Test: Other MPI error, error stack:
MPI_Test(153)......................: MPI_Test(request=0xadb128, flag=0x7fff33cba448, status=0x7fff33cba430) failed
MPIDI_CH3I_Progress(150)...........:
MPID_nem_mpich2_test_recv(800).....:
MPID_nem_tcp_connpoll(1720)........:
state_commrdy_handler(1556)........:
MPID_nem_tcp_recv_handler(1459)....:
MPID_nem_handle_pkt(493)...........:
MPIDI_CH3_PktHandler_EagerSend(589): Failed to allocate memory for an unexpected message. 261890 unexpected messages queued.
rank 1 in job 11  slave_36134   caused collective abort of all ranks
  exit status of rank 1: killed by signal 9

====================================================================