Antonio, Thanks for the info. I'll have a try. All our IB cards are from Mellanox, both DDR and QDR. The only thing is installing the Mellanox OFED stack to get mxm library and modules Regards On 2015-01-15 17:21, Antonio J. Peña wrote:

Hi Ramiro,

Our folks that contributed that netmod are looking into this issue. In the meantime, I'd suggest trying the MXM netmod. MXM is a netmod for IB networks using the MXM API instead of Verbs to interact with the HCA.

Best,   Antonio

On 01/15/2015 05:52 AM, Ramiro Alba wrote:

...

Hi all,

I've compiled mpich-3.1.3 on centos 6.5 with the following options:

        --enable-fortran=yes         --with-device=ch3:nemesis:ib         --with-pm=hydra:gforker

and the package 'libibverbs-devel' installed.

When I try to run a test hello program using two IB DDR  nodes, using the command:

mpiexec.hydra -np 16 -bind-to core -launcher rsh -iface ib0 -hosts jff201,jff202 mpi_hello

I've got the errors bellow, even running using with root user.

If I compile with:

--with-device=ch3:nemesis

it works with no errors.

I am also using both openmpi and mvapich2 on Infiniband and they work fine

Am I doing something wrong when compiling and/or running? Any sugestion is welcomed?

Thanks in advance Regards

##########################################################################

...

MPICH OVER IB: RUN ERRORS

##########################################################################

...

Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(498): MPID_Init(177).......: channel initialization failed MPIDI_CH3_Init(89)...: MPID_nem_init(320)...: MPID_nem_ib_init(264): MPID_nem_ib_com_open failed Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(498): MPID_Init(177).......: channel initialization failed MPIDI_CH3_Init(89)...: MPID_nem_init(320)...: MPID_nem_ib_init(264): MPID_nem_ib_com_open failed [root@jff201 mpich]# mpirun -np 2 -iface eth0 mpi_hello-mpich IB device not foundFatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(498): MPID_Init(177).......: channel initialization failed MPIDI_CH3_Init(89)...: MPID_nem_init(320)...: MPID_nem_ib_init(264): MPID_nem_ib_com_open failed IB device not foundFatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(498): MPID_Init(177).......: channel initialization failed MPIDI_CH3_Init(89)...: MPID_nem_init(320)...: MPID_nem_ib_init(264): MPID_nem_ib_com_open failed

##########################################################################

...

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss [1]

-- Antonio J. Peña Postdoctoral Appointee Mathematics and Computer Science Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 240, Of. 3148 Argonne, IL 60439-4847 [email protected] www.mcs.anl.gov/~apenya [2]

-- Aquest missatge ha estat analitzat per MAILSCANNER [3] a la cerca de virus i d'altres continguts perillosos, i es considera que está net.

Links: ------ [1] https://lists.mpich.org/mailman/listinfo/discuss [2] http://www.mcs.anl.gov/~apenya [3] http://www.mailscanner.info/

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

-- Ramiro Alba Centre Tecnològic de Tranferència de Calor http://www.cttc.upc.edu Escola Tècnica Superior d'Enginyeries Industrial i Aeronàutica de Terrassa Colom 11, E-08222, Terrassa, Barcelona, Spain Tel: (+34) 93 739 8928 -- Aquest missatge ha estat analitzat per MailScanner a la cerca de virus i d'altres continguts perillosos, i es considera que est� net. _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss