Have you tried MPICH 3.0.4? Hydra has been improved a great deal since the 2.4 release. Jeff Sent from my iPhone On Jun 1, 2013, at 9:21 AM, Markus Geimer <[email protected]> wrote:
Dear MPICH developers,
We are experiencing some problems getting MPICH jobs to run under SLURM (Debian package slurm-llnl 2.3.4-2+b1) on our small test cluster. When starting an MPI job with more than one rank, the program crashes immediately with the following output:
----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< -----
*** glibc detected *** ./hello: double free or corruption (fasttop): 0x00000000014c9680 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x76d76)[0x7f4ee51fed76] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7f4ee5203aac] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPIDI_Populate_vc_node_ids+0x3f9)[0x7f4ee5dec5d9] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPID_Init+0x136)[0x7f4ee5de6dd6] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPIR_Init_thread+0x23f)[0x7f4ee5e9cf7f] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPI_Init+0xae)[0x7f4ee5e9c90e] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7fa889cddead] ./hello[0x400799]
----- >8 ----- >8 ----- >8 ----- >8 ----- >8 ----- >8 -----
Single rank jobs run fine, but they are obviously of little interest ;-) The test program is a simple 'hello world' printing the rank, started using a minimal batch script:
----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< -----
#!/bin/sh #SBATCH -n 4 mpiexec ./hello
----- >8 ----- >8 ----- >8 ----- >8 ----- >8 ----- >8 -----
We have tried with MPICH 3.0.4 as well as MPICH2 1.5, configured using only
--prefix=... --enable-shared --enable-debuginfo
Both are showing the same symptoms. MPICH2 1.4.1p1, however, works without problems. Any idea what's going wrong in the newer versions?
Thanks, Markus
-- Dr. Markus Geimer Juelich Supercomputing Centre Institute for Advanced Simulation Forschungszentrum Juelich GmbH 52425 Juelich, Germany
Phone: +49-2461-61-1773 Fax: +49-2461-61-6656 E-mail: [email protected] WWW: http://www.fz-juelich.de/jsc/
------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ _______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss