Dear MPICH developers, We are experiencing some problems getting MPICH jobs to run under SLURM (Debian package slurm-llnl 2.3.4-2+b1) on our small test cluster. When starting an MPI job with more than one rank, the program crashes immediately with the following output: ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- *** glibc detected *** ./hello: double free or corruption (fasttop): 0x00000000014c9680 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x76d76)[0x7f4ee51fed76] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7f4ee5203aac] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPIDI_Populate_vc_node_ids+0x3f9)[0x7f4ee5dec5d9] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPID_Init+0x136)[0x7f4ee5de6dd6] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPIR_Init_thread+0x23f)[0x7f4ee5e9cf7f] /opt/mpich/3.0.4-gcc/lib/libmpich.so.10(MPI_Init+0xae)[0x7f4ee5e9c90e] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7fa889cddead] ./hello[0x400799] ----- >8 ----- >8 ----- >8 ----- >8 ----- >8 ----- >8 ----- Single rank jobs run fine, but they are obviously of little interest ;-) The test program is a simple 'hello world' printing the rank, started using a minimal batch script: ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- 8< ----- #!/bin/sh #SBATCH -n 4 mpiexec ./hello ----- >8 ----- >8 ----- >8 ----- >8 ----- >8 ----- >8 ----- We have tried with MPICH 3.0.4 as well as MPICH2 1.5, configured using only --prefix=... --enable-shared --enable-debuginfo Both are showing the same symptoms. MPICH2 1.4.1p1, however, works without problems. Any idea what's going wrong in the newer versions? Thanks, Markus -- Dr. Markus Geimer Juelich Supercomputing Centre Institute for Advanced Simulation Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49-2461-61-1773 Fax: +49-2461-61-6656 E-mail: [email protected] WWW: http://www.fz-juelich.de/jsc/ ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------