On 4/18/2013 7:04 PM, Pavan Balaji wrote:
Bharath,
Can you send the output of "mpiexec -info"? mpiexec -info HYDRA build details: Version: 3.0.2 Release Date: Wed Feb 27 19:12:19 EST 2013 CC: gcc -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 CXX: g++ -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 F77: gfortran -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 F90: gfortran Configure options: '--disable-option-checking' '--prefix=/opt/apps/gcc4_5/mvapich2/1.9b' '--with-ib-libpath=/opt/ofed/ofed/lib64/' '--with-ib-include=/opt/ofed/ofed/include/' '--with-pbs=/opt/torque/torque' '--enable-fast=O3' '--enable-shared' 'CC=gcc' 'CFLAGS= -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 -O3' 'CXX=g++' 'CXXFLAGS= -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 -O3' 'F77=gfortran' 'FFLAGS=-L/opt/ofed/ofed/lib64/ -L/lib -L/lib -Wl,-rpath,/opt/apps/gcc/4.5.3/lib64 -O3' 'FC=gfortran' '--enable-embedded-mode' '--cache-file=/dev/null' '--srcdir=.' 'LDFLAGS=-L/opt/ofed/ofed/lib64/ -L/lib -L/lib -L/lib -Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib -L/lib -L/lib' 'LIBS=-libmad -lrdmacm -libumad -libverbs -ldl -lrt -lm -lpthread ' 'CPPFLAGS=-I/opt/ofed/ofed/include/ -I/opt/build/BUILD/mvapich2-1.9b/src/mpl/include -I/opt/build/BUILD/mvapich2-1.9b/src/mpl/include -I/opt/build/BUILD/mvapich2-1.9b/src/openpa/src -I/opt/build/BUILD/mvapich2-1.9b/src/openpa/src -I/opt/build/BUILD/mvapich2-1.9b/src/mpi/romio/include -I/include -I/include -I/include -I/include' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge pbs manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs Checkpointing libraries available: Demux engines available: poll select
I'm particularly interested in whether mpiexec recognizes pbs as a possible launcher or not. If it recognizes pbs as a launcher, it should automatically select it where possible.
It is using pbs as the launcher. The problem is with the setting of the session id for the forked hydra_pmi_proxy process on mother superior. Since the hydra_pmi_proxy process has the session id set to the bash process instead mpi_exec process. As a result of this torque is unable to track the resources consumed by the process. If you look at the openmpi processes their session id is set correctly and torque is able to track the resources consumed. -- Bharath