Can you try mpiexec env | grep LD_LIBRARY_PATH? As Reuti suggested, this looks like a problem with your .bashrc (or equivalent) setup. -- Pavan On 09/06/2012 01:27 PM US Central Time, Marcelo C. R. Melo wrote:

Following your investigation,

#!/bin/sh echo $LD_LIBRARY_PATH ldd /path/to/your/application

returned the correct value for the environment variable:

/usr/local/gromos/md++/lib/:/usr/local/gromos/lib/:/usr/local/boost/lib/:/usr/local/openmm/lib/:/opt/intel/composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/ipp/../compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/ipp/lib/intel64:/opt/intel/composerxe-2011.3.174/compiler/lib/intel64:/opt/intel/composerxe-2011.3.174/mkl/lib/intel64:/opt/intel/composerxe-2011.3.174/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/opt/intel/composerxe-2011.3.174/mpirt/lib/intel64:/usr/local/mpich2.shared.exec/lib/

and ldd returned:

linux-vdso.so.1 => (0x00007fff0e2fd000) libboost_program_options.so.1.42.0 => /usr/local/boost/lib/libboost_program_options.so.1.42.0 (0x00007f6c29f42000) libboost_filesystem.so.1.42.0 => /usr/local/boost/lib/libboost_filesystem.so.1.42.0 (0x00007f6c29d2c000) libboost_thread.so.1.42.0 => /usr/local/boost/lib/libboost_thread.so.1.42.0 (0x00007f6c29b17000) libmpichcxx.so.3 => /usr/local/mpich2.shared.exec/lib/libmpichcxx.so.3 (0x00007f6c298f5000) libpmpich.so.3 => /usr/local/mpich2.shared.exec/lib/libpmpich.so.3 (0x00007f6c294b8000) libopa.so.1 => /usr/local/mpich2.shared.exec/lib/libopa.so.1 (0x00007f6c292b7000) libmpl.so.1 => /usr/local/mpich2.shared.exec/lib/libmpl.so.1 (0x00007f6c290b2000) librt.so.1 => /lib/librt.so.1 (0x00007f6c28e8f000) libpthread.so.0 => /lib/libpthread.so.0 (0x00007f6c28c72000) libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f6c2895e000) libm.so.6 => /lib/libm.so.6 (0x00007f6c286da000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f6c284c3000) libc.so.6 => /lib/libc.so.6 (0x00007f6c28140000) libboost_system.so.1.42.0 => /usr/local/boost/lib/libboost_system.so.1.42.0 (0x00007f6c27f3b000) libimf.so => /opt/intel/composerxe-2011.3.174/compiler/lib/intel64/libimf.so (0x00007f6c27b56000) libsvml.so => /opt/intel/composerxe-2011.3.174/compiler/lib/intel64/libsvml.so (0x00007f6c274a2000) libintlc.so.5 => /opt/intel/composerxe-2011.3.174/compiler/lib/intel64/libintlc.so.5 (0x00007f6c27352000) libdl.so.2 => /lib/libdl.so.2 (0x00007f6c2714e000) /lib64/ld-linux-x86-64.so.2 (0x00007f6c2a19e000)

which seems correct to me.

As for the

mpiexec echo \$LD_LIBRARY_PATH

it didn't work here. I only got

$LD_LIBRARY_PATH

I tested

mpiexec "echo \$LD_LIBRARY_PATH" >> $log_file 2>&1 , mpiexec echo "\$LD_LIBRARY_PATH" >> $log_file 2>&1

and

mpiexec echo \$LD_LIBRARY_PATH >> $log_file 2>&1

but got the same result every time. I am not sure how the substitution of environment variables works when mpiexec is called but it looks like there is no substitution at all, since I didn't get a blank line, as if the variable was empty or not set, I got its name.

------------------------------

Message: 4 Date: Tue, 4 Sep 2012 22:28:56 +0200 From: Reuti <[email protected] <javascript:;>> To: [email protected] <javascript:;> Subject: Re: [mpich-discuss] hydra_pmi_proxy: error while loading shared libraries: libimf.so: cannot open shared object file: No such file or directory (Reuti) Message-ID: <[email protected] <javascript:;>> Content-Type: text/plain; charset=us-ascii

Am 04.09.2012 um 22:09 schrieb Marcelo C. R. Melo:

> Yes Reuti, the LD_LIBRARY_PATH is configured correctly. > I can use the software normally when logged into any node.

The environment will be different when started by a queuing system, at least for SGE I know this for sure and I think also for Torque/PBS. To investigate, can you please submit a job with:

#!/bin/sh echo $LD_LIBRARY_PATH ldd /path/to/your/application

If we know this, we have to check the processes on the slave nodes of the parallel job. Usually they are not started by an `ssh`, but some builtin mechanism inside the queuingsystem. So, submitting a job with this should show it:

#!/bin/sh mpiexec echo \$LD_LIBRARY_PATH

It's necessary to escape the $, as we want to get the result from the slave, not where the jobscript is executed.

-- Reuti

> The problem is that when I Spawn another executable from my initial launcher program, the spawned executable does not see the environment that the launcher received. > > Is there any portable way to pass the environment variables to Spawned processes? > > I posted this question on stack overflow a few days ago: http://stackoverflow.com/questions/12252490/why-cant-and-environment-variabl... > > Best regards, > Marcelo

_______________________________________________ mpich-discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss

-- Pavan Balaji http://www.mcs.anl.gov/~balaji