Hi, We have encountered a bug with mpiexec when the length of argv is greater than or equal to 1000. This was uncovered in the use of a utility called pbsdcp that uses MPI_Bcast to efficiently copy files to internal disks for parallel jobs. If someone uses a wildcard to pass lists of files for copying on the command line it can result in a large number of arguments being passed to the execution of mpiexec. This results in a seg fault. The problem can be trivially reproduced with the nonsensical command line shown below. owens-rw01:~> mpiexec `seq 999` [proxy:0:[email protected]] HYDU_create_process (utils/launch/launch.c:75): execvp error on file 1 (No such file or directory) owens-rw01:~> mpiexec `seq 1000` Segmentation fault owens-rw01:~> mpiexec --version HYDRA build details: Version: 3.2.1 Release Date: General Availability Release CC: icc CXX: icpc F77: ifort F90: ifort Configure options: '--disable-option-checking' '--prefix=/opt/mvapich2/intel/18.0/2.3' '--enable-shared' '--with-mpe' '--enable-romio' '--enable-mpit-pvars=mv2' '--with-file-system=ufs+nfs+gpfs' '--with-pbs=/opt/torque' '--with-pbs-lib=/opt/torque/lib64' '--with-pbs-include=/opt/torque/include' 'CC=icc' 'CXX=icpc' 'FC=ifort' 'F77=ifort' '--cache-file=/dev/null' '--srcdir=.' 'CFLAGS= -DNDEBUG -DNVALGRIND -O2' 'LDFLAGS=-L/lib -L/lib -L/lib -Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib -L/lib -L/lib' 'LIBS=-libmad -lrdmacm -libumad -libverbs -lrt -lpthread ' 'CPPFLAGS= -I/dev/shm/tmp.PxAZLCzt4P/src/mpl/include -I/dev/shm/tmp.PxAZLCzt4P/src/mpl/include -I/dev/shm/tmp.PxAZLCzt4P/src/openpa/src -I/dev/shm/tmp.PxAZLCzt4P/src/openpa/src -D_REENTRANT -I/dev/shm/tmp.PxAZLCzt4P/src/mpi/romio/include -I/include -I/include -I/include -I/include' 'MPLLIBNAME=mpl' Process Manager: pmi Launchers available: ssh rsh fork slurm ll lsf sge pbs manual persist Topology libraries available: hwloc Resource management kernels available: user slurm ll lsf sge pbs cobalt Checkpointing libraries available: Demux engines available: poll select The root cause seems to be in src/pm/hydra/include/hydra.h, #define HYD_NUM_TMP_STRINGS 1000 This define value is used for static allocation when processing argv in mpiexec. I suppose we could increase this value as a short term workaround that may fix some use cases (could use ARG_MAX, 'getconf ARG_MAX'.) However, a cleaner fix is probably to pass argc along with argv so that dynamic allocations can be used when processing arguments. Thanks, Doug