Hello Salvatore, The current MPICH uses pipe to redirect stdin to processes, but pipe has a limited capacity (e.g., 64KB in Linux. Please refer to http://man7.org/linux/man-pages/man7/pipe.7.html). Therefore, if we redirect a file, which is larger than the pipe capacity, to stdin, the current MPICH implementation is not handling it properly because write() does not succeed to write a large data (> the pipe capacity) to the pipe for stdin. I created a ticket for this issue (https://trac.mpich.org/projects/mpich/ticket/2336), but I'd recommend you to use file I/O in order to handle a large file instead of using stdin redirection. — Sangmin
On Jun 1, 2016, at 4:37 AM, Filippone, Salvatore <[email protected]> wrote:
Hello, I have found something I do not understand with mpich 3.2.0. The problem started with an application, but to demonstrate it I have written a very simple-minded program which I am including here. The program simply reads lines from standard input and reprints them on standard output from a second process.
Now if I redirect stdin from a small(ish) file it works as expected: ====================================================================== [sfilippo@localhost MPI_cat]$ mpicc -v mpicc for MPICH version 3.2 Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/opt/gnu/5.3.0/libexec/gcc/x86_64-unknown-linux- gnu/5.3.0/lto-wrapper Target: x86_64-unknown-linux-gnu Configured with: ../gcc-5.3.0/configure --prefix=/opt/gnu/5.3.0 -- enable-languages=c,c++,fortran --with- gmp=/usr/local/travel/GCC/BUILDS/gmp --with- mpfr=/usr/local/travel/GCC/BUILDS/mpfr --with- mpc=/usr/local/travel/GCC/BUILDS/mpc --with- isl=/usr/local/travel/GCC/BUILDS/isl Thread model: posix gcc version 5.3.0 (GCC) [sfilippo@localhost MPI_cat]$ mpicc -o mpicat mpicat.c [sfilippo@localhost MPI_cat]$ head mpicat.c | mpirun -np 2 ./mpicat #include <stdlib.h> #include <stdio.h> #include "mpi.h" #define LNSZ 129 #define TAGI 234 #define TAGC 567 int main(int argc, char *argv[] ) { char linebuf[LNSZ]; int ierr, iam, np, itmp ; [sfilippo@localhost MPI_cat]$ ====================================================================== However if I use a long(ish) file (such as a configure script) I get either no output or an error. On another machine with GNU 4.9.3 I always get the silent failure. ====================================================================== [sfilippo@localhost MPI_cat]$ mpirun -np 2 ./mpicat < configure [sfilippo@localhost MPI_cat]$ mpirun -np 2 ./mpicat < configure [proxy:0:[email protected]] HYD_pmcd_pmip_control_cmd_cb (../../../../mpich-3.2/src/pm/hydra/pm/pmiserv/pmip_cb.c:937): process reading stdin too slowly; can't keep up [proxy:0:[email protected]] HYDT_dmxu_poll_wait_for_event (../../../../mpich-3.2/src/pm/hydra/tools/demux/demux_poll.c:76): callback returned error status [proxy:0:[email protected]] main (../../../../mpich- 3.2/src/pm/hydra/pm/pmiserv/pmip.c:206): demux engine error waiting for event [sfilippo@localhost MPI_cat]$ =======================================================================
Am I missing something? Is this supposed to work? I could find nothing relevant in the FAQ
Thanks a lot Salvatore -- Salvatore Filippone, PhD Centre for Computational Engineering Sciences, School of Aerospace, Transport and Manufacturing, Bldg 52 Cranfield University Cranfield MK43 0AL United Kingdom tel: +44(0) 1234 758361
<mpicat.c>_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss