The key term here is "larg(ish)", which I used in my original email to mean something larger than the limit I was hitting (64KB) but not *really* large (in my original case I was using a file of around 300KB). I naturally moved to normal file I/O; MPI I/O was not quite called for, because these were just the control parameters for a series of benchmark runs, not actual input data (which was read and handled separately). Thanks Salvatore On Wed, 2016-06-08 at 12:46 +0000, Balaji, Pavan wrote:

If you want to read large files, you should really use MPI-I/O, not stdin. We support reading from stdin for users who migrate from sequential programs to parallel programming, but that's not meant to be the real mechanism to read files. I'm not supportive of trying to modify mpich for such use cases.

-- Pavan

...

On Jun 5, 2016, at 11:18 AM, Seo, Sangmin <sseo at anl.gov> wrote:

Hello Salvatore,

The current MPICH uses pipe to redirect stdin to processes, but pipe has a limited capacity (e.g., 64KB in Linux. Please refer to ​ http://man7.org/linux/man-pages/man7/pipe.7.html). Therefore, if we redirect a file, which is larger than the pipe capacity, to stdin, the current MPICH implementation is not handling it properly because write() does not succeed to write a large data (> the pipe capacity) to the pipe for stdin.

I created a ticket for this issue (https://trac.mpich.org/projects/mpich/ticket/2336), but I'd recommend you to use file I/O in order to handle a large file instead of using stdin redirection.

— Sangmin

_______________________________________________ discuss mailing list [email protected] To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss