Hi Abhishek:


We had a similar issue and was fixed when the following were set as root by our sys admin.

# NOTE: "Unlimited" is not truly unlimited.  It

# will set the given limit to the maximum value

# determined by your hardware configuration


ulimit -t unlimited              # cputime

   ulimit -f unlimited              # filesize

   ulimit -d unlimited              # datasize

   ulimit -s unlimited              # stacksize

   ulimit -c unlimited              # coredumpsize

   ulimit -m unlimited              # memoryuse

   ulimit -v unlimited              # vmemoryuse

   ulimit -n unlimited              # descriptors

   ulimit -l unlimited              # memorylocked

   ulimit -u unlimited              # maxproc


Hope this helps,

Sarika



On Wed, Sep 17, 2014 at 11:04 AM, Balaji, Pavan <balaji@anl.gov> wrote:

On Sep 17, 2014, at 1:00 PM, Abhishek Bhat <abhat@trinityconsultants.com> wrote:
> The application works when there is a less resource intensive runs (larger grid with larger grid spacing).  The issue occurs when we have nested grid runs.  Also the application works without any issues for less than 7 processes (1 I/O and 6 nodes).

That’s not an indication that the application is correct with more processes.  Many applications work at smaller scales but fail at larger scals.  From the error, the indications point to the application.  I’d recommend digging into the application and figuring out what’s breaking.  The -print-all-exitcodes that Sangmin suggested, or attaching a debugger might be useful for this.

  — Pavan

--
Pavan Balaji  ✉️
http://www.mcs.anl.gov/~balaji

_______________________________________________
discuss mailing list     discuss@mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss