Hi all,
We just updated margo (origin/master, tag v0.4.3, or spack margo(a)0.4.3) to have it automatically increase the Argobots stack size. I would strongly encourage you to update if you are using Margo or any libraries built atop it.
The underlying problem is that Argobots has a very small default stack size for newly created ULTs. Libraries like Margo (that might make deep call paths into things like Mercury and libfabric) can overflow this value, in which case you might run into some very difficult to diagnose memory corruption problems. We encountered this on two different platforms recently with two very different symptoms.
You can see the exact tunable options we are setting for Argobots here if you are curious. The first is something we've been setting for a while already, the second is new. Both have links to more in depth explanation.
https://xgitlab.cels.anl.gov/sds/margo/blob/master/src/margo.c#L180
We are still working through some options and may tweak this further, but we wanted to go ahead and get a reasonable interim fix out quickly.
thanks,
-Phil
We are very pleased to announce that the release of mercury 1.0.1 is
now available and can be downloaded from:
http://mercury-hpc.github.io/download/
Updated documentation is available at:
http://mercury-hpc.github.io/documentation/
Please submit any issue you may encounter to:
https://github.com/mercury-hpc/mercury/issues
This is mostly a bug fix release, which includes:
* Update CMake policy for CMake 3.12 and above
* Fix potential race when forcing a handle to be destroyed
* Fix cancelation of HG operations (fix #267)
* Fix HG_Reset() to reset NA resources upon NA class change (fix #272)
* NA OFI: fix cancelation of operations that cannot be canceled
* NA OFI: remove extra fi_addr_xxx prefix from generated URI
* NA SM: remove page size check that would prevent to run on system
w pages larger than 4KB (fix #268)
Please see the accompanying changelog for more details:
https://github.com/mercury-hpc/mercury/blob/v1.0.1/CHANGELOG
