discuss

Download

[email protected]

April 2019

5 participants
2 discussions

Re: [argobots-discuss] Argobots ABT_eventual_set too slow
by Iwasaki, Shintaro 25 Apr '19

25 Apr '19

Hi, Houjun, Please use ABT_xstream_set_cpubind or ABT_xstream_set_affinity. You can also disable the xstream affinity by setting ABT_SET_AFFINITY=no. Thanks, Shintaro On Thu, Apr 25, 2019, 18:08 Shintaro Iwasaki <wleaf.iwashi(a)gmail.com<mailto:[email protected]>> wrote: Hi, Houjun, Please use ABT_xstream_set_cpubind or ABT_xstream_set_affinity. You can also disable the xstream affinity by setting ABT_SET_AFFINITY=no. Thanks, Shintaro On Thu, Apr 25, 2019 at 5:27 PM Houjun Tang via discuss <discuss(a)lists.argobots.org<mailto:[email protected]>> wrote: Hi Pavan, Yes, the argobots and execution are under my control, but I'm having issues to use pthread_setaffinity_np(), I added some code from the example in http://man7.org/linux/man-pages/man3/pthread_getaffinity_np.3.html, but it seems it is only supported by the GNU compilers as I have to add "#define _GNU_SOURCE" to the beginning of the code. But the application is usually compile with Intel compilers on NERSC machines, so it won't work in that case. I'm not very familiar with the thread binding related things, is there any other ways to do it? Thanks, Houjun Tang On Thu, Apr 25, 2019 at 1:24 PM Balaji, Pavan <balaji(a)anl.gov<mailto:[email protected]>> wrote: Hi Houjun, > On Apr 25, 2019, at 3:19 PM, Houjun Tang <htang4(a)lbl.gov<mailto:[email protected]>> wrote: > I have tried different things as you suggested, and it seems setting OMP_PROC_BIND=false (previously it is set to "spread") can solve the problem. With that variable set, ABT_eventual_set() takes similar time (under 0.00001 seconds) to when OpenMP is disabled. So maybe it's because the Argobots threads and OpenMP threads are executing on the same core and causes the slowdown? That's a good find. Yes, it looks like the OpenMP runtime is forcing the pthread to a specific core, so any child threads created by that thread would also be bound to the same core. Presumably the argobots thread and execution stream are under your control. Can you unset the binding in your execution stream when you create it? You can use pthread_setaffinity_np, although that needs to be executed inside your ULT. -- Pavan _______________________________________________ discuss mailing list discuss(a)lists.argobots.org<mailto:[email protected]> https://lists.argobots.org/mailman/listinfo/discuss

1 0

Argobots ABT_eventual_set too slow
by Houjun Tang 25 Apr '19

25 Apr '19

Hi, I'm using Argobots as the engine for executing asynchronous I/O operations in the background of an HDF5 application, but found it to be slow in some operations. With profiling, the slowdown comes mostly from *ABT_eventual_set*. Below is a boxplot of the ABT_eventual_set time (measured by calling gettimeofday before and after it) from 385 operations, running with one process and one Argobots thread. The *_fn are different functions executed by Argobots. In most cases it's below 0.1s, but there are several cases that are taking more than 0.25 seconds. As these HDF5 operations take less than 0.1 seconds, the overhead of ABT_eventual_set becomes dominant. Any idea what could have caused this? Thanks, Houjun Tang [image: image.png]

5 20