- discuss - mailman3.cels.anl.gov

Argonne Spam Quarantine Notification
by Argonne Anti Spam Quarantine 17 Dec '25

17 Dec '25

1 0

Argonne Spam Quarantine Notification
by Argonne Anti Spam Quarantine 23 Sep '23

23 Sep '23

1 0

constraining overall stack memory allocation
by Phil Carns 20 Jun '22

20 Jun '22

Hi all, I was rummaging around in the code looking for ideas just now and figured I might save myself some time by asking on the list to see if anyone else has encountered this. A quick review of the use case: we are using large stack sizes (2 MiB right now, though we could probably go lower but it will still be much larger than the ABT default). We also create, execute, and complete a large number of detached ULTs. Only a very few are intentionally long lived. Our current strategy is that a central producer (who drives network progress) creates ULTs that may be placed on other pools/ESs depending on configuration. I had *thought* that the ULT stacks were not allocated until the ULT was selected for execution by a scheduler, but I see now that's not the case. The stack is allocated up front at ABT_thread_create() time. I'm kicking myself for not understanding that sooner. It didn't matter so much when we used to use small stack sizes. At any rate, at this point this strategy has a few implications. If the ES schedulers don't retire old ULTs fast enough (even if they are very "close" to completion) then we can balloon memory consumption even if it doesn't look like our actual concurrency is all that high, simply because we are greedily taking more memory for stacks without regard to ULT completion. Secondly, the one producer is always paying the allocation cost, and the memory is always local to that one core. What would be ideal for me would be if ABT_thread_create() would defer stack allocation somehow. Ideally not consuming so much memory for a thread until a) it can really be executed and b) the scheduler thinks it is a good idea to do so. Even better if the the allocation were in the context of the ES that popped the thread, rather than the ES that spawned the thread. Is this possible? It would be neat if this could be done internal to Argobots somehow for generality for my use case, but walking through the code I have the sinking feeling that we need to do this above Argobots (explicitly queueing up work and letting the "worker" execution streams create their own ULTs to perform that work a needed, rather than letting the ULT pools within Argobots serve double duty as our work queue). I'm comfortable with custom pools and schedulers, but it looks like the key step is already out of our hands at ULT creation time so there isn't much a custom pool or scheduler could do. Thanks for hearing me out, and thanks in advance for feedback (even if it takes the form of "that's a silly idea" :) ). thanks, -Phil

2 3

Re: [argobots-discuss] How to yield a running Xstream?
by Raghav Gupta 17 Feb '22

17 Feb '22

Thanks Shintaro for your reply, I have a follow-up question. I have an ABT_thread, which is a long-running thread, and I cannot modify the thread's function, i.e. it is not possible to insert the instruction "ABT_self_yield()" inside the thread's function. In this scenario, is it possible to signal the thread to yield to the xstream? Best, Raghav

2 1

How to yield a running Xstream?
by Raghav Gupta 07 Feb '22

07 Feb '22

Hi All, I have a scenario where two *Xstreams*, *xstream-0* and *xstream-1*, are bound to *core-0* and *core-1*. And, I have an *ABT_thread*, which is a long-running thread executing on *xstream-0*. I want to know if we could do the following with Argobots APIs 1. Is it possible to yield the execution of *xstream-0*, which is executing the long-running *ABT_thread*? 2. If the above is true, is it also possible to transfer the partial execution of that *ABT_thread* to *xstream-1*? Best, Raghav

2 1

Re: [argobots-discuss] externally allocated stacks and unnamed (detached) threads
by Faccini, Bruno 18 Nov '21

18 Nov '21

Phil, Shintaro, Just in case, we have implemented it in DAOS, and it works just fine ! You can find the whole code for the feature at https://github.com/daos-stack/daos/pull/5111 . Cheers, Bruno. From: Shintaro Iwasaki via discuss <discuss(a)lists.argobots.org<mailto:[email protected]>> Reply-To: "discuss(a)lists.argobots.org<mailto:[email protected]>" <discuss(a)lists.argobots.org<mailto:[email protected]>> Date: Thursday 18 November 2021 at 02:29 To: "discuss(a)lists.argobots.org<mailto:[email protected]>" <discuss(a)lists.argobots.org<mailto:[email protected]>> Cc: Shintaro Iwasaki <shintaro.iwasaki.work(a)gmail.com<mailto:[email protected]>> Subject: Re: [argobots-discuss] externally allocated stacks and unnamed (detached) threads Hi Phil, Thanks for your question! Calling a custom stack destructor for an unnamed ULT is possible within the current Argobots infrastructure. This is related to this issue: https://github.com/pmodels/argobots/issues/16. For the detailed implementation, please find the complete code here (collapsed): https://github.com/pmodels/argobots/issues/274#issuecomment-785221242. Though you have to call init_stack_key() and finalize_stack_key() to use this, it should be simple enough: you just basically change allocate_stack() and free_stack() for your workload. I believe this is the quick path to implement your idea. But maybe a more intuitive stack-free-hook interface would be preferable (this request is not the first time). Please let me know if you need such a new interface (especially if your idea works well). Thanks, Shintaro On Wed, Nov 17, 2021 at 2:15 PM Phil Carns via discuss <discuss(a)lists.argobots.org<mailto:[email protected]>> wrote: Hi all, We've been thinking about some memory optimizations for which it might be easier if our own code provided the stack memory for newly created ULTs so that we have full control over the allocation strategy. This seems pretty straightforward with the ABT_thread_attr_set_stack() function: https://www.argobots.org/doxygen/latest/d2/df6/group__ULT__ATTR.html#gac8f3… However, our use case often creates unnamed (detached) threads by setting the ABT_thread_create() newthread argument to NULL. I can elaborate if needed, but the short story is that this is a really natural fit for RPC handlers in our model: https://www.argobots.org/doxygen/latest/d0/d6d/group__ULT.html#ga73bd1f04ce… Is it possible to do both (provide an externally managed stack pointer, and use unnamed ULTs)? From the documentation it looks like the answer is probably no, because the caller who created the stack allocation doesn't explicitly free the ULT, and thus doesn't have a way to free the stack after the ULT is complete. I wanted to double check though before I start thinking about alternatives, though. thanks! -Phil _______________________________________________ discuss mailing list discuss(a)lists.argobots.org<mailto:[email protected]> https://lists.argobots.org/mailman/listinfo/discuss --------------------------------------------------------------------- Intel Corporation SAS (French simplified joint stock company) Registered headquarters: "Les Montalets"- 2, rue de Paris, 92196 Meudon Cedex, France Registration Number: 302 456 199 R.C.S. NANTERRE Capital: 4,572,000 Euros This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

2 1

externally allocated stacks and unnamed (detached) threads
by Phil Carns 17 Nov '21

17 Nov '21

Hi all, We've been thinking about some memory optimizations for which it might be easier if our own code provided the stack memory for newly created ULTs so that we have full control over the allocation strategy. This seems pretty straightforward with the ABT_thread_attr_set_stack() function: https://www.argobots.org/doxygen/latest/d2/df6/group__ULT__ATTR.html#gac8f3… However, our use case often creates unnamed (detached) threads by setting the ABT_thread_create() newthread argument to NULL. I can elaborate if needed, but the short story is that this is a really natural fit for RPC handlers in our model: https://www.argobots.org/doxygen/latest/d0/d6d/group__ULT.html#ga73bd1f04ce… Is it possible to do both (provide an externally managed stack pointer, and use unnamed ULTs)? From the documentation it looks like the answer is probably no, because the caller who created the stack allocation doesn't explicitly free the ULT, and thus doesn't have a way to free the stack after the ULT is complete. I wanted to double check though before I start thinking about alternatives, though. thanks! -Phil

2 1

Writing recursive task parallel program with argobots
by Raghav Gupta 07 Oct '21

07 Oct '21

Hi, I am trying to use argobots for writing a recursive task parallel program. I have taken a very simple recursive task program example, where I am creating a parent argobot task which creates two child argobots tasks and then waits for them by calling ABT_task_free(). When I run this program with one xstream I find that it doesn't terminate. *(Code snippet - * https://gist.github.com/raghav0307/9252fc7b568bc35baab29bdc9da70618* )* My understanding is that this is because the parent task never yields the xstream to the child tasks, and hence the program gets stuck. This program terminates when I use argobot threads instead of argobot tasks. Could someone point me to why this is not working with argobot tasks? Best, Raghav

2 2

condition variable and rwlock
by Dorier, Matthieu 10 Aug '21

10 Aug '21

Hi, I suspect the answer is no, but is there a way of using an Argobots condition variable with an rwlock instead of a mutex? (my use-case is a structure that receives many read requests and a few writes, clearly justifying using a rwlock instead of a mutex, but I may want some of the reads to block until a write has happened, which means I need a condition variable). Thanks, Matthieu

3 7

Assertion failing in ABT_Finalize
by Dorier, Matthieu 21 Jul '21

21 Jul '21

Hi, In a program we are hitting this assertion: https://github.com/pmodels/argobots/blob/main/src/global.c#L346 This happens when we try to finalize Argobots. Any idea what could be causing it? Thanks, Matthieu

2 2