December 2021 - mochi-devel - mailman3.cels.anl.gov

new SSG version 0.5.0 out -- important API changes
by Snyder, Shane 23 Dec '21

23 Dec '21

Hi all, Christmas comes early this year and we have a new version of SSG out for Mochi users! This new release contains some key changes to SSG's API and semantics that users should consider. That said, we are hopeful this newly refactored version of SSG will address some usability/stability concerns that users have brought up in the past. Important changes: * All SSG functions return error codes now, which means that the return values from prior versions of SSG functions have been shifted into functional arguments that are passed by reference * ssg_strerror() function can be used to obtain an error string * ssg_group_observe()/ssg_group_unobserve() have been removed and replaced with ssg_group_refresh() * this function may be repeatedly called by non-group members to obtain updated group views from group members * group members do not need to use this call, as their views are automatically managed by the SWIM protocol * ssg_init() can no longer be called before margo_init(), a capability we originally added to allow users to extract key group info before initializing Margo. New helper functions have been added to support these use cases: * ssg_get_group_cred_from_buf()/ssg_get_group_cred_from_file() to retrieve a group credential from a group file or buffer, before initializing SSG * ssg_get_group_transport_from_buf()/ssg_get_group_transport_from_file() to retrieve the group transport protocol from a group file or buffer, before initializing SSG * ssg_group_id_get_cred() and ssg_group_id_get_addr_str() functions have been removed * The former is replaced by the credential helper function mentioned above * The latter is replaced by new function ssg_get_group_member_addr_str() * ssg_group_join_target(), ssg_group_leave_target(), and ssg_group_refresh_target() all expect input in form of a Mercury address now, rather than an address string Please let us know if you have any questions/feedback about the changes or if you run into any problems! Happy holidays! --Shane

1 0

Yokan 0.2 is out!
by Dorier, Matthieu 08 Dec '21

08 Dec '21

Dear all, We just released Yokan 0.2. This release brings some major new features, including * A complete document-store interface * Custom, user-provided filters loaded from dynamic libraries * New modes, in particular one mode that makes RPC avoid relying on RDMA to transfer data The documentation is up to date: https://mochi.readthedocs.io/en/latest/yokan.html The document store interface provides a way of storing values (or “documents”) without having to provide a key. The key is a unique, 64-bit identifier (yk_id_t) returned by the database. Document storage filters include the possibility to parse the document’s content as JSON data within a Lua script. Matthieu

1 0

Memory Registration time
by Philip Davis 03 Dec '21

03 Dec '21

Hello, Working on Frontera, I am seeing some fairly long times for memory registration. It takes 3.2 ms pretty consistently to do a margo_bulk_create for a 8MiB buffer. In light of recent discoveries regarding huge pages, I disabled THP on Frontera with prctl and saw registration time increase to 3.8ms. Anecdotally, pinning memory is a fairly expensive operation but I’m not sure what to expect. The same call on Summit takes about 500-600 usec. When multiple bulk handle creates are in contention, I see more variance, including some registrations that only take about 1.5 milliseconds, so I assume there’s some amount of caching somewhere. However, I’m running with FI_MR_CACHE_MAX_COUNT=0, since that’s the only way I can make rxm;verbs work for most (all?) recent version of libfabric. This leads me to a couple questions. First, is there any bulk handle caching inside Margo? Second, does margo_bulk_create block the calling execution stream in Margo, or are they being completed asynchronously while the handler yields? If the former, is there a different call I can use to do the latter? Thanks, Philip

2 2