February 2020 - mochi-devel - mailman3.cels.anl.gov

margo_bulk_transfer() not working on sm transport
by Pradeep Subedi 10 Feb '20

10 Feb '20

Hi, I am currently working on developing a mochi service provider. The provider I developed works on tcp and sockets transport layer, but when I use sm transport, I get errors. On the client side the handle is created as: hret = margo_bulk_create(provider->client->mid, 1, (void**)(&data), &rdma_size, HG_BULK_WRITE_ONLY, &in.handle); On the server side the handle is created as hret = margo_bulk_create(mid, 1, (void**)&buffer, &size, HG_BULK_READ_ONLY, &bulk_handle); The server fails on : hret = margo_bulk_transfer(mid, HG_BULK_PUSH, info->addr, in.handle, 0, bulk_handle, 0, size); with following errors: # NA -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/na/na_sm.c:3741 # na_sm_put(): process_vm_writev() failed (Operation not permitted) # HG -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/mercury_bulk.c:829 # hg_bulk_transfer_pieces(): Could not transfer data (NA_PROTOCOL_ERROR) # HG -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/mercury_bulk.c:988 # hg_bulk_transfer(): Could not transfer data pieces # HG -- Error -- /tmp/pradsubedi/spack-stage/spack-stage-mercury-master-6hruzald7l6bjf67o6nptv5inrqk2h3z/spack-src/src/mercury_bulk.c:1829 # HG_Bulk_transfer_id(): Could not start transfer of bulk data Is this a known issue with shared memory transport for margo_bulk_transfer or should I be doing something different? Thanks, Pradeep Subedi

2 1

Re: [Mochi-devel] Serializing raw buffers in mercury
by Dorier, Matthieu 06 Feb '20

06 Feb '20

Hi Philip, Here is an example of how it's done in one of our services: https://xgitlab.cels.anl.gov/sds/sds-keyval/blob/master/src/sdskv-rpc-types… Thanks, Matthieu On 06/02/2020, 10:12, "mochi-devel on behalf of Philip Davis" <mochi-devel-bounces(a)lists.mcs.anl.gov on behalf of philip.e.davis(a)rutgers.edu> wrote: Hello, I have a need to serialize a simple raw buffer (void * and length.) I’m looking at the “serializing complex data structures” page on the Margo documentation, and I want to make sure I’m doing this right. Should I be writing an hg_proc_* function that serializes the hg_size_t and uses hg_proc_raw to copy the buffer, or is there some existing hg_proc_* function I can leverage to encode/decode the buffer/length pair in a single call? It doesn’t look to me like hg_proc_raw can do this, since the length is copied by value. Thanks, Philip _______________________________________________ mochi-devel mailing list mochi-devel(a)lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/mochi-devel https://www.mcs.anl.gov/research/projects/mochi

2 3

Serializing raw buffers in mercury
by Philip Davis 06 Feb '20

06 Feb '20

Hello, I have a need to serialize a simple raw buffer (void * and length.) I’m looking at the “serializing complex data structures” page on the Margo documentation, and I want to make sure I’m doing this right. Should I be writing an hg_proc_* function that serializes the hg_size_t and uses hg_proc_raw to copy the buffer, or is there some existing hg_proc_* function I can leverage to encode/decode the buffer/length pair in a single call? It doesn’t look to me like hg_proc_raw can do this, since the length is copied by value. Thanks, Philip

1 0