Re: [ASG] FW: [EXTERNAL] Re: 20140804 - Sandia Status
It uses some boost headers (for preprocessor macros to help define the encoders), but it doesn't actually use any boost libraries at link time. The only proper external libraries that it uses are for pthreads and whatever transport it is using. Technically Mercury can be built without Boost at all (it is a configure option), but IIRC declaring the encoder functions for data types that will be sent over the wire can be tedious without them. We've never tried using it in that mode. thanks, -Phil On 08/07/2014 02:19 PM, Curry, Matthew Leon wrote:
Ah, sorry I got that wrong. It does rely on some Boost libraries, though, right?
On 8/7/14, 12:17 PM, "Phil Carns" <[email protected]> wrote:
Thanks for the clarification. It's definitely true that putting Mercury in the Linux kernel (item 1) isn't on the Mercury roadmap at all. FWIW it's written in C rather than C++, but regardless it was not created with that use case in mind.
thanks, -Phil
On 08/07/2014 01:56 PM, Ross, Robert B. wrote:
Thanks for the explanation Matt. ― Rob
On Aug 7, 2014, at 12:52 PM, Curry, Matthew Leon <[email protected]> wrote:
Hello all,
Geoff and I spoke, and he reminded me of the talking points he and Lee had. Here are the basic ideas. Please correct me if I’m mistaken on any Mercury functionality. The current preferred strategy is to strip down SSI and use it with rpcgen’s serialization/deserialization stuff.
1) We expect the Sirocco client to live in the Linux kernel, and SSI was written to be relatively easy to port there. Mercury is written entirely in C++, and would require significant porting effort for client stuff to make it into the kernel.
2) SSI has absolutely fantastic support for SSM (if I may say so myself), practically bug-free and ready for service at any moment. That’s our #1 feature, and Mercury doesn’t yet have that. The other side of that coin is that, admittedly, SSI does not support anything else. SSM is our BMI, and we have two now-stable versions (0.4 and 0.6) that SSI works with. Geoff reports that he made some major contributions to the Mercury’s SSM plugin, but it’s still not ready for real use.
3) SSI of course has its own problems. SSI is a bit larger than it needs to be. It includes a serialization API that we wouldn’t use. It also includes some other stuff to cover up some SSM shortcomings that have since become extinct. Luckily, those things are easy to strip out. The important features that it does include, buffer management and function registration, are there. Function registration is much less refined in SSI than Mercury, but it’s there.
4) Right now, we don’t have a compelling use case for transparently handling large messages. We will already have to write code to localize bulk arguments, so we just intend to treat larger messages the same way (except, of course, having to deserialize it once it’s transferred).
Matthew
On 8/7/14, 9:01 AM, "Phil Carns" <[email protected]> wrote:
Just to elaborate a little bit more, Mercury has the ability to use SSM for communication and XDR for encoding, which is nominally what you would be doing anyway. Both of those components are modularized.
Mercury is really an RPC system at its core, though, so it operates at a higher level and adds on a few things that you don't get by using SSM and XDR directly:
- buffer mgmt on the server side: you don't have to explicitly manage a pool of buffers for incoming requests; you just call a function to drive progress on incoming RPC operations
- transparent handling of large messages: it automatically switches to rendezvous transfers to larger RDMA buffers if a specific RPC request becomes too large for the pre-posted buffer pool
- expressing server requests as functions: because Mercury is an RPC system it just presents a higher level abstraction: you don't manage individual requests, acks, and buffers. You just post RPC functions to be executed and wait for them to finish.
- despite the above, in addition to the RPC mechanism you can still also do bulk RDMA operations as well to transfer large chunks of data, skipping the XDR encoding and RPC handshaking. Triton uses the two in tandem: it issues an RPC to inform the server of an IO request but lets the server issue explicit RDMA operations to the client to put or get the actual data payload.
Before we adopted Mercury we had a significant volume of code embedded within Triton to take care of those things; it was a relief to get some of that offloaded into a shared component in order to distribute the maintenance and testing load (there are several interested parties in the Mercury project).
Mercury doesn't have Aesop bindings; we built that part ourselves within Triton. It is a pretty compact chunk of code, though. We could split it out into a shared component if that would be helpful.
That concludes my sales pitch, but if you order now I'll throw in a complimentary set of Ginsu steak knives and a Shamwow towel :) We would like to know if there are specific things about Mercury that seem problematic, though. It might save some development and maintenance time if you were able to adopt it.
-Phil
On 08/05/2014 04:45 PM, Ross, Robert B. wrote:
They are really two completely different things. ― Rob
On Aug 5, 2014, at 3:39 PM, Danielson, Geoffrey Charles <[email protected]> wrote:
On 8/5/14, 2:39 PM, "Danielson, Geoffrey Charles" <[email protected]> wrote:
We will, but we¹re not planning to use mercury, we didn¹t see the added value over doing things manually with XDR.
On 8/5/14, 9:47 AM, "Phil Carns" <[email protected]> wrote:
Thanks Geoff. Keep us posted when you get to the transport stage. Mercury has been working pretty well for our purposes so far; we would be curious if you guys have any feedback about it.
thanks, -Phil
On 08/04/2014 02:29 PM, Danielson, Geoffrey Charles wrote:
No, we're just working on the arguments to pass on the wire right now, pretty agnostic of transport as yet.
Regards, Geoff Danielson [email protected] [email protected] Cell: (505) 263-0751
________________________________________ From: Phil Carns <[email protected]> Sent: Monday, August 4, 2014 10:12 AM To: Danielson, Geoffrey Charles; Curry, Matthew Leon Subject: [EXTERNAL] Re: [ASG] 20140804 - Sandia Status
Hi Geoff and Matthew,
Are you guys working with Mercury for the wire protocol stuff?
thanks, -Phil
On 08/04/2014 09:08 AM, Danielson, Geoffrey Charles wrote:
All,
Last week, Sandia continued working on transaction locking in localstore and the wire protocol for our server.
Regards, Geoff Danielson [email protected] [email protected] Cell: (505) 263-0751 _______________________________________________ ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
ASG mailing list [email protected] https://lists.mcs.anl.gov/mailman/listinfo/asg
participants (1)
-
Phil Carns