- devel - mailman3.cels.anl.gov

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Jeff Squyres (jsquyres) 04 Sep '15

04 Sep '15

It's the work of 30 seconds, so I'll just resubmit without the signoff and whitespace changes. You can use my re-submission, or if you've done it locally already, cool. > On Sep 4, 2015, at 3:31 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > > On Sep 4, 2015, at 3:28 PM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov> wrote: >> >> This patch looks correct. I'm just running it through our Jenkins to sanity check it. A few minor issues. >> >> 1. I'd like to omit the whitespace modifications so it's clear that it's just the 2 lines that were added for this change. > > Sure, no problem. I did it because everything was lined up; it looked like someone did that on purpose, so I just maintained it. > > Would you like me to re-submit without the whitespace change? I'm happy to do so. > >> 2. Our project uses "Signed-off-by" as an indication of code review. While we are aware this is contrary to how most git projects work, this is the current situation. For consistency, I would remove your signoff and add my own to indicate I reviewed the patch before pushing to master. Does that sound okay? > > No problem from me. If you'd like me to re-submit for the whitespace stuff, I can remove my signoff. > > -- > Jeff Squyres > jsquyres(a)cisco.com > For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/ > -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Jeff Squyres (jsquyres) 04 Sep '15

04 Sep '15

On Sep 4, 2015, at 3:28 PM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov> wrote: > > This patch looks correct. I'm just running it through our Jenkins to sanity check it. A few minor issues. > > 1. I'd like to omit the whitespace modifications so it's clear that it's just the 2 lines that were added for this change. Sure, no problem. I did it because everything was lined up; it looked like someone did that on purpose, so I just maintained it. Would you like me to re-submit without the whitespace change? I'm happy to do so. > 2. Our project uses "Signed-off-by" as an indication of code review. While we are aware this is contrary to how most git projects work, this is the current situation. For consistency, I would remove your signoff and add my own to indicate I reviewed the patch before pushing to master. Does that sound okay? No problem from me. If you'd like me to re-submit for the whitespace stuff, I can remove my signoff. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Kenneth Raffenetti 04 Sep '15

04 Sep '15

On 09/04/2015 12:27 PM, Jeff Squyres (jsquyres) wrote: > On Sep 4, 2015, at 12:27 PM, Rob Latham <robl(a)mcs.anl.gov> wrote: >> >> We have a formal workflow for frequent contributors that involves sign-offs and such. For occasional contributions, sending a patch to devel@mpich (as you did) works great. >> >> Ken's going to review this. > > Thanks! > > This patch is important to us because Cisco's usNIC libfabric support is actual UDP under the covers, hence, if you want ordering (and you really do when you use the tag-matching interface; it doesn't make sense, otherwise), then you need to ask for it. > > Other libfabric interfaces that provide the tag-matching support (e.g., PSM) are inherently ordered already, so even if you didn't ask for it, you got it, and it all worked out ok. > Jeff, This patch looks correct. I'm just running it through our Jenkins to sanity check it. A few minor issues. 1. I'd like to omit the whitespace modifications so it's clear that it's just the 2 lines that were added for this change. 2. Our project uses "Signed-off-by" as an indication of code review. While we are aware this is contrary to how most git projects work, this is the current situation. For consistency, I would remove your signoff and add my own to indicate I reviewed the patch before pushing to master. Does that sound okay? Thanks, Ken

1 0

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Jeff Squyres (jsquyres) 04 Sep '15

04 Sep '15

On Sep 4, 2015, at 12:27 PM, Rob Latham <robl(a)mcs.anl.gov> wrote: > > We have a formal workflow for frequent contributors that involves sign-offs and such. For occasional contributions, sending a patch to devel@mpich (as you did) works great. > > Ken's going to review this. Thanks! This patch is important to us because Cisco's usNIC libfabric support is actual UDP under the covers, hence, if you want ordering (and you really do when you use the tag-matching interface; it doesn't make sense, otherwise), then you need to ask for it. Other libfabric interfaces that provide the tag-matching support (e.g., PSM) are inherently ordered already, so even if you didn't ask for it, you got it, and it all worked out ok. -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Rob Latham 04 Sep '15

04 Sep '15

On 09/04/2015 10:16 AM, Jeff Squyres (jsquyres) wrote: > Just curious, what's the process for getting this patch accepted into MPICH? > We have a formal workflow for frequent contributors that involves sign-offs and such. For occasional contributions, sending a patch to devel@mpich (as you did) works great. Ken's going to review this. ==rob > Thanks. > > >> On Sep 3, 2015, at 1:24 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: >> >> Per https://github.com/ofiwg/libfabric/issues/1217#issuecomment-131264316, >> you really need to specify FI_ORDER_SAS for both transmit and receive >> if you want to guarantee ordering on FI_EP_RDM endpoints (and you do; >> the MPI-style tagging interface doesn't make much sense unless the >> underlying fabric provider guarnatees ordering). >> >> Signed-off-by: Jeff Squyres <jsquyres(a)cisco.com> >> --- >> src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c | 10 ++++++---- >> 1 file changed, 6 insertions(+), 4 deletions(-) >> >> diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c >> index 5de47b7..2cf3337 100644 >> --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c >> +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c >> @@ -77,10 +77,12 @@ int MPID_nem_ofi_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_ >> /* We expect to register all memory up front for use with this */ >> /* endpoint, so the netmod requires dynamic memory regions */ >> /* ------------------------------------------------------------------------ */ >> - hints = fi_allocinfo(); >> - hints->mode = FI_CONTEXT; >> - hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ >> - hints->caps = FI_TAGGED; /* Tag matching interface */ >> + hints = fi_allocinfo(); >> + hints->mode = FI_CONTEXT; >> + hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ >> + hints->caps = FI_TAGGED; /* Tag matching interface */ >> + hints->tx_attr->msg_order = FI_ORDER_SAS; >> + hints->rx_attr->msg_order = FI_ORDER_SAS; >> >> hints->ep_attr->mem_tag_format = MEM_TAG_FORMAT; >> MPIU_Assert(pg_p->size < ((1 << MPID_RANK_BITS) - 1)); >> -- >> 2.4.3 >> > > -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA

1 0

Re: [mpich-devel] [PATCH] ofi: need to specify ordering for FI_EP_RDM
by Jeff Squyres (jsquyres) 04 Sep '15

04 Sep '15

Just curious, what's the process for getting this patch accepted into MPICH? Thanks. > On Sep 3, 2015, at 1:24 PM, Jeff Squyres (jsquyres) <jsquyres(a)cisco.com> wrote: > > Per https://github.com/ofiwg/libfabric/issues/1217#issuecomment-131264316, > you really need to specify FI_ORDER_SAS for both transmit and receive > if you want to guarantee ordering on FI_EP_RDM endpoints (and you do; > the MPI-style tagging interface doesn't make much sense unless the > underlying fabric provider guarnatees ordering). > > Signed-off-by: Jeff Squyres <jsquyres(a)cisco.com> > --- > src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c | 10 ++++++---- > 1 file changed, 6 insertions(+), 4 deletions(-) > > diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c > index 5de47b7..2cf3337 100644 > --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c > +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c > @@ -77,10 +77,12 @@ int MPID_nem_ofi_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_ > /* We expect to register all memory up front for use with this */ > /* endpoint, so the netmod requires dynamic memory regions */ > /* ------------------------------------------------------------------------ */ > - hints = fi_allocinfo(); > - hints->mode = FI_CONTEXT; > - hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ > - hints->caps = FI_TAGGED; /* Tag matching interface */ > + hints = fi_allocinfo(); > + hints->mode = FI_CONTEXT; > + hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ > + hints->caps = FI_TAGGED; /* Tag matching interface */ > + hints->tx_attr->msg_order = FI_ORDER_SAS; > + hints->rx_attr->msg_order = FI_ORDER_SAS; > > hints->ep_attr->mem_tag_format = MEM_TAG_FORMAT; > MPIU_Assert(pg_p->size < ((1 << MPID_RANK_BITS) - 1)); > -- > 2.4.3 > -- Jeff Squyres jsquyres(a)cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/

1 0

[PATCH] ofi: need to specify ordering for FI_EP_RDM
by Jeff Squyres 03 Sep '15

03 Sep '15

Per https://github.com/ofiwg/libfabric/issues/1217#issuecomment-131264316, you really need to specify FI_ORDER_SAS for both transmit and receive if you want to guarantee ordering on FI_EP_RDM endpoints (and you do; the MPI-style tagging interface doesn't make much sense unless the underlying fabric provider guarnatees ordering). Signed-off-by: Jeff Squyres <jsquyres(a)cisco.com> --- src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c index 5de47b7..2cf3337 100644 --- a/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c +++ b/src/mpid/ch3/channels/nemesis/netmod/ofi/ofi_init.c @@ -77,10 +77,12 @@ int MPID_nem_ofi_init(MPIDI_PG_t * pg_p, int pg_rank, char **bc_val_p, int *val_ /* We expect to register all memory up front for use with this */ /* endpoint, so the netmod requires dynamic memory regions */ /* ------------------------------------------------------------------------ */ - hints = fi_allocinfo(); - hints->mode = FI_CONTEXT; - hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ - hints->caps = FI_TAGGED; /* Tag matching interface */ + hints = fi_allocinfo(); + hints->mode = FI_CONTEXT; + hints->ep_attr->type = FI_EP_RDM; /* Reliable datagram */ + hints->caps = FI_TAGGED; /* Tag matching interface */ + hints->tx_attr->msg_order = FI_ORDER_SAS; + hints->rx_attr->msg_order = FI_ORDER_SAS; hints->ep_attr->mem_tag_format = MEM_TAG_FORMAT; MPIU_Assert(pg_p->size < ((1 << MPID_RANK_BITS) - 1)); -- 2.4.3

1 0

Argonne Spam Quarantine Notification
by Argonne Anti Spam Quarantine 13 Aug '15

13 Aug '15

++ Note: This message has been sent by a notification only system. Please do not reply ++ IronPort Spam Quarantine Notification If a spam message was missed, or if a message was incorrectly flagged as spam, you can add the sender to a personal safelist or blocklist via the quarantine. Senders on the safelist will not be flagged as spam, and senders on the blocklist will be automatically quarantined. For more information, visit: https://wiki.inside.anl.gov/inside/Ironport_Anti-Spam_Appliances/Documentat… The message(s) below have been quarantined by the Argonne National Laboratory spam filtering system. There are 1 new messages in your Email Quarantine since you received your last Spam Quarantine Notification. If the messages below are spam, you do not need to take any action. Messages will be automatically deleted from the quarantine after 28 day(s). If any of the messages below are not spam, click the 'Release' link to have them sent to your Inbox. Messages that have been correctly flagged as spam do not need to be reported. ----------- New Quarantine Messages --------------- Message 1 From: "Dave Sullivan" <DaveSullivan(a)ximilo.faith> Subject: *****SPAM*****Why You Must Stop Taking 'Little Blue Pills"NOW Date: 12 Aug 2015 Release: http://mailgateway.anl.gov:8080/Message?action=Release&mid=90390294&h=cf4d1… --------------------------------------------------- To manage your quarantine please visit the URL below: http://mailgateway.anl.gov:8080/Search?h=2ffb2bd0571f729efb89bf11a9fc382c&e… If you have any questions regarding the Laboratory spam filtering product, please contact the CIS helpdesk at help(a)anl.gov or x-9999 option 2.

1 0

Argonne Spam Quarantine Notification
by Argonne Anti Spam Quarantine 08 Aug '15

08 Aug '15

++ Note: This message has been sent by a notification only system. Please do not reply ++ IronPort Spam Quarantine Notification If a spam message was missed, or if a message was incorrectly flagged as spam, you can add the sender to a personal safelist or blocklist via the quarantine. Senders on the safelist will not be flagged as spam, and senders on the blocklist will be automatically quarantined. For more information, visit: https://wiki.inside.anl.gov/inside/Ironport_Anti-Spam_Appliances/Documentat… The message(s) below have been quarantined by the Argonne National Laboratory spam filtering system. There are 2 new messages in your Email Quarantine since you received your last Spam Quarantine Notification. If the messages below are spam, you do not need to take any action. Messages will be automatically deleted from the quarantine after 28 day(s). If any of the messages below are not spam, click the 'Release' link to have them sent to your Inbox. Messages that have been correctly flagged as spam do not need to be reported. ----------- New Quarantine Messages --------------- Message 1 From: "Central Air For A Fraction" <cooler(a)bsdaircons.com> Subject: *****SPAM*****Ductless Air Cooling, any house, a fraction what you thought. Date: 07 Aug 2015 Release: http://mailgateway.anl.gov:8080/Message?action=Release&mid=89965111&h=e26cd… Message 2 From: "Central Air For A Fraction" <cooler(a)bsdaircons.com> Subject: *****SPAM*****Ductless Air Cooling, any house, a fraction what you thought. Date: 07 Aug 2015 Release: http://mailgateway.anl.gov:8080/Message?action=Release&mid=89964975&h=9f1b5… --------------------------------------------------- To manage your quarantine please visit the URL below: http://mailgateway.anl.gov:8080/Search?h=2ffb2bd0571f729efb89bf11a9fc382c&e… If you have any questions regarding the Laboratory spam filtering product, please contact the CIS helpdesk at help(a)anl.gov or x-9999 option 2.

1 0

Argonne Spam Quarantine Notification
by Argonne Anti Spam Quarantine 07 Aug '15

07 Aug '15

++ Note: This message has been sent by a notification only system. Please do not reply ++ IronPort Spam Quarantine Notification If a spam message was missed, or if a message was incorrectly flagged as spam, you can add the sender to a personal safelist or blocklist via the quarantine. Senders on the safelist will not be flagged as spam, and senders on the blocklist will be automatically quarantined. For more information, visit: https://wiki.inside.anl.gov/inside/Ironport_Anti-Spam_Appliances/Documentat… The message(s) below have been quarantined by the Argonne National Laboratory spam filtering system. There are 2 new messages in your Email Quarantine since you received your last Spam Quarantine Notification. If the messages below are spam, you do not need to take any action. Messages will be automatically deleted from the quarantine after 28 day(s). If any of the messages below are not spam, click the 'Release' link to have them sent to your Inbox. Messages that have been correctly flagged as spam do not need to be reported. ----------- New Quarantine Messages --------------- Message 1 From: "Upgraded To MBA" <mbaedu(a)onlinembaedu.com> Subject: *****SPAM*****MBA program -- 12 months to done Date: 06 Aug 2015 Release: http://mailgateway.anl.gov:8080/Message?action=Release&mid=89863975&h=f23e8… Message 2 From: "Upgraded To MBA" <mbaedu(a)onlinembaedu.com> Subject: *****SPAM*****MBA program -- 12 months to done Date: 06 Aug 2015 Release: http://mailgateway.anl.gov:8080/Message?action=Release&mid=89863469&h=912d6… --------------------------------------------------- To manage your quarantine please visit the URL below: http://mailgateway.anl.gov:8080/Search?h=2ffb2bd0571f729efb89bf11a9fc382c&e… If you have any questions regarding the Laboratory spam filtering product, please contact the CIS helpdesk at help(a)anl.gov or x-9999 option 2.

1 0