- devel - mailman3.cels.anl.gov

Re: [mpich-devel] [hwloc-devel] Build failure in OS X, libxml required?
by Lisandro Dalcin 07 Mar '15

07 Mar '15

OK, I finally figured out what's going on. I recently upgraded to Mac OS X Yosemite, and I did not install the Xcode command line tools. My /usr/include directory was missing, however I had /usr/bin/clang (not sure wether it is there because of the default Xcode install, or it was a leftover of my previous OS X system). I also had pkg-config installed trough Homebrew, and "pkg-config --cflags libxml-2.0" prints "-I/usr/include/libxml2", however remember I did not have "/usr/include". Do this pkg-config install is assuming /usr/include do exist. The thing is that MPICH is able to build fine (and in fact, many other software packages) as long as you pass --disable-libxml2 to workaround the hwloc issue. But after spending some time looking at hwloc build system, I realized the dependence on pkg-config to look for libxml2, and then there is no easy workaround. Finally, I've installed the Xcode command line tools (with "xcode-select --install") and MPICH + embed hwloc built just fine. Now I had the cmd line tools in my system, but it makes sense anyway, after all I use it everyday for development and the lack of it will likely cause me headaches in the near future with some other software. Some final comment for hwloc folks: * Relying in pkg-config for building is totally fine, but please note this tool is non-standard in OS X. * Looking in config/hwloc_pkg.m4, I noticed you somehow do check for broken stuff by attempting to link with package libraries. Well, if would be nice to extend this macro to also check you are able to compile by #include'ing package headers. IMHO, such sanity checks save time in the long run for both developer and users. Sorry I do not offer a patch, I have very limited knowledge of autotools and M4. On 7 March 2015 at 00:24, Brice Goglin <Brice.Goglin(a)inria.fr> wrote: > Hello, > > Sorry but we don't add configure checks without knowing what's causing > the problem. So far it only looks like a broken libxml install. > > We cannot check for all broken installs in the world. Otherwise one day > somebody will remove printf from his libc and request a new configure > check for printf() as well. And another check in case he modified > printf() to return different values. And another check in case printf() > was renamed to pruntf(). Endless. > > Please try to understand what caused this broken/partial libxml install > (do you have logs of the install?). > > By the way, let's make this new hwloc feature official: hwloc can now > detect broken libxml installs by failing to build :) > > Brice > > > > > > Le 06/03/2015 21:14, Balaji, Pavan a écrit : >> Hi, >> >> This is a problem with hwloc, which I had reported in the past. I believe this is not fixed in hwloc yet. >> >> The suggestion given at that point was to remove and install libxml again, which I did, and things started working correctly again. But, I agree, hwloc should detect this in configure and abort if there's an issue. >> >> I've cc'ed hwloc-devel. >> >> -- Pavan >> >>> On Mar 6, 2015, at 2:10 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: >>> >>> This is with the 3.1.4 tarball. Not sure if this is actually a problem >>> in MPICH or in my system. >>> >>> I got the following build failure. Shouldn't configure catch that beforehand? >>> >>> Making all in tools/topo/hwloc/hwloc >>> Making all in src >>> CC topology.lo >>> CC traversal.lo >>> CC distances.lo >>> CC components.lo >>> CC bind.lo >>> CC bitmap.lo >>> CC pci-common.lo >>> CC diff.lo >>> CC misc.lo >>> CC base64.lo >>> CC topology-noos.lo >>> CC topology-synthetic.lo >>> CC topology-custom.lo >>> CC topology-xml.lo >>> CC topology-xml-nolibxml.lo >>> CC topology-xml-libxml.lo >>> CC topology-darwin.lo >>> CC topology-x86.lo >>> topology-xml-libxml.c:17:10: fatal error: 'libxml/parser.h' file not found >>> #include <libxml/parser.h> >>> ^ >>> 1 error generated. >>> make[4]: *** [topology-xml-libxml.lo] Error 1 >>> make[4]: *** Waiting for unfinished jobs.... >>> make[3]: *** [all-recursive] Error 1 >>> make[2]: *** [all-recursive] Error 1 >>> make[1]: *** [all-recursive] Error 1 >>> make: *** [all] Error 2 >>> >>> >>> -- >>> Lisandro Dalcin >>> ============ >>> Research Scientist >>> Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) >>> Numerical Porous Media Center (NumPor) >>> King Abdullah University of Science and Technology (KAUST) >>> http://numpor.kaust.edu.sa/ >>> >>> 4700 King Abdullah University of Science and Technology >>> al-Khawarizmi Bldg (Bldg 1), Office # 4332 >>> Thuwal 23955-6900, Kingdom of Saudi Arabia >>> http://www.kaust.edu.sa >>> >>> Office Phone: +966 12 808-0459 >>> _______________________________________________ >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/devel >> -- >> Pavan Balaji ✉️ >> http://www.mcs.anl.gov/~balaji >> >> _______________________________________________ >> hwloc-devel mailing list >> hwloc-devel(a)open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel >> Link to this post: http://www.open-mpi.org/community/lists/hwloc-devel/2015/03/4400.php > -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

1 0

Re: [mpich-devel] Build failure in OS X, libxml required?
by Balaji, Pavan 06 Mar '15

06 Mar '15

Hi, This is a problem with hwloc, which I had reported in the past. I believe this is not fixed in hwloc yet. The suggestion given at that point was to remove and install libxml again, which I did, and things started working correctly again. But, I agree, hwloc should detect this in configure and abort if there's an issue. I've cc'ed hwloc-devel. -- Pavan > On Mar 6, 2015, at 2:10 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > > This is with the 3.1.4 tarball. Not sure if this is actually a problem > in MPICH or in my system. > > I got the following build failure. Shouldn't configure catch that beforehand? > > Making all in tools/topo/hwloc/hwloc > Making all in src > CC topology.lo > CC traversal.lo > CC distances.lo > CC components.lo > CC bind.lo > CC bitmap.lo > CC pci-common.lo > CC diff.lo > CC misc.lo > CC base64.lo > CC topology-noos.lo > CC topology-synthetic.lo > CC topology-custom.lo > CC topology-xml.lo > CC topology-xml-nolibxml.lo > CC topology-xml-libxml.lo > CC topology-darwin.lo > CC topology-x86.lo > topology-xml-libxml.c:17:10: fatal error: 'libxml/parser.h' file not found > #include <libxml/parser.h> > ^ > 1 error generated. > make[4]: *** [topology-xml-libxml.lo] Error 1 > make[4]: *** Waiting for unfinished jobs.... > make[3]: *** [all-recursive] Error 1 > make[2]: *** [all-recursive] Error 1 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 > > > -- > Lisandro Dalcin > ============ > Research Scientist > Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) > Numerical Porous Media Center (NumPor) > King Abdullah University of Science and Technology (KAUST) > http://numpor.kaust.edu.sa/ > > 4700 King Abdullah University of Science and Technology > al-Khawarizmi Bldg (Bldg 1), Office # 4332 > Thuwal 23955-6900, Kingdom of Saudi Arabia > http://www.kaust.edu.sa > > Office Phone: +966 12 808-0459 > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji

1 0

Re: [mpich-devel] patch for ticket #1526 (MPI_Alloc_mem registered memory)
by Zhao, Xin 06 Mar '15

06 Mar '15

Hi Steffen, Thank you very much for your contribution. We pushed your patch into MPICH (03d4c77b2148769436b8ce0b682b9b7a662baeef) with some modification. You can directly check it from our repo: git.mpich.org/mpich.git. Thanks, Xin ________________________________________ From: Steffen Christgau [christgau(a)cs.uni-potsdam.de] Sent: Tuesday, January 20, 2015 7:57 AM To: devel(a)mpich.org Subject: [mpich-devel] patch for ticket #1526 (MPI_Alloc_mem registered memory) Hi, I'm currently going to improve the RMA operations (and only those) of an existing CH3 channel for the underlying hardware in a local development branch of MPICH. I appreciate the feature to override the window creation functions in the channel implementation, and the possibility to override the RMA functions via the function table associated to each window. These together simplify development a lot. As one can override the OSC/RMA functions, I really wonder why it is not possible to do the same with the implementation of MPI_Alloc_mem at a CH3 channel level. I found that there was an attempt to introduce this in ticket #1526. Although the original ticket points at a slightly different spot, the discussion is about the issue I'm currently facing. Anyways, it seems to be dormant for a long time and I was not able to find traces of the OSC branch mentioned in the comments (likely that is has been merged into master or it is lost in history) where a solution for this issue was presented according to the comment. That said, I'd like to propose a patch to allow a channel implementation to provide a custom implementation of MPI_Alloc_mem resp. MPI_Free_mem. The idea is borrowed from MPIDI_CH3_Channel_close() which is declared if the MPIDI_CH3_HAS_CHANNEL_CLOSE macro is defined or falls back to the default preprocessor macro "implementation". Same thing here: if according macros are defined, the channel provides an implementation. If not, the defaults are used, i.e. the current MPIU-calls. The CH3 device implementation had to be adjusted to account these changes. As the solution falls back to macro substitution by default, I don't expect any performance impacts, broken code or test failures for existing channel devices. The RMA test cases succeeded both for the my channel device as they did before. The patch (see attachment as well) is based on the v3.1.3 tag. Maybe it helps other implementers (if there's any ;-)) to integrate the allocation of special memory into their channel. diff --git a/src/mpid/ch3/include/mpidimpl.h b/src/mpid/ch3/include/mpidimpl.h index 6a9db2d..1b0ae17 100644 --- a/src/mpid/ch3/include/mpidimpl.h +++ b/src/mpid/ch3/include/mpidimpl.h @@ -1222,6 +1222,19 @@ int MPIDI_Win_sync(MPID_Win *win); void *MPIDI_Alloc_mem(size_t size, MPID_Info *info_ptr); int MPIDI_Free_mem(void *ptr); +#ifdef MPIDI_CH3_HAS_ALLOC_MEM +void* MPIDI_CH3_Alloc_mem(size_t size, MPID_Info *info_ptr); +/* fallback to MPIU_Malloc if channel does not have its own RMA memory allocator */ +#else +#define MPIDI_CH3_Alloc_mem(size, info_ptr) MPIU_Malloc(size) +#endif + +#ifdef MPIDI_CH3_HAS_FREE_MEM +int MPIDI_Free_mem(void *ptr); +#else +#define MPIDI_CH3_Free_mem(ptr) MPIU_Free(ptr); +#endif + /* Pvars */ void MPIDI_CH3_RMA_Init_Pvars(void); diff --git a/src/mpid/ch3/src/ch3u_rma_ops.c b/src/mpid/ch3/src/ch3u_rma_ops.c index f30c464..7a3e22b 100644 --- a/src/mpid/ch3/src/ch3u_rma_ops.c +++ b/src/mpid/ch3/src/ch3u_rma_ops.c @@ -472,7 +472,7 @@ void *MPIDI_Alloc_mem( size_t size, MPID_Info *info_ptr ) MPIDI_FUNC_ENTER(MPID_STATE_MPIDI_ALLOC_MEM); - ap = MPIU_Malloc(size); + ap = MPIDI_CH3_Alloc_mem(size, info_ptr); MPIDI_FUNC_EXIT(MPID_STATE_MPIDI_ALLOC_MEM); return ap; @@ -490,7 +490,7 @@ int MPIDI_Free_mem( void *ptr ) MPIDI_FUNC_ENTER(MPID_STATE_MPIDI_FREE_MEM); - MPIU_Free(ptr); + MPIDI_CH3_Free_mem(ptr); MPIDI_FUNC_EXIT(MPID_STATE_MPIDI_FREE_MEM); return mpi_errno; Regards, Steffen

1 0

Re: [mpich-devel] Build failure in OS X, libxml required?
by Jeff Hammond 06 Mar '15

06 Mar '15

Write hwloc list for details... Jeff Sent from my iPhone > On Mar 6, 2015, at 12:10 AM, Lisandro Dalcin <dalcinl(a)gmail.com> wrote: > > This is with the 3.1.4 tarball. Not sure if this is actually a problem > in MPICH or in my system. > > I got the following build failure. Shouldn't configure catch that beforehand? > > Making all in tools/topo/hwloc/hwloc > Making all in src > CC topology.lo > CC traversal.lo > CC distances.lo > CC components.lo > CC bind.lo > CC bitmap.lo > CC pci-common.lo > CC diff.lo > CC misc.lo > CC base64.lo > CC topology-noos.lo > CC topology-synthetic.lo > CC topology-custom.lo > CC topology-xml.lo > CC topology-xml-nolibxml.lo > CC topology-xml-libxml.lo > CC topology-darwin.lo > CC topology-x86.lo > topology-xml-libxml.c:17:10: fatal error: 'libxml/parser.h' file not found > #include <libxml/parser.h> > ^ > 1 error generated. > make[4]: *** [topology-xml-libxml.lo] Error 1 > make[4]: *** Waiting for unfinished jobs.... > make[3]: *** [all-recursive] Error 1 > make[2]: *** [all-recursive] Error 1 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 > > > -- > Lisandro Dalcin > ============ > Research Scientist > Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) > Numerical Porous Media Center (NumPor) > King Abdullah University of Science and Technology (KAUST) > http://numpor.kaust.edu.sa/ > > 4700 King Abdullah University of Science and Technology > al-Khawarizmi Bldg (Bldg 1), Office # 4332 > Thuwal 23955-6900, Kingdom of Saudi Arabia > http://www.kaust.edu.sa > > Office Phone: +966 12 808-0459 > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel

1 0

Build failure in OS X, libxml required?
by Lisandro Dalcin 06 Mar '15

06 Mar '15

This is with the 3.1.4 tarball. Not sure if this is actually a problem in MPICH or in my system. I got the following build failure. Shouldn't configure catch that beforehand? Making all in tools/topo/hwloc/hwloc Making all in src CC topology.lo CC traversal.lo CC distances.lo CC components.lo CC bind.lo CC bitmap.lo CC pci-common.lo CC diff.lo CC misc.lo CC base64.lo CC topology-noos.lo CC topology-synthetic.lo CC topology-custom.lo CC topology-xml.lo CC topology-xml-nolibxml.lo CC topology-xml-libxml.lo CC topology-darwin.lo CC topology-x86.lo topology-xml-libxml.c:17:10: fatal error: 'libxml/parser.h' file not found #include <libxml/parser.h> ^ 1 error generated. make[4]: *** [topology-xml-libxml.lo] Error 1 make[4]: *** Waiting for unfinished jobs.... make[3]: *** [all-recursive] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 -- Lisandro Dalcin ============ Research Scientist Computer, Electrical and Mathematical Sciences & Engineering (CEMSE) Numerical Porous Media Center (NumPor) King Abdullah University of Science and Technology (KAUST) http://numpor.kaust.edu.sa/ 4700 King Abdullah University of Science and Technology al-Khawarizmi Bldg (Bldg 1), Office # 4332 Thuwal 23955-6900, Kingdom of Saudi Arabia http://www.kaust.edu.sa Office Phone: +966 12 808-0459

1 0

Re: [mpich-devel] Possible problem with accfence1 test
by Zhao, Xin 24 Feb '15

24 Feb '15

Hi Nathan, Jeff, We are aware of this bug and it is recently fixed in mpich/master. Could you try tonight’s mpich nightly snapshot (http://www.mpich.org/static/downloads/nightly/master/mpich/) ? Thanks, Xin ________________________________________ From: Jeff Hammond [jeff.science(a)gmail.com] Sent: Tuesday, February 24, 2015 5:56 PM To: devel(a)mpich.org Subject: Re: [mpich-devel] Possible problem with accfence1 test Rajeev pointed out offline that I cannot read. He said it nicely, but that doesn't mean I'm not a bleeping idiot. Sorry for my epic fail. I hate stupid people, especially when it's me. Jeff Sent from my iPhone > On Feb 24, 2015, at 1:32 PM, Nathan Hjelm <hjelmn(a)lanl.gov> wrote: > > > Hey, I am looking that the accfence1 test in MPICH and I think the tests > is invalid. The test attempts to execute an MPI_Accumulate with different > datatypes for the origin and destination. The problem is that the test > violates MPI 3.0 § 11.3.4 p. 426 1-4: > > > Each datatype argument must be a predefined datatype or a derived > datatype, where all basic components are of the same predefined > datatype. Both datatype arguments must be constructed from the same > predefined datatype. The operation op applies to elements of that > predefined type. > > > I see no wording in the standard that says this restriction does not > apply when the operator is MPI_REPLACE. The test is correct if the > operation is changed from MPI_Accumulate to MPI_Put. > > If the test is correct then it might make sense to make it explicit in > the standard that the datatype restriction does not apply to MPI_REPLACE. > > -Nathan > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel _______________________________________________ To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/devel

1 0

Re: [mpich-devel] Possible problem with accfence1 test
by Jeff Hammond 24 Feb '15

24 Feb '15

Rajeev pointed out offline that I cannot read. He said it nicely, but that doesn't mean I'm not a bleeping idiot. Sorry for my epic fail. I hate stupid people, especially when it's me. Jeff Sent from my iPhone > On Feb 24, 2015, at 1:32 PM, Nathan Hjelm <hjelmn(a)lanl.gov> wrote: > > > Hey, I am looking that the accfence1 test in MPICH and I think the tests > is invalid. The test attempts to execute an MPI_Accumulate with different > datatypes for the origin and destination. The problem is that the test > violates MPI 3.0 § 11.3.4 p. 426 1-4: > > > Each datatype argument must be a predefined datatype or a derived > datatype, where all basic components are of the same predefined > datatype. Both datatype arguments must be constructed from the same > predefined datatype. The operation op applies to elements of that > predefined type. > > > I see no wording in the standard that says this restriction does not > apply when the operator is MPI_REPLACE. The test is correct if the > operation is changed from MPI_Accumulate to MPI_Put. > > If the test is correct then it might make sense to make it explicit in > the standard that the datatype restriction does not apply to MPI_REPLACE. > > -Nathan > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel

1 0

Re: [mpich-devel] Possible problem with accfence1 test
by Jeff Hammond 24 Feb '15

24 Feb '15

On Tue, Feb 24, 2015 at 1:32 PM, Nathan Hjelm <hjelmn(a)lanl.gov> wrote: > > Hey, I am looking that the accfence1 test in MPICH and I think the tests > is invalid. The test attempts to execute an MPI_Accumulate with different > datatypes for the origin and destination. The problem is that the test > violates MPI 3.0 § 11.3.4 p. 426 1-4: Not everyone has the standard open on their computer all the time, so for reference, you are referring to: "Each datatype argument must be a predefined datatype or a derived datatype, where all basic components are of the same predefined datatype. Both datatype arguments must be constructed from the same predefined datatype." > Each datatype argument must be a predefined datatype or a derived > datatype, where all basic components are of the same predefined > datatype. Both datatype arguments must be constructed from the same > predefined datatype. The operation op applies to elements of that > predefined type. Indeed, I see that e.g. MTestGetSendDerivedDatatypes use as the underlying predefined type MPI_DOUBLE on one side and MPI_CHAR on the other, which violated the aforementioned condition. > I see no wording in the standard that says this restriction does not > apply when the operator is MPI_REPLACE. The test is correct if the > operation is changed from MPI_Accumulate to MPI_Put. Because MPI_Put doesn't have to be atomic, whereas MPI_Accumulate with MPI_REPLACE does. > If the test is correct then it might make sense to make it explicit in > the standard that the datatype restriction does not apply to MPI_REPLACE. We don't want to do that. Trust me. I'll be happy to have the debate on the MPI Forum RMA WG list if you want, but the right answer here is the trivial mods to the MPICH test suite to not violate the MPI standard. Best, Jeff -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0

Possible problem with accfence1 test
by Nathan Hjelm 24 Feb '15

24 Feb '15

Hey, I am looking that the accfence1 test in MPICH and I think the tests is invalid. The test attempts to execute an MPI_Accumulate with different datatypes for the origin and destination. The problem is that the test violates MPI 3.0 § 11.3.4 p. 426 1-4: Each datatype argument must be a predefined datatype or a derived datatype, where all basic components are of the same predefined datatype. Both datatype arguments must be constructed from the same predefined datatype. The operation op applies to elements of that predefined type. I see no wording in the standard that says this restriction does not apply when the operator is MPI_REPLACE. The test is correct if the operation is changed from MPI_Accumulate to MPI_Put. If the test is correct then it might make sense to make it explicit in the standard that the datatype restriction does not apply to MPI_REPLACE. -Nathan

1 0

Re: [mpich-devel] MPICH memory pools
by Jeff Hammond 20 Jan '15

20 Jan '15

The usage justification for request-based RMA should not be sensitive to software overhead. I would expect that users concerned about, e.g. message-rate, would not use Rput but rather Put. Jeff On Tue, Jan 20, 2015 at 9:38 AM, Archer, Charles J <charles.j.archer(a)intel.com> wrote: > There are other inflexibilities with the handle encoding that can lead to unintended behavior, too. > Although it’s not obvious, the pooling disallows using different type of object use, each handle must map to an object 1-1, which can lead to space inefficiencies. > This is probably best illustrated by example. > > Let’s say you want to allocate a different request object for two sided (MPID_REQUEST kind) and 1 sided communication (MPID_WREQUEST kind). > Each object has a different memory footprint, so you want the pools to be distinct. > > Now, if you call MPI_RPut, under the covers, you call your window object pool to allocate the request (allocating an MPID_WREQUEST object). > Everything is fine until you call MPI_Wait, the code will call Request_get_ptr macro, which points you to the wrong pool. > The pool is hardcoded to use MPID_REQUEST into the macro name, so your code will faceplant. > > The set of (non-ideal) options I see are: > 1) In common code to decode both the handle kind and the object kind (more branches) to get the actual pointer to the request. Call Request_get_actual_ptr, which checks the object kind and the handle kind. > 2) Waste space allocating a union (likely the wrequest and the request will not be the same size, leading to space). > 3) Allocate a window request from the window pool, and a signal request from the request pool, and complete signal request on window request completion. This is both space and time inefficient. pamid does this today. > > So you get your choice of #1 (extra branches/cycles) or #2 (wasted space) or #3 (wasted space and extra cycles). > It’s not obvious which of #1 or #2 is better, but #3 should be the worst option. > > It seems there’s no good/easy way to hack in your own fancy allocator (like tcmalloc/jemalloc, whatever) and have the handle encoding point to this allocation directly, so we may be stuck with choice #3 for external MPI objects. > > > >> On Jan 20, 2015, at 8:47 AM, Dave Goodell (dgoodell) <dgoodell(a)cisco.com> wrote: >> >> The existing mempool stuff isn't particularly time efficient, as I recall. You might want to benchmark it for your use case against a proper memory allocator like Hoard or tcmalloc and make sure it meets your needs. It's also not overly space efficient, since IIRC it won't ever return memory to the OS or even to other memory users in the same process. >> >> The mempool stuff only really exists for two reasons: >> >> 1. So that the "encode the predefined type width in the handle value" optimization can be implemented in MPICH. IMO this is a pretty questionable on modern processors, but if we were to argue about that we should probably do some benchmarking rather than waving our hands. >> >> 2. So that one can implement all handles as integers, which simplifies the implementation of the Fortran bindings and avoids penalizing Fortran codes with one or more handle translation lookups on every MPI call. The "kind" field of the handle value helps with type checking, which you would otherwise get from the compiler if properly pointers were used as the handle type instead of integers. >> >> It seems unlikely that you need either of these features in some subsystem. >> >> If I needed some new allocation logic in my netmod/device/whatever, I'd look for something off the shelf first, then roll my own second. I'd probably stay away from the existing mempool stuff unless there was a killer feature there I'm forgetting about. >> >> -Dave >> >> On Jan 20, 2015, at 10:06 AM, Archer, Charles J <charles.j.archer(a)intel.com> wrote: >> >>> Hi. >>> >>> MPICH has some pretty nice functionality for memory pools implemented, but as far as I can tell, it’s a bit limited for internal device use because each pool you implement needs to consume an entry in the handle space. >>> >>> Looking at the available “kinds” of memory pools already implemented: >>> >>> typedef enum MPID_Object_kind { >>> MPID_COMM = 0x1, >>> MPID_GROUP = 0x2, >>> MPID_DATATYPE = 0x3, >>> MPID_FILE = 0x4, /* only used obliquely inside MPID_Errhandler objs */ >>> MPID_ERRHANDLER = 0x5, >>> MPID_OP = 0x6, >>> MPID_INFO = 0x7, >>> MPID_WIN = 0x8, >>> MPID_KEYVAL = 0x9, >>> MPID_ATTR = 0xa, >>> MPID_REQUEST = 0xb, >>> MPID_PROCGROUP = 0xc, /* These are internal device objects */ >>> MPID_VCONN = 0xd, >>> MPID_GREQ_CLASS = 0xf >>> } MPID_Object_kind; >>> >>> It looks like only 0xe is available for implementing a new type of memory pool, limiting me to one additional pool. >>> Furthermore, the internal device objects don’t need publishable handles, right? >>> It looks like the handle contains 2 bits for (internal, valid, invalid, direct), and 4 bits to contain the object kind. >>> >>> Are there any memory pool routines that I’m missing somewhere that aren’t restricted to the limits of what we can publish in a handle? >>> Since my object pools are internal, I don’t need to encode anything into a handle. >>> >>> Furthermore, if we had a set of non-handle pool routines, internal pools like procgroup, vconn, wouldn’t consume entries in the handle space that could be used for future versions of MPI. >>> >>> Looking for some guidance here, I don’t want to publish any internal device gorp into the object_kind space…but I want to use memory pools. >>> I’ve used the mpich pools on internal objects with a garbage kind value (unintentionally set to the wrong enum value), and it appears I get a new pool and everything is working, but just because it works, doesn’t mean it’s correct. >>> >>> What should I do? Bracing for "implement your own memory pools, lazy”. >>> _______________________________________________ >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/devel >> >> _______________________________________________ >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/devel > > _______________________________________________ > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/devel -- Jeff Hammond jeff.science(a)gmail.com http://jeffhammond.github.io/

1 0