- itaps-parallel - mailman3.cels.anl.gov

Re: itaps-parallel [Fwd: **** iMesh.h and iBase.h changes ****]
by Carl Ollivier-Gooch 09 Jan '09

09 Jan '09

Lori A. Diachin wrote: > Hi All, > > Karen just pointed out that I read her email backwards and she is not > available on the 21st, so I'd like to go with the second best option > which is Tuesday the 20th at 1 pm so everyone can participate for at > least part of the time. It's also a bit better from the perspective of > the paper. > > Sorry for the confusion. > > Date: Tuesday, Jan 20 > Time: 1 pm PST > Number: 866-914-3976 > Passcode: 662912# > > Lori > > At 01:09 PM 1/8/2009, Lori A. Diachin wrote: >> Hi All, >> >> It looks like there is mutual exclusivity until Jan 21. Everyone who >> responded is available from 1-2 pm pacific on that day. Please be >> prepared to talk about your status in updating your implementations >> wrt to the iMesh.h and iBase.h changes Jason proposed and to discuss >> item 6. >> >> Carl, if there are any significant areas we should look at in the >> paper, please send it out in advance with pointers to the relevant >> areas of text. >> >> The LLNL number: 866-914-3976 The passcode is 662912# >> >> Lori >> >>> Hi All, >>> >>> I'd like to check in on the status of everyone's changes wrt to the >>> iMesh.h and iBase.h files that Jason sent out back in October. We >>> decided to defer these until after SC08 and gave ourselves an >>> internal deadline of Dec 15... so, are we done yet? :-) >>> >>> Seriously - it was my understanding that folks had agreed in >>> principle to all of these changes; the one exception was what to do >>> about item #6 below. It may very well be that we need a quick >>> telecon to discuss this as well as get an update on status. Carl is >>> planning to send out the iMesh paper by Jan 23, so we may want to >>> touch base before then.. Speaking of which, here's the latest version. Things I'd like people to look at: o The canonical ordering picture / table. I think I managed to make it clear, but please have a look and confirm/deny. o In the section on the swapping service, I've left in the comparisons of memory and CPU time for all three implementations, despite the fact that FMDB and MOAB don't have the critical calls tweaked. Tim, Mark (et al), I'm happy to take those comparisons out for now, although I think we'd be well-advised to have the fully optimized versions in the final paper either way. o Lori has promised I'll have new FE results before this goes to press. :-) And thanks to Jason for cleaning up the Mesquite section. Carl -- ------------------------------------------------------------------------ Dr. Carl Ollivier-Gooch, P.Eng. Voice: +1-604-822-1854 Associate Professor Fax: +1-604-822-2403 Department of Mechanical Engineering email: cfog(a)mech.ubc.ca University of British Columbia http://www.mech.ubc.ca/~cfog Vancouver, BC V6T 1Z4 http://tetra.mech.ubc.ca/ANSLab/ ------------------------------------------------------------------------

2 1

ITAPS meeting in March
by Lori A. Diachin 05 Jan '09

05 Jan '09

Hi All, I hope everyone had a good holiday break and a bit of a rest after our big push to get the SC08 Tutorial in good shape. It's time though to get back to ITAPS and start planning for the review this spring. I still don't have a date from Bill Spotz, but I know from talking to Bill and Walt Polansky that they will require us to submit some written material in advance of the review to send to the reviewers. So I'd like to start taking care of some of the more mechanical items so that they are in place and also ask that you start thinking about and compiling a list of major accomplishments we should highlight. So mechanical item #1 - we need up update our publication list on the web site. It stops rather abruptly at 2005, so please send me your ITAPS-related publications since then by Friday, Jan 23. Please take a look at the web site for format (authors, title, journal/conference info). Also, we will have an all hands meeting in association with the SIAM CSE conference in Miami to further plan for the review and for the next two and half years of the project. I would like all senior personnel to attend and, if convenient, more junior folks as well. The ITAPS minisymposium is on Thursday afternoon (2-6:30), so it makes sense to me to have the ITAPS all hands on Friday. We could also have it on Wednesday - both are full conference days - please let me know your preference ASAP so that I can arrange a meeting room at the hotel. Thanks, Lori

1 0

Re: itaps-parallel VisIt Linked with Parallel FMDB
by Onkar Sahni 19 Nov '08

19 Nov '08

Thats good... keep us updated and in case if you need help in debugging (and also if errors are coming from FMDB). I guess we would not be able to resolve this before SC08 but if we do I will be happy to load a mesh in VisIt and just project it and show that it loads. - Onkar > Hi Onkar, > > Some good news. I've managed to link VisIt with parallel FMDB vi iMeshP > and open a multi-part sms file. However, when I go to plot it in VisIt, > it is segv'ing somewhere in VisIt. So, I still have some work to do. > However, up until now, it has been a major chore to get everything to > link and operate correctly. VisIt's shared libraries have made it > particularly difficult. > > Mark > > On Wed, 2008-11-12 at 16:33, Onkar Sahni wrote: >> I think my point is being missed, here is my last attempt... if in case >> of >> MOAB "AC_CONFIG_HEADERS is called to produce MBCN_FCDefs.h" why not one >> can produce "ITAPS_FCDefs.h" in a central place and one central >> iMesh_protos.h includes ITAPS_FCDefs.h that is generated during as an >> install/build process (all ITAPS header files can live in one location, >> for example, /usr/local/itaps/headers). >> >> In summary, user/app. would do following basic steps: >> >> * Download itaps-headers (say in /<path-itaps-headers>/) >> * Run configure to generate ITAPS_FCDefs.h in /<path-itaps-headers>/ (an >> app./user would do this step with platform and compiler combination of >> their choice) >> >> * Download any implementation/service (such as FMDB/MOAB/GRUMMP or >> iZoltan) >> * Run configure and compile it (point to /<path-itaps-headers>/ to get >> header files from central place) >> >> * Write app. code like MHD solver >> * Build it and link with appropriate libs. (ofcourse fortran name >> mangling >> has to be consistent while compling app. code with ITAPS_FCDefs.h >> generated by app.) >> >> Is this not possible? >> >> - Onkar >> >> > On Wed, 2008-11-12 at 16:53 -0500, Onkar Sahni wrote: >> >> > The FC_FUNC macro winds up getting defined by something; Autoconf's >> >> configure script does some work to define it. Or, like Onkar >> suggests, >> >> you can define some logic ahead of time to set whatever FC_FUNC does >> >> using logic like so... >> >> > >> >> > #ifdef AIX >> >> > #define FC_FUNC(lowFunc,UPfunc) _lowFunc >> >> > #elif GFORTRAN >> >> > #define FC_FUNC(lowFunc,UPfunc) UPfunc >> >> > #elif ... >> >> > .. >> >> > . >> >> >> >> So here is my question (basically what I was trying to ask in >> >> tele-con) >> >> which may or may not be possible: >> >> >> >> Can "#define FC_FUNC............" be generated in a central place >> >> during >> >> the process to install and build itaps softwares (before building any >> > >> > I don't think it can because its definition depends on the name >> mangling >> > properties of the fortran compiler being used at time of >> (configuration) >> > compilation. Yes, the number of possible definitions for FC_FUNC is >> > finite, but know which it should be set to for a given compiler (and >> > what to use in the way of a detecting that) varies from compiler to >> > compiler. The BEST you can achieve is to identify what FC_FUNC should >> be >> > identified for common cases or cases that you know of so that it >> doesn't >> > need to be re-computed with each configure/compile. >> > >> >> implementation/library or service). This would be an extra configure >> >> step in say /usr/local/itaps/headers (central location where all >> headers >> >> sit along with probably configure stuff) and this step would just >> >> generate a new header file say "itaps_fc_func.h". >> > >> > >> >> Say if one uses gfortran then the header file generated in "central" >> >> place >> >> (as an extra step using configure in "central" place) will contain >> >> following lines and every service and implementation simply use it. >> >> >> >> #ifndef _ITAPS_FC_FUNC_H_ >> >> #define _ITAPS_FC_FUNC_H_ >> >> >> >> /* following line is for specific platform, compiler... combination >> */ >> >> #define FC_FUNC(lowFunc,UPfunc) UPfunc >> >> >> >> #endif >> >> >> >> - Onkar >> >> >> >> > >> >> > The only issue I see with this approach is that you need to have >> >> defined >> >> ahead of time all the possible #ifdef #elif cases above and whenever >> you >> >> go to a new machine, then you'll have to re-visit this logic. So, I >> like >> >> the approach where Autconf's configure is used to set FC_FUNC. >> >> > >> >> > Finally, whatever FC_FUNC is set to, it MUST match the actual name >> >> mangling used by whatever fortran compiler is being used to compile >> the >> >> code. So, it is not just a matter of everyone agreeing on an >> >> > implementation for FC_FUNC. FC_FUNC must also address whatever name >> >> mangling is being used by a given fortran compiler on a given >> platform >> >> when the code is getting compiled. For that reason, I think the best >> >> approach is the Autoconf approach. >> >> > >> >> > Mark >> >> > >> >> > >> >> > -- >> >> > Mark C. Miller, Lawrence Livermore National Laboratory >> >> > email: mailto:[email protected] >> >> > (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!) >> >> > (Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!) >> >> > >> >> > >> >> >> >> >> >> >> >> >> > -- >> > Mark C. Miller, Lawrence Livermore National Laboratory >> > email: mailto:[email protected] >> > (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!) >> > (Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!) >> > >> > > -- > Mark C. Miller, Lawrence Livermore National Laboratory > email: mailto:[email protected] > (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!) > (Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!) > >

3 2

Will start to burn at 5CST
by Tim Tautges 14 Nov '08

14 Nov '08

Hi all, I'll start the dvd burning at 5 CST, which is in about 20 mins. During that time please don't modify anything under /usr/local/itaps. - tim -- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory (tautges(a)mcs.anl.gov) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706

1 0

getting ready to download the slides
by Lori A. Diachin 14 Nov '08

14 Nov '08

any objections? I may be able to download them later while on travel, but the file is rather large and I'm not going to count on it... I've included the changes made this morning to the HandsOn files and printed them out. All further changes will be on line only. Lori

2 3

FronTier tutorial text
by Ryan Kaufman 14 Nov '08

14 Nov '08

I have added a few lines to the text file to make the purpose of the example a little clearer. If anyone has time to post comments or suggestions feel free. And thanks to Karen for fixing the path names. -- ~Ryan Kaufman

1 0

Directory rearrangement on mesh?
by Tim Tautges 14 Nov '08

14 Nov '08

Hi all, How do we rearrange the disk stuff on mesh? I'm thinking we'll have subdirectories under /usr/local/itaps: Implementations Services Applications Dist Tutorial_Exercises Tutorial_Notes Anything else? Documentation? I know we discussed this but don't remember the conclusion. I'd say we/I should make those directories and let others move/install into them when they're ready. - tim -- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory (tautges(a)mcs.anl.gov) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706

4 5

Tutorial slides
by Devine, Karen D 14 Nov '08

14 Nov '08

I am going to update the tutorial slides to reflect the iMesh-Defs.inc changes to the Makefiles. So that we can avoid any conflicts, I will do the update now and let you know when I have committed my changes. Karen

3 5

Can I reboot now?
by Tim Tautges 14 Nov '08

14 Nov '08

- tim -- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory (tautges(a)mcs.anl.gov) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706

1 1

Re: itaps-parallel [Fwd: Re: problem with data]
by Tim Tautges 14 Nov '08

14 Nov '08

I don't get that behavior. Are you sure it's something wrong with the iBase.h and not the application? I've compiled all the Example applications with C using the original enum definition... - tim Mark Miller wrote: > I found this in working with FMDB but then thought it might be important > to people in general... > > > > ------------------------------------------------------------------------ > > Subject: > Re: problem with data > From: > Mark Miller <miller86(a)llnl.gov> > Date: > Fri, 14 Nov 2008 06:07:29 -0800 > To: > txie(a)scorec.rpi.edu > > To: > txie(a)scorec.rpi.edu > CC: > Onkar Sahni <osahni(a)scorec.rpi.edu> > > > enum def'n in iBase.h are somehow incompatibile with C compiler. > > I modified hello_iMeshP.c with iMeshP_getAllVtxCoords and needed > iBase_StorageOrder. However, when I tried to compile with C compiler on > mesh machine, it would fail... > > typedef enum { > iBase_BLOCKED, > iBase_INTERLEAVED, > iBase_UNDETERMINED > } iBase_StorageOrder; > > hello_iMeshP.c: In function ï¿½mainï¿½: > hello_iMeshP.c:54: error: ï¿½iBase_StorageOrderï¿½ undeclared (first use in > this function) > hello_iMeshP.c:54: error: (Each undeclared identifier is reported only > once > hello_iMeshP.c:54: error: for each function it appears in.) > hello_iMeshP.c:54: error: expected ï¿½;ï¿½ before ï¿½storageOrder2ï¿½ > hello_iMeshP.c:56: error: ï¿½storageOrder2ï¿½ undeclared (first use in this > function) > > To correct it, I think all the enums in iBase.h need to be re-formatted > like so... > > typedef enum { > iBase_BLOCKED, > iBase_INTERLEAVED, > iBase_UNDETERMINED > } iBase_StorageOrder; > > > On Fri, 2008-11-14 at 06:04, txie(a)scorec.rpi.edu wrote: >> Hi, Mark, >> >> Currently we assume that one part per process in the FMDB_iMeshP >> implementation. And imesh_load can be used in more than one process. >> >> Thanks, >> >> Ting >> >> >>> imesh_load appears to enter inf. loop if you happen to start on a number >>> of processors less than number of parts. >>> >>> Mark >>> >>> On Thu, 2008-11-13 at 22:44, Mark Miller wrote: >>>> Onkar, >>>> >>>> I am really puzzled by the data I am getting back from parallel FMDB via >>>> iMesh interface for the mesh_cube_660_4proc test case you gave me. >>>> >>>> On proc 0, the call to getallvtxcoords returns 64 coordinate values >>>> which is what I expect. However, the call to imesh_getvtxcoordindex >>>> returns 'connectivity' index offsets that are much larger than 64. >>>> I wonder if somehow I am getting back 'global' coordinate index values >>>> from this method? >>>> >>>> Mark >>>> >>>> avtITAPS_FMDBFileFormat::GetMesh >>>> Made it past call to "imesh_getentities_" at line 1244 in file >>>> avtITAPS_FMDBFileFormat.C >>>> entRegnHdls_size = 165 >>>> Made it past call to "imesh_getentities_" at line 1251 in file >>>> avtITAPS_FMDBFileFormat.C >>>> entFaceHdls_size = 379 >>>> Made it past call to "imesh_getentities_" at line 1258 in file >>>> avtITAPS_FMDBFileFormat.C >>>> entCurvHdls_size = 277 >>>> Made it past call to "imesh_getentities_" at line 1265 in file >>>> avtITAPS_FMDBFileFormat.C >>>> entVertHdls_size = 64 >>>> Made it past call to "imesh_getallvtxcoords_" at line 1274 in file >>>> avtITAPS_FMDBFileFormat.C >>>> 1, 1, 0 >>>> -1, -1, 0 >>>> 1, -1, -1 >>>> 1, 1, 0 >>>> 1, 1, 1 >>>> 1, -1, 1 >>>> 1, -0.333054, 0.661111 >>>> 1, 0.333892, 1 >>>> 0.338889, 1, 1 >>>> -0.333888, -1, 1.32222 >>>> 0.332224, -1, 1 >>>> 0.522005, -1, -1 >>>> 0.519968, -1, 0.667783 >>>> -0.636653, -1, 1 >>>> 0.00498673, -1, -0.333054 >>>> -1, -0.579238, 0 >>>> 0.580276, 1, 1 >>>> 0.476439, -0.476542, 0.333892 >>>> -0.554779, -0.558122, 0 >>>> 0.562053, 0.556558, 0.338889 >>>> 0.194421, -0.703906, 1 >>>> 0.753433, -0.744433, 0 >>>> 0.67532, -0.254162, -0.333888 >>>> -0.100118, -0.518252, -1 >>>> 0.536317, 0.0201365, 0 >>>> -0.525001, -0.0705034, 0.332224 >>>> 0.00463185, -0.0108681, -1 >>>> 1, -0.562141, 0 >>>> 1, 0.551738, 0.522005 >>>> 1, 0.553414, -1 >>>> 1, -0.46401, 0.48233 >>>> 1, 0.438439, 0.519968 >>>> 1, -0.00404706, -1 >>>> 1, -0.01364, 1.03979 >>>> -0.413533, -0.358763, -0.636653 >>>> 0.0333995, -0.382789, -1 >>>> -0.00137382, 0.0753535, 0.696981 >>>> -0.414604, 0.0216768, 0.00498673 >>>> -0.406887, -0.370834, -1 >>>> 0.0920263, -0.433486, 0.773063 >>>> 0, 0, -1 >>>> -0.472829, 0.0315931, -0.579238 >>>> -0.0108454, 0.00129541, 0.47684 >>>> 0.408, -0.408, 0.580276 >>>> 0.448358, 0.00914508, 1 >>>> 0.501977, -0.452777, 0.65203 >>>> 0.41117, -0.0437623, 0.476439 >>>> 0.408, -0.408, -0.476542 >>>> 0.432157, -0.0151216, 0 >>>> 0, 0.408, -0.554779 >>>> 0.387662, 0.41085, -0.558122 >>>> 0.422741, 0.350308, 0 >>>> 0.368157, 0.402968, 0.562053 >>>> 0.395943, 0.390392, 0.556558 >>>> 0.00183323, -0.709481, 0 >>>> -0.195426, -0.641512, 0.194421 >>>> 0.677074, -0.247432, -0.703906 >>>> 0.669767, 0.24477, 0.378452 >>>> 0.713723, 0.257543, 0.753433 >>>> 0.620712, 0.197067, -0.744433 >>>> 0.245164, 0.667283, 0.720594 >>>> 0.639625, 0.250531, 0.67532 >>>> 0.693725, 0.686982, -0.254162 >>>> 0.203988, -0.204012, 0.390573 >>>> Made it past call to "imesh_getvtxcoordindex_" at line 1334 in file >>>> avtITAPS_FMDBFileFormat.C >>>> zoneType = 10: 86, 81, 85, 84, >>>> zoneType = 10: 86, 80, 82, 81, >>>> zoneType = 10: 84, 81, 80, 86, >>>> zoneType = 10: 86, 80, 84, 83, >>>> zoneType = 10: 80, 83, 86, 82, >>>> zoneType = 10: 90, 84, 86, 85, >>>> zoneType = 10: 84, 87, 90, 86, >>>> zoneType = 10: 87, 86, 84, 83, >>>> zoneType = 10: 94, 91, 93, 85, >>>> zoneType = 10: 85, 91, 81, 94, >>>> zoneType = 10: 94, 81, 85, 86, >>>> zoneType = 10: 96, 85, 94, 93, >>>> zoneType = 10: 85, 90, 96, 94, >>>> zoneType = 10: 90, 94, 85, 86, >>>> zoneType = 10: 104, 92, 94, 86, >>>> zoneType = 10: 104, 82, 103, 92, >>>> zoneType = 10: 86, 92, 82, 104, >>>> zoneType = 10: 82, 100, 104, 103, >>>> zoneType = 10: 105, 94, 96, 90, >>>> zoneType = 10: 105, 86, 104, 94, >>>> zoneType = 10: 90, 94, 86, 105, >>>> zoneType = 10: 122, 90, 105, 96, >>>> zoneType = 10: 6, 30, 63, 46, >>>> zoneType = 10: 48, 6, 63, 46, >>>> zoneType = 10: 66, 63, 30, 80, >>>> zoneType = 10: 66, 63, 80, 68, >>>> zoneType = 10: 66, 70, 68, 80, >>>> zoneType = 10: 82, 81, 80, 70, >>>> zoneType = 10: 66, 70, 80, 81, >>>> zoneType = 10: 30, 63, 46, 80, >>>> zoneType = 10: 63, 48, 46, 80, >>>> zoneType = 10: 46, 48, 84, 80, >>>> zoneType = 10: 41, 7, 31, 62, >>>> zoneType = 10: 41, 7, 62, 71, >>>> zoneType = 10: 31, 62, 66, 91, >>>> zoneType = 10: 66, 81, 31, 91, >>>> zoneType = 10: 31, 62, 91, 41, >>>> zoneType = 10: 82, 92, 70, 103, >>>> zoneType = 10: 67, 62, 24, 91, >>>> zoneType = 10: 91, 67, 92, 24, >>>> zoneType = 10: 67, 92, 24, 78, >>>> zoneType = 10: 7, 24, 62, 71, >>>> zoneType = 10: 24, 62, 71, 91, >>>> zoneType = 10: 47, 44, 41, 93, >>>> zoneType = 10: 41, 22, 7, 71, >>>> zoneType = 10: 85, 91, 93, 41, >>>> zoneType = 10: 62, 41, 71, 91, >>>> zoneType = 10: 91, 93, 41, 71, >>>> zoneType = 10: 93, 71, 78, 75, >>>> zoneType = 10: 41, 47, 93, 85, >>>> zoneType = 10: 94, 93, 78, 96, >>>> zoneType = 10: 75, 78, 96, 77, >>>> zoneType = 10: 78, 25, 24, 67, >>>> zoneType = 10: 67, 70, 92, 103, >>>> zoneType = 10: 25, 64, 67, 103, >>>> zoneType = 10: 92, 67, 103, 25, >>>> zoneType = 10: 92, 67, 25, 78, >>>> zoneType = 10: 25, 64, 103, 73, >>>> zoneType = 10: 25, 4, 64, 73, >>>> zoneType = 10: 4, 64, 73, 60, >>>> zoneType = 10: 4, 26, 64, 60, >>>> zoneType = 10: 64, 103, 73, 60, >>>> zoneType = 10: 104, 73, 103, 60, >>>> zoneType = 10: 104, 73, 60, 76, >>>> zoneType = 10: 96, 78, 76, 77, >>>> zoneType = 10: 20, 73, 76, 60, >>>> zoneType = 10: 76, 20, 60, 21, >>>> zoneType = 10: 4, 20, 60, 73, >>>> zoneType = 10: 122, 76, 74, 77, >>>> zoneType = 10: 60, 104, 76, 105, >>>> zoneType = 10: 30, 126, 80, 46, >>>> zoneType = 10: 85, 126, 84, 81, >>>> zoneType = 10: 125, 126, 85, 81, >>>> zoneType = 10: 66, 125, 31, 81, >>>> zoneType = 10: 80, 126, 84, 46, >>> -- >>> Mark C. Miller, Lawrence Livermore National Laboratory >>> email: mailto:[email protected] >>> (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!) >>> (Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!) >>> >>> -- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory (tautges(a)mcs.anl.gov) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706

1 0