- itaps-parallel - mailman3.cels.anl.gov

Request for info
by Devine, Karen D 22 Jan '08

22 Jan '08

Where would I find descriptions of the following functions: iMesh_getNumEntSets iMesh_getEntSets They are listed in the interface and user's guide, but there is no description of their purpose or arguments. Thanks. Karen

2 2

Overloading entity-set functions
by Devine, Karen D 21 Jan '08

21 Jan '08

At bootcamp and in subsequent email, we discussed the idea of overloading the entity set handle in many ITAPS functions with a part handle. In light of our recent decision to augment the interface with Carl's 16 functionsï¿½, do you think this overloading capability is still needed? Talking points: - Carl's 16 functionsï¿½ accomplish many of the same capabilities when the entity set passed to them is the root set. If we did not allow the overloading, would we need additional functions beyond Carl's sixteen? - If we pass part handles in place of entity set handles, do we also need to pass partition handles in place of mesh instances? Or would parts "know" which partition they belong to? (I raised this question before, thinking that parts would be uniquely identified by a (partition_handle, part_handle) pair, just as entities are identified by a (mesh_instance, entity_handle) pair. But maybe the pairing isn't necessary or is too inconvenient for parts?) - Do all iMesh functions require the mesh instance as the first argument? (Seems so for the multiplexer.) Or could a substituted partition handle "know" which mesh instance to use? When creating a partition handle, we do pass it a mesh instance and a communicator, so a partition handle could "know" which mesh instance to use. Karen

2 1

Functions that need cloning for per-part queries
by Carl Ollivier-Gooch 18 Jan '08

18 Jan '08

Hello, all. First, I apologize for being slower than I'd promised in getting these out. Second, I apologize for an earlier miscount: by the time we clone getNumOfType/Topo (twice each) and the global entity query functions and both sets of iterator functions (once each), there are a total of sixteen new functions, of which the whole-partition getNumOf* funcs are already mandated elsewhere. Here they are; let the debate begin. :-) Carl -- ------------------------------------------------------------------------ Dr. Carl Ollivier-Gooch, P.Eng. Voice: +1-604-822-1854 Associate Professor Fax: +1-604-822-2403 Department of Mechanical Engineering email: cfog(a)mech.ubc.ca University of British Columbia http://www.mech.ubc.ca/~cfog Vancouver, BC V6T 1Z4 http://tetra.mech.ubc.ca/ANSLab/ ------------------------------------------------------------------------ Serial: void iMesh_getNumOfType(iMesh_Instance instance, const iBase_EntitySetHandle entity_set_handle, const int entity_type, int *num_type, int *err); void iMesh_getNumOfTopo(iMesh_Instance instance, const iBase_EntitySetHandle entity_set_handle, const int entity_topology, int *num_topo, int *err); Per part: void prefix_getNumOfType(iMesh_Instance instance, const prefix_PartHandle part_handle, const iBase_EntitySetHandle entity_set_handle, const int entity_type, int *num_type, int *err); void prefix_getNumOfTopo(iMesh_Instance instance, const prefix_PartHandle part_handle, const iBase_EntitySetHandle entity_set_handle, const int entity_topology, int *num_topo, int *err); Whole partition: void prefix_getNumOfTypePar(const prefix_PartitionHandle partition_handle, const iBase_EntitySetHandle entity_set_handle, const int entity_type, int *num_type, int *err); void prefix_getNumOfTopoPar(const prefix_PartitionHandle partition_handle, const iBase_EntitySetHandle entity_set_handle, const int entity_topology, int *num_topo, int *err); Entity queries: Serial: void iMesh_getAllVtxCoords (iMesh_Instance instance, const iBase_EntitySetHandle entity_set_handle, n double** coordinates, int* coordinates_allocated, int* coordinates_size, int** in_entity_set, int* in_entity_set_allocated, int* in_entity_set_size, int* storage_order, int *err); void iMesh_getVtxCoordIndex (iMesh_Instance instance, const iBase_EntitySetHandle entity_set_handle, const int requested_entity_type, const int requested_entity_topology, const int entity_adjacency_type, int** offset, int* offset_allocated, int* offset_size, int** index, int* index_allocated, int* index_size, int** entity_topologies, int* entity_topologies_allocated, int* entity_topologies_size, int *err); void iMesh_getEntities(iMesh_Instance instance, const iBase_EntitySetHandle entity_set_handle, const int entity_type, const int entity_topology, iBase_EntityHandle** entity_handles, int* entity_handles_allocated, int* entity_handles_size, int *err); void iMesh_getAdjEntities(iMesh_Instance instance, const iBase_EntityHandle entity_set_handle, const int entity_type_requestor, const int entity_topology_requestor, const int entity_type_requested, iBase_EntityHandle** adj_entity_handles, int* adj_entity_handles_allocated, int* adj_entity_handles_size, int** offset, int* offset_allocated, int* offset_size, int** in_entity_set, int* in_entity_set_allocated, int* in_entity_set_size, int *err); Per part: void prefix_getAllVtxCoords (iMesh_Instance instance, prefix_PartHandle part_handle, const iBase_EntitySetHandle entity_set_handle, double** coordinates, int* coordinates_allocated, int* coordinates_size, int** in_entity_set, int* in_entity_set_allocated, int* in_entity_set_size, int* storage_order, int *err); void prefix_getVtxCoordIndex (iMesh_Instance instance, prefix_PartHandle part_handle, const iBase_EntitySetHandle entity_set_handle, const int requested_entity_type, const int requested_entity_topology, const int entity_adjacency_type, int** offset, int* offset_allocated, int* offset_size, int** index, int* index_allocated, int* index_size, int** entity_topologies, int* entity_topologies_allocated, int* entity_topologies_size, int *err); void prefix_getEntities(iMesh_Instance instance, prefix_PartHandle part_handle, const iBase_EntitySetHandle entity_set_handle, const int entity_type, const int entity_topology, iBase_EntityHandle** entity_handles, int* entity_handles_allocated, int* entity_handles_size, int *err); void prefix_getAdjEntities(iMesh_Instance instance, prefix_PartHandle part_handle, const iBase_EntityHandle entity_set_handle, const int entity_type_requestor, const int entity_topology_requestor, const int entity_type_requested, iBase_EntityHandle** adj_entity_handles, int* adj_entity_handles_allocated, int* adj_entity_handles_size, int** offset, int* offset_allocated, int* offset_size, int** in_entity_set, int* in_entity_set_allocated, int* in_entity_set_size, int *err); Entity iterators: Serial: void iMesh_initEntIter(iMesh_Instance instance, const iBase_EntitySetHandle entity_set_handle, const int requested_entity_type, const int requested_entity_topology, iMesh_EntityIterator* entity_iterator, int *err); void iMesh_getNextEntIter(iMesh_Instance instance, iMesh_EntityIterator entity_iterator, iBase_EntityHandle* entity_handle, int *has_data, int *err); void iMesh_resetEntIter(iMesh_Instance instance, iMesh_EntityIterator entity_iterator, int *err); void iMesh_endEntIter(iMesh_Instance instance, iMesh_EntityIterator entity_iterator, int *err); Per part: void prefix_initEntIter(iMesh_Instance instance, prefix_PartHandle part_handle, const iBase_EntitySetHandle entity_set_handle, const int requested_entity_type, const int requested_entity_topology, iMesh_EntityIterator* entity_iterator, int *err); void prefix_getNextEntIter(iMesh_Instance instance, prefix_PartHandle part_handle, iMesh_EntityIterator entity_iterator, iBase_EntityHandle* entity_handle, int *has_data, int *err); void prefix_resetEntIter(iMesh_Instance instance, prefix_PartHandle part_handle, iMesh_EntityIterator entity_iterator, int *err); void prefix_endEntIter(iMesh_Instance instance, prefix_PartHandle part_handle, iMesh_EntityIterator entity_iterator, int *err); Array iterators Serial: void iMesh_initEntArrIter(iMesh_Instance instance, const iBase_EntitySetHandle entity_set_handle, const int requested_entity_type, const int requested_entity_topology, const int requested_array_size, iMesh_EntityArrIterator* entArr_iterator, int *err); void iMesh_getNextEntArrIter(iMesh_Instance instance, iMesh_EntityArrIterator entArr_iterator, iBase_EntityHandle** entity_handles, int* entity_handles_allocated, int* entity_handles_size, int *has_data, int *err); void iMesh_resetEntArrIter(iMesh_Instance instance, iMesh_EntityArrIterator entArr_iterator, int *err); void iMesh_endEntArrIter(iMesh_Instance instance, iMesh_EntityArrIterator entArr_iterator, int *err); Per part: void prefix_initEntArrIter(iMesh_Instance instance, prefix_PartHandle part_handle, const iBase_EntitySetHandle entity_set_handle, const int requested_entity_type, const int requested_entity_topology, const int requested_array_size, iMesh_EntityArrIterator* entArr_iterator, int *err); void prefix_getNextEntArrIter(iMesh_Instance instance, prefix_PartHandle part_handle, iMesh_EntityArrIterator entArr_iterator, iBase_EntityHandle** entity_handles, int* entity_handles_allocated, int* entity_handles_size, int *has_data, int *err); void prefix_resetEntArrIter(iMesh_Instance instance, prefix_PartHandle part_handle, iMesh_EntityArrIterator entArr_iterator, int *err); void prefix_endEntArrIter(iMesh_Instance instance, prefix_PartHandle part_handle, iMesh_EntityArrIterator entArr_iterator, int *err);

5 6

examples using itaps-parallel functionality and missing ones
by Onkar Sahni 17 Jan '08

17 Jan '08

Karen, I am forwarding the email I sent before (on 12/20/07), which talks about examples using itaps-parallel functionality and also talks about some of the missing ones. If possible we would like to spent some time on it during our ITAPS phone conf. tomorrow (01/18/08). Thanks, Onkar > > The ITAPS Parallel Interface Subcommittee will have a phone conference on > Friday, 1/18, at 1PM PST. > > Please read Carl's list of functions that would have to be modified to > include partition and part handles. Be prepared to decide whether this > list > is small enough to be accepted into the parallel interface and large > enough > to provide the needed functionality. > > Karen > > > ---------------------------- Original Message ---------------------------- Subject: examples using itaps-parallel functionality and missing ones From: "Onkar Sahni" <osahni(a)scorec.rpi.edu> Date: Thu, December 20, 2007 12:15 pm To: itaps-parallel(a)mcs.anl.gov -------------------------------------------------------------------------- We have created 4 applicable examples that demonstrate usage of basic/low-level itaps-parallel functionality (i.e., not loading of mesh but more on inter-part links etc.). I have attached the .pdf file and source .tex file too. Please review it and let us know your comments and/or questions. In this process, we found few things missing and inconsistent in our current ITAPS document. I apologize to not catch them before but the issues are minor IMO (will not cause major heartburns). All the issues are related to entity functionality, see below: 1) Missing func. on setting of remote copies: We added functionality to get remote copies of entities but do not have anything for setting them. One example, where it will be needed is in mesh modifications like split/division of elements (that breaks inter-part links which requires repair/update). 2) Missing func. on residence parts: Again, we added functionality for remote copies which indirectly tells residence parts of an entity but no functionality to explicitly (and only) get residence parts, i.e., without remote copies (more of convenience). For applications that just need to know residence parts and not remote copies of entities. And mostly we would not need functionality to set residence parts as it will be induced by setting of remote copies. 3) Inconsistency in entity functionality with part handle: In entity functionality for providing copies of entities we would need part handle too. Current document talks in terms of partition instance and entity handle. 4) Inconsistency in copies of entities: In first function under "provide information about copies of entities", we may want to rephrase last part as: "copies of the entity on parts in the partition" (currently is says "copies of the entity on processes in the partition's communicator"). Moreover, do we want to include entity itself when providing (remote-) copies. Please let us know your comments and/or questions. Thanks, Onkar

1 0

Phone conf today
by Devine, Karen D. 08 Jan '08

08 Jan '08

My calendar shows that we have a phone conference scheduled for today at 1:30pm PST. Please let me know if you expect to miss this meeting (or if you think my calendar is incorrect). There has been lots of good discussion since our last meeting. Please take time to review it before the phone conference. Below are topics for discussion. - Tim asked that we clarify the questions we are addressing. He listed the following: 1) Do we want to do booleans on query results below the interface level? An example query would be "What are all regions in some set in a particular part?" 2) How do we express and query sets spread across processors? 3) How do we refer to parts and partitions in the interface, and which functions do we use to get information about them? So far, our efforts have been addressing (3) and, more recently, (1). With respect to (3), we have agreed to provide helper-functions that expose partition and part information through the interface. These helper-functions are natural for an application to use and allow the greatest flexibility for the implementations. With respect to (1), we identified the need to answer queries such as the example Tim provided. Several options were proposed to answer these queries (see below). Tim noted that similar capability could be useful for intersections of multiple entity sets. We have not deeply explored question (2) yet, and unless there is a reason to do so immediately, I'd like to resolve (3) and (1) before we start into (2). - I'd like to try to reach agreement on a solution to question (1) above. Our original idea of overloading the mesh instance argument is insufficient for answering these queries, especially in a multiplexing environment. Carl summarized three options; I'd like to discuss the pros/cons of each. > a) Adding args to about 10 functions, for both serial and parallel. > Requires: universal agreement about what a null handle is, or else > definition of an implementation-specific global variable that apps can > use to find out what the null handle is for that impl. Also, for every > call, the function has to identify which args it was passed and dispatch > accordingly (creating more functions internal to the interface, > probably, but not user-visible). Changes to serial code. > > b) Leaving those 10 functions alone in serial, and adding parallel > versions. Requires: adding probably one function for each of the > originals in the parallel interface. My guess from the current document > on parallel is that we'll be looking at >100 functions already, so this > may or may not be considered significant. Serial code left untouched. > > c) Adding some magic to squish three args into one in parallel. > Requires: A function to squish (and unsquish, internally) the args; > therefore, extra effort for the app and impl for each of those 10 > functions every time they're called (even in serial, probably). > Depending on how the squishing is done, risk of handle collision. > Changes to serial code. - Finally, I'd like to get clarity on how decisions are made in ITAPS. When the iMesh interface was defined, for example, was unanimous agreement needed on a function before it was accepted? Or was a vote taken (as I believe CCA does)?

5 6

Re: itaps-parallel Phone conf today
by Mark Shephard 08 Jan '08

08 Jan '08

Tim Tautges wrote: > With all due respect, no, I do not intend to stop talking about the data > model. As I have spoken about many times in talks (and not nearly > enough in papers, I guess), the data model is extremely important to our > project. In fact, I believe it is as important as the API. Until tools > exchange tool-specific data in terms of the ITAPS data model we have > chosen, we will not have true interoperability. > > I would also assert that the geometric modeling tools are a good example > of the failure of API's to provide interoperability. Each of the > commercial geometric modelers has its own API, but they are not > interoperable, even after smoothing the differences between just the API > functions with common functionality. For example, one API (ACIS, or > Parasolids) tells you how to get bounding vertices from faces, so you > write your application to count on that. Then, that application does > not work on the next modeler and its API (e.g. ProToolkit), because that > modeler doesn't provide vertices in the first place. We have found the solid modelers API very robust and have worked well with them. > > Another example coming up soon is our parallel interface and the > interoperability of tools built on that interface. If that interface > were written in terms of our existing data model, and if the iMesh and > iGeom entity type-based functions were moved down to iBase, I could > write a single partitioning tool that applied to both mesh and geometry > equally well, with only a different function for determining the weights > of nodes and edges in the graph. Same goes for visualization, boundary > condition editing, and lots of other functions we both know are part of > a "real" simulation process. Incidentally, that's one of the reasons I > still advocate keeping around the SIDL interface (because that provides > polymorphism necessary to write abstract tools like that). I see no real point here. To me this is a matter of the API, not how I do or do not store my data. > > - tim > > > Mark Shephard wrote: >> >> >> In yesterdays phone call we yet again went back to the discussion on >> data model vs API. As near as I can tell iMesh, iGeom, etc. are APIï¿½s >> where there are agreed to functions that take specific information as >> input and provide specific information as answers to the question. For >> example, the one we keep discussing is give me the group of mesh faces >> that meet two conditions (one example, but only one example, of which >> is the mesh faces in a specific part and has a specific BC). This can >> be discussed at the API level quite independently of one does it with >> carrying out Booleans of sets of does something else underneath to >> provide the answer to the question. >> >> The geometric modeling tools have done this well for many years >> focused on the API end of it and no one has problems with it. >> >> Since it is quite clear that different ITAPS groups have very >> different data models for how they do thing which are actually driven >> by valid concepts, that have also been discussed many times before, >> can we please focus on the API level and quite telling each other we >> have to adopt a specific data model. >> >> Mark >> >> Devine, Karen D. wrote: >>> My calendar shows that we have a phone conference scheduled for today at >>> 1:30pm PST. Please let me know if you expect to miss this meeting >>> (or if >>> you think my calendar is incorrect). >>> >>> There has been lots of good discussion since our last meeting. >>> Please take >>> time to review it before the phone conference. >>> >>> Below are topics for discussion. >>> >>> - Tim asked that we clarify the questions we are addressing. He >>> listed the >>> following: >>> >>> 1) Do we want to do booleans on query results below the interface >>> level? >>> An example query would be "What are all regions in some set in a >>> particular part?" >>> >>> 2) How do we express and query sets spread across processors? >>> >>> 3) How do we refer to parts and partitions in the interface, and >>> which >>> functions do we use to get information about them? >>> >>> So far, our efforts have been addressing (3) and, more recently, >>> (1). With >>> respect to (3), we have agreed to provide helper-functions that expose >>> partition and part information through the interface. These >>> helper-functions are natural for an application to use and allow the >>> greatest flexibility for the implementations. With respect to (1), we >>> identified the need to answer queries such as the example Tim provided. >>> Several options were proposed to answer these queries (see below). Tim >>> noted that similar capability could be useful for intersections of >>> multiple >>> entity sets. We have not deeply explored question (2) yet, and >>> unless there >>> is a reason to do so immediately, I'd like to resolve (3) and (1) >>> before we >>> start into (2). >>> >>> - I'd like to try to reach agreement on a solution to question (1) >>> above. >>> Our original idea of overloading the mesh instance argument is >>> insufficient >>> for answering these queries, especially in a multiplexing >>> environment. Carl >>> summarized three options; I'd like to discuss the pros/cons of each. >>> >>>> a) Adding args to about 10 functions, for both serial and parallel. >>>> Requires: universal agreement about what a null handle is, or else >>>> definition of an implementation-specific global variable that apps can >>>> use to find out what the null handle is for that impl. Also, for every >>>> call, the function has to identify which args it was passed and >>>> dispatch >>>> accordingly (creating more functions internal to the interface, >>>> probably, but not user-visible). Changes to serial code. >>>> >>>> b) Leaving those 10 functions alone in serial, and adding parallel >>>> versions. Requires: adding probably one function for each of the >>>> originals in the parallel interface. My guess from the current >>>> document >>>> on parallel is that we'll be looking at >100 functions already, so this >>>> may or may not be considered significant. Serial code left untouched. >>>> >>>> c) Adding some magic to squish three args into one in parallel. >>>> Requires: A function to squish (and unsquish, internally) the args; >>>> therefore, extra effort for the app and impl for each of those 10 >>>> functions every time they're called (even in serial, probably). >>>> Depending on how the squishing is done, risk of handle collision. >>>> Changes to serial code. >>> >>> - Finally, I'd like to get clarity on how decisions are made in ITAPS. >>> When the iMesh interface was defined, for example, was unanimous >>> agreement >>> needed on a function before it was accepted? Or was a vote taken (as I >>> believe CCA does)? >> >> >

1 0

Proposal for handling queries with parts, sets, and partitions
by Carl Ollivier-Gooch 21 Dec '07

21 Dec '07

Hello, all. During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day. Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions. In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance". In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...). I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way. Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant. Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance: / partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle The other iMesh function that is obviously of this type is getNumOfTopo. A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities. My guess is that iterator functions will fall into the second category. Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again. Now, there are a couple of beneath-the-hood requirements for implementations here: 1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID. 2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration. I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array). Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult. Carl -- ------------------------------------------------------------------------ Dr. Carl Ollivier-Gooch, P.Eng. Voice: +1-604-822-1854 Associate Professor Fax: +1-604-822-2403 Department of Mechanical Engineering email: cfog(a)mech.ubc.ca University of British Columbia http://www.mech.ubc.ca/~cfog Vancouver, BC V6T 1Z4 http://tetra.mech.ubc.ca/ANSLab/ ------------------------------------------------------------------------

8 33

Diagram of set-based parallel data
by Tim Tautges 20 Dec '07

20 Dec '07

Hi all, As promised, attached is a cartoon describing set-based parallel data. Note I've stuck to the simplest example (as wisely advocated by Mark S). I show the representation of the same mesh in the serial and 3-processor cases. I don't show ghosted elements, for simplicity, though those "fit" in the model. PARALLEL In the parallel representation, each processor has a set it understands as "partition1" (labeled "Partn1"). In this example, on each processor, that set contains one set corresponding to the part assigned to that processor. The part set contains 3d entities assigned to that processor. In this example, all 3d entities are objects in the decomposition, therefore each 3d entity is a member of exactly one Part. There are other entity sets defined in this mesh, including a boundary condition set (BC1), material sets (Matl1, Matl2), and application-defined sets (App1-App3). In this example, the BC1 set contains faces in the mesh (2d elements), but not all of them. Matl1 and Matl2 are meant to form a covering over 3d elements, defining the materials for those elements. Application-defined sets are for arbitrary groupings. Whether such "shared" sets would exist on processors even if there were no set contents on those processors is currently implementation-defined, though maybe there should be a common convention for that. Some of the sets on each processor are implicitly linked with sets on other processors. For example, the Partn1 set on each processor is linked to the corresponding sets on other procs. For this set, that means that if the mesh were ever to be assembled in a single file, the contents of Partn1 on various processors would all go into a single set, which would be assigned the union of all tags on that set on all processors. Presumably those tags would all be the same. Similarly, BC1 and Matl1, Matl2 are sets that are implicitly linked and which would be combined. App1-App3 are application-defined sets which would be considered separate, even if the mesh was written to a single file. Note that the decision or designation of which sets across the processors are really the same sets is application- or implementation-defined. In MOAB, we do it based on tag and optionally tag value. So, for example, any set with a "MATERIAL_SET" tag whose value is the same is understood to be the same set. This is used in the HDF5-based file writer in MOAB, which writes a single file in parallel from all processors. SERIAL In the serial case, all sets having implicitly-defined counterparts on other processors have been combined. Thus, part sets (Part1-Part3) are contained in the Partn1 set, and there are single BC1, Matl1 and Matl2 sets. App1-App3 are still separate, since they were not designated to be the same sets. The Part1-Part3 sets and Matl1-Matl2 sets are different coverings of all 3d entities in this mesh for this example. The BC1 set includes a subset of 2d entities in the mesh. DISCUSSION As I've mentioned before, I've implemented MOAB's parallel reader such that any collection of sets which forms a covering over the things I want to partition can be used to define the partition. Thus, in this example, one could use Matl1/Matl2 as the part sets. In this case, those sets would serve multiple purposes. Similarly, I don't see any reason we shouldn't allow a given part set to be a member of multiple partitions. Part sets should also be allowed to include sets; in this case, maybe the sets are the objects in the partition. This would be a very nice thing to use for e.g. hypergraph partitioning methods. Counter-intuitively, I would lean toward using contains relations to go down that hypergraph hierarchy. If a given implementation did not implement Parts as legitimate entity sets, then this property could be used to reduce bookkeeping for adaptive applications, e.g. have the part contain sets which themselves contain 3d elements of a given material or geometric volume. Of course, that assumes you're using sets for those other things... This method for representing partitions also has the following benefits: - allows multiple partitions to exist in the same file/instance/database - would allow current set operations to be used to query contents of parts and partitions - parallel view is only different when asking about parallel data; in other cases it looks the same as serial interface - other parts of RPI partition model can be embedded similarly, by using sets related to part sets using parent/child relations I'll be elaborating on this at the SIAM mtg in March, as well as pulling in other examples where sets can be used to do various things. I guess that means I've already started my talk, which has to be some sort of record for me! Comments welcome, fire when ready! - tim -- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory (tautges(a)mcs.anl.gov) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706

3 10

Re: [Fwd: Re: itaps-parallel Today's phone conf]
by Onkar Sahni 20 Dec '07

20 Dec '07

I would like to see feasibility of option (d), which was original proposal by Carl of overloading first arg. of iMesh instance with either part or partition instances, note considering parts and partitions as instances not handles (I do not know all differences between "instance" and "handle" under ITAPS terminology except that a handle can be id- or index- based.). If option (d) is out, then I am leaning towards (b) too. - Onkar > At this point, we're looking at a comparison between: > > a) Adding args to about 10 functions, for both serial and parallel. > Requires: universal agreement about what a null handle is, or else > definition of an implementation-specific global variable that apps can > use to find out what the null handle is for that impl. Also, for every > call, the function has to identify which args it was passed and dispatch > accordingly (creating more functions internal to the interface, > probably, but not user-visible). Changes to serial code. > > b) Leaving those 10 functions alone in serial, and adding parallel > versions. Requires: adding probably one function for each of the > originals in the parallel interface. My guess from the current document > on parallel is that we'll be looking at >100 functions already, so this > may or may not be considered significant. Serial code left untouched. > > c) Adding some magic to squish three args into one in parallel. > Requires: A function to squish (and unsquish, internally) the args; > therefore, extra effort for the app and impl for each of those 10 > functions every time they're called (even in serial, probably). > Depending on how the squishing is done, risk of handle collision. > Changes to serial code. > > Personally, I think I'd lean towards b, given these three alternatives, > provided of course that my count of the number of functions involved > holds up. > > Carl > > -- > ------------------------------------------------------------------------ > Dr. Carl Ollivier-Gooch, P.Eng. Voice: +1-604-822-1854 > Associate Professor Fax: +1-604-822-2403 > Department of Mechanical Engineering email: cfog(a)mech.ubc.ca > University of British Columbia http://www.mech.ubc.ca/~cfog > Vancouver, BC V6T 1Z4 http://tetra.mech.ubc.ca/ANSLab/ > ------------------------------------------------------------------------ > >

2 1

Today's phone conf
by Devine, Karen D. 20 Dec '07

20 Dec '07

Here are the notes from today's phone conference. - We discussed Carl's proposal to overload the mesh instance argument with partition instances and/or part handles to perform set operations. Jason expressed concerns that the overloading would make implementing the multiplexer difficult (at best), and would complicate the implementations as well. - Lori proposed adding partition information to the argument lists of the existing iMesh set-based functions. We agreed that we could change the iMesh interface, adding partition instance and part handle arguments to needed functions. Serial implementations could pass NULL for these arguments. - Karen will fold this idea and other proposed syntax into the combined document for review before the next phone conference. - Mark M. discussed his code that performs many parallel ITAPS functions using the serial interface. He can use this code to identify functionality that is missing or extraneous in the combined document. Mark M. commented that the parallel interface is not yet concrete enough for pseudo-code applications. - Lori discussed the agenda for the March 11 bootcamp. This bootcamp will be our chance to describe the parallel interface to our other ITAPS team members, so we should have something concrete to present and, perhaps, some pseudo-code showing how to use it. The bootcamp will also include a discussion of Carl's paper and a talk by Ahmed on ORNL activities. - The proposed time for the next phone conference is Jan 7 at 1:30 PST. Let me know if that time doesn't work for you.

3 2